Equivariant Denoisers Cannot Copy Graphs: Align Your Graph Diffusion Models
Authors: Najwa Laabid
Severi Rissanen
Markus Heinonen
Arno Solin
Vikas Garg
Venue: ICLR 2025 Poster (2025)
Keywords: Graph Diffusion
Discrete Diffusion
Equivariance
Symmetries
Primary area: learning on graphs and other geometries & topologies
TL;DR: We propose aligned permutation equivariance in order to break the symmetries of equivariant denoisers in graph diffusion.

Abstract: Graph diffusion models, dominant in graph generative modeling, remain underexplored for graph-to-graph translation tasks like chemical reaction prediction. We demonstrate that standard permutation equivariant denoisers face fundamental limitations in these tasks due to their inability to break symmetries in noisy inputs. To address this, we propose \emph{aligning} input and target graphs to break input symmetries while preserving permutation equivariance in non-matching graph portions. Using retrosynthesis (i.e., the task of predicting precursors for synthesis of a given target molecule) as our application domain, we show how alignment dramatically improves discrete diffusion model performance from $5$\% to a SOTA-matching $54.7$\% top-1 accuracy. Code is available at https://github.com/Aalto-QuML/DiffAlign.

Published as a conference paper at ICLR 2025
EQUIVARIANT DENOISERS CANNOT COPY GRAPHS: ALIGN YOUR GRAPH DIFFUSION MODELS
Najwa " Rissanen's, Markus Heinonen's Arno Solin', Vikas Garg Department of Computer Science, Aalto University
Yai Ltd jwa. laabid, severi. rissaren, markus. o. heinonen faalto. fi,
arno. arno.solinealloo. fi, vgarg@csail edi
ABSTRACT
Graph diffusion models, dominant in graph generative modeling, remain underex- plored for graph-to-graph translation tasks like chemical reaction prediction. We demonstrate that standard permutation equivariant denoisers face fundamental limitations in these tasks due to their inability to break symmetries in noisy inputs. To address this, propose aligning input and target graphs to break input symmetries while preserving permutation equivariance in non-matching graph portions. Using retrosynthesis (i.e., the task of predicting precursors for synthesis of a given target molecule) as our application domain, we show how alignment dramatically improves discrete diffusion model performance from 5% to SOTA-matching 54.7% top-1 accuracy. Code is available at https: //tt:///gttttbb/ com/
com/Aalto Aallto-oooooooooaaae Satttto-M/DDDifAAiiig
INTRODUCTION
Aligning didun and sumple using matching node identifiers
Graphs appear ubiquitously across domains from knowledge representation to drug discovery (In- graham et al., 2019; Ji et al., 2022; Hoogeboom al., 2022b; Verma et al., 2023). Graph-to-graph translation serves numerous applications: molec- ular editing (Jin et al., 2019b), where the goal is generating molecular graphs with specific structures or properties; chemical reaction prediction (Shi et al., 2020b), which requires predicting reactant graphs from product graphs and vice-versa; and any task predicting future graph states from initial configurations (Rossi et al., 2020). Generative models offer natural, probabilistic approach to graph-to-graph translation. Graph diffusion models present a particularly promising framework given their excellent performance on graph data (Niu et al., 2020: Vignac al., 2023; Mercatali al., 2024) and the widespread success of diffusion models in generative modeling in general.
imput Aligned permutation equivariance
target
Figure 1: Nodes in the input and output graphs given the same identifiers to en- force alignment, while the model is free to remain permutation equivariant w.r.t to the unmatched nodes.
Permutation equivariance forms fundamental inductive bias graph- machine learning mod- els, including graph diffusion models. This property ensures consistent model outputs when input nodes are reordered relabeled, eliminating the need for data augmentation with different node per- mutations. While powerful, permutation equivariance faces fundamental limitations when mapping highly symmetric inputs to less symmetric outputs. We identify the root cause of this issue: when trying to satisfy equivariance with symmetric input, optimally -trineed neural network predict identical distributions for all elements essentially the marginal distribution of labels in the training data (see Theorem 1). Our findings align with previous and concurrent research highlighting
only
"qqual contribution

---

Published as a conference paper at ICLR 2025
Sec. 3.1 The Theoretical Limitations of Equivariant Denoisers Copying graphs as a case-study on the limitations of equivariance. The optimal permutation equivariant denoiser.
Fig. Theorem 1
Sec. 3.2 Solution: Aligned Equivariant Denoisers
Relaxing permutation equivariance through alignment. Aligned denoisers induce aligned permutation invariant distributions.
Fig. Theorem 2
Sec. 3.3 Suitable Alignment Methods Skip-connection, positional encodings, input-alignment.
Fig. A9
Sec. 4 Experiments
Proof concept through simple graph copying tasks. Application: Retrosynthesis. sonded-generation for retrosynthesis. Ibuprofen synthesis through inpainting.
Fig. Table 2
Fig. 8a
Fig. 7
Figure 2: Overview of our contributions.
the limitations imposed by equivariance and symmetries in neural network (Lawrence et al., 2024; Xie & Smidt, 2024).
The key question becomes: how can we enable symmetry breaking while preserving the benefits of equivariance? We propose aligned equivariance our solution, visualized in Fig. 1. This approach uses node identifiers in both input and target graphs to align corresponding nodes through vari- methods (see Sec. 3.3), relaxing equivariance constraints when identifiers match. This targeted symmetry breaking creates structural anchors that guide the generation process while maintaining permutation equivariance in non-matchin non-matching subgraphs. Our experimental results shows that a com- bination of our alignment methods achieves SOTA-matching results on retrosynthesis, the task of predicting precursor molecules for given target. We summarize our contributions in Fig. 2.
2
PRELIMINARIES
2.1 GRAPH-TO-GRAPH TRANSLATION
Consider a database of Nots graphs D = ((Xn, )X where X,, represents the tar- get graph, Y, the input graph, and PY are matrices defining node mappings between the two graphs. The graph translation task is: given that the data is sampled from an unknown distribution p(X, Y. PY-X), predict valid targets X pXX| Y) for given input Y.
We start by formally defining the graph objects X and To do so, we consider the space one-hot vectors of dimension K as vert( AK-1), where vert(.) denotes the vertices of the probability simplex AK-1 with K degrees freedom. We then define the target X as tuple (X-V X of node feature matrix X-V E (vert(A A ( and edge feature matrix X (vert( A K. J
NxxNxx xNx
where K2 and Kú, are node and edge feature dimensions respectively. The input graph Y is similarly defined as (Y-V, F) with the same node and edge features, but potentially different number of nodes Ny. The features consist of node labels (with K, possible values) and edge labels (with K₁ possible values), each including empty value 1 to represent missing nodes edges.
Node mapping matrices establish correspondence between nodes in the input and target graphs. Formally, define such matrices as: PY-X (0, Nxx1N.xx.N with P 1 if the 7th node of the target is the atom-mapped counterpart the 7"h node the input, and zero otherwise. This node mapping plays crucial role in breaking the inherent symmetries present in graph diffusion models. Without this correspondence information, a model cannot distinguish which specific nodes in the input should map which specific nodes in the output, leading averaged, symmetrical predictions that fail to capture the structural relationships between input and target graphs, as we discuss later.
2

---

Published as a conference paper at ICLR 2025
Target (reactunts)
(product)
(Xx,
P ()
XX Pol
V.P.
X
GNM
GNN
X
GNM
XT
Allgo
Allon
Figure 3: An overview a graph translation task using discrete diffusion models, illustrated through chemical reactions. We adopt absorbing state diffusion (Austin et al., 2021), such that samples from the stationary distribution are made nodes and edges with the 'none' label. The condition and sample graphs aligned using node mapping information (highlighted with the matching colours) which identifies pairs of matched nodes across the input and target graphs.
The product pY-XY.v equals X with the non-mapped nodes zeroed out. Similarly, P equals X with edges to non-mapped nodes zeroed out. We expect in- ductive bias such that if an edge exists between two mapped nodes in X. there is a high probability of edge between their corresponding nodes in as well. This reflects the fundamental intuition that graphs are structurally similar for their node-mapped portions.
2.2 DISCRETE DIFFUSION MODELS FOR CONDITIONAL. GRAPH CENERATION
We follow the framework of Vignac et al. (2023), which adapts discrete diffusion models (Austin et al., 2021) to graphs. We present the main constituents of the framework. We assume Markov forward process
(4+1 X,) = I
x
II
x )
(1)
to diffuse the reactant to noise, and a reverse process
Po (X, X, Y) = - pa X V. X. III (X E.yy X,, Y):
(2)
defining our generative model. Here, 0 represents neural network parameters. Note that we always have time conditioning implicitly in po (X, X, Y, (). but we drop / for notational convenience. We will also condition on the node mapping Y Sec. 3.3, but we will not include in the no-
tation until then. The full generative distribution is pa X Y = p(X)) II Pa (X X., Y), where p(XT) is a predefined prior such that p (Xr) = q XT X Following Hoogeboom et al. (2021) and Austin et al. (2021), we use the neural network specifically to predict ground truth labels from noised samples, meaning that the neural network outputs a distribution po (Xo Xt Y).
The reverse process is then parameterized by
(X Xr, Y) Ex, q( (X,-1 X Xo) (Xo (Xo X,, Y).
(3)
Both (X,-1 X1, Xo) and po (No X,, Y) factorize over dimensions:
Nx
Nx
q (X,-1 Xo) II
g
Eddd
x
Bdid x Bidd)
(4)
Nx
Nx
Po ( X Y
II io
X2 Y)
II
(x
io
E.j. X,, Y)
(5)
1.
We write the connection to Eq. (2) explicitly in App. B. Throughout the paper, we denote the direct output of the neural network as Do (X,, Y) (D, Y) (X, ") where Dy(X, Y)W i.e., have probability vector for each (A <x.-I and (x(,, Y) (A (AKo-I)(NXXN,
node and edge. This implies pa (Xol X., Y) is distribution factorized over nodes and edges.
The single-step transition for nodes (and similarly for edges) is defined with transition matrix Q. Ni x
(X.N
Cat(X,
3

---

Published as a conference paper at ICLR 2025
Algorithm Loss calculation
Algorithm 2 Sampling
Input: condition Y. target Xu, and optional per- mutation matrix P px-Y for alignment
Input: condition Y Choose (for alignment): P py- R.XxxN Xr p(X,)
t ~~ Uniform (40......
X, (X, Xo)
for t 7 do
X. Dx(X, Y, Y,PY-X)
X. = (x, Y. PY-X
Return Cross-Entroe Mooss-Ettropy(X... Xn)
x Ex 9( X X
Return Xo
In our experiments, we use the absorbing-state formulation from Austin et al. (2021), where nodes and edges gradually transfer to the absorbing state, defined as the empty state 1. Formally, Q: = (1 B,)l + 3 B. where 3r defines the diffusion schedule and tn is one-hot on the absorbing state. Then, the marginal q(x, Xx)) and conditional posterior qX,,1 Xu) also have a closed form for the absorbing state transitions. The prior p(Xx) is correspondingly chosen to be delta distribution at a graph with no edges and nodes set to the state. The noise schedule 3g is defined using the mutual information criterion proposed in Austin al. (2021). While other transitions, like uniform and marginal (Vignac et al., 2023) are equally possible, the absorbing state model is simpler and empirically outperforms others (Austin et al., 2021; Lou et al., 2024), motivating our choice.
We use the cross-entropy loss, as discussed in Austin et al. (2021) and Vignac et al. (2023):
-Eq(Xo. ()(t))(X) xx) [log po Xo X₁, Y)).
where (() is a uniform distribution over / E (1 (1...7) 7)-
Fig. 3 provides overview of the conditional graph diffusion framework use. For completeness, a comprehensive definition of the model's is components for nodes and edges is given in App. B. The training and sampling procedures with graph diffusion models are presented in Alg. I and Alg. 2, along with optional conditioning on P Y -x as described in Sec. 3.3.
Permutation Equivariance and Invariance Permutation equivariance for the denoiser is defined as D& (PX) = (X) where P is an arbitrary permutation matrix, and PX = (PX* P This makes single-step reverse transitions equivariant as well, in the sense that (X X,) po pPX, --1 PX;). For our conditional setting, permutation equivariance can be written as D Y)= (X, Y). The prior in graph diffusion models also usually permutation in-
variant s.t. p (PXT) p(XT) (Niu et al., 2020; Vignac et al., 2023; Hoogeboom et al., 2022b).
2.3 RELATED WORK
Graph-to-graph translation methods Graph-to-graph models have demonstrated success in di- verse tasks including handwritten mathematical expression recognition (Wu et al., 2021), molecular optimization (Jin et al., 2019a), and retrosynthesis (Lin et al., 2023). Recently, Igashov et al. (2024) introduced Markov bridge model for proaucctto-reactant graph mapping, representing the closest approach to diffusion-based graph-to-graph translation. While their paper does not explicitly ad- design elements, the authors implement technique
dress equivariance and symmetry-breaking similar to our "input alignment" described in Sec. 3.3. This suggests our theoretical framework may also explain their model's effectiveness.
Equivariance and mommetry-breaking Equivariant neural networks have attracted interest for their ability incorporate known symmetries, typically enhancing generalization. However, Smidt et al. (2021) identified fundamental limitation: these networks struggle with self-symmetric inputs because they cannot break inherent data symmetries. This challenge appears across various applica- tions, including prediction tasks on symmetric domains and generative models reconstructing from highly symmetric latent spaces (I (Lawrence et al., 2024). Researchers have explored this issue dif- ferent contexts, including graph representation learning (Srinivasan & Ribeiro, 2020), set generation (Zhang et al., 2022), and physical system modeling (Kaba et al., 2023).
Permutation equivariance in graph diffusion Equivariance features prominently in graph diffu- sion models to parameterize the reverse process (Niu et al., 2020; Vignac et al., 2023; Hoogeboom et al., 2022b; Huang al., 2022). One key motivation is that permutation equivariant neural net- works induce permutation invariant distributions in diffusion models (Niu et al., 2020), ensuring dif- ferent permutations of the same graph have identical probabilities under the model. In recent work,
4

---

Published as a conference paper at ICLR 2025
Yan et al. (2023) demonstrated that relaxing permutation equivariance in graph diffusion through absolute positional encodings empirically improves performance.
3 WHY PERMUTATION EQUIVARIANCE FAILS AND HOW TO FIX IT
3.1 PERMUTATION EQUIVARIANT DENOISERS CANNOT LEARN THE IDENTITY FUNCTION
In this section, consider data set D {(x,, Y,, ...PPY-)) Nale dubbed the 'identity data', where for all data points X,, PY-+Xy,, In other words, both the input and the target are equivalent, up to some permutation, as defined by the node-mapping matrix P PY-X This seem- ingly simple scenario reveals a fundamental limitation of standard graph diffusion models. Graph translation can be viewed as similar to copying graphs (learning the identity function) because of the structural similarity between input and target graphs. Yet, we demonstrate that standard graph diffusion models struggle with this basic task due to their inability to break symmetries.
A denoiser implementing the identity function should output DX, Y) PY-XY, placing all probability mass on the correct node/edge labels (up to a permutation of Y). We pass the condi- tioning graph Y by concatenating to the input X along the node dimension, creating a graph with Nx + Ny nodes and (Nx + Ny) x (Nx + Ny) edges, with no edges between the X and Y sub- graphs. However, permutation equivariant denoisers Dg are constrained output identical "mean" solutions for all nodes and edges in the early stages of generation.
To understand this intuitively, consider a highly noisy input at T and a permutation equivariant denoiser. Permutation equivariance requires that D&RRX, Y) = RDDo (Xr. Y) for any permuta- tion matrix R E {0, 1] Mr Since XT contains no information about Y (due to high noise), the model learns to ignore the X input, leading to (RXT,, Y) DD(XT, Y) PY-XY. This creates a contradiction: RPY-XY
PY-XX Y, which cannot generally be true. The only way to satisfy both equivariance (RX, Y)- RDo Y)) and input-independence (D,(RXy, Y) (Xy, Y)) is if Dx Y) outputs identical probability vectors for each node and edge. This symmetry problem prevents the model from distinguishing between nodes that should have different labels. The following theorem formalizes this limitation:
Theorem 1. (The optimal permutation equivariant denoiser) Let Do(XT, Y) be permutation equivariant s.I. PP Y) = PDo(X, Y), and let q(XT) be permutation invariant. The optimal solution with respect to the cross-entrops loss with the identity data is. for all nodes i and j
Do YOİ . = E. Yu. Yi,ks Dx Y) = F - / Yijkk,
where Sh and E are the marginal distributions of node and edge values in Y.
We visualize the theorem in Fig. 4, and give the proof in App. A.2, with an extension to other time steps in App. A.7. The symmetry constraint makes it impossible for the model to solve the task in single step with T = 1. With multiple smaller steps, the model can gradually break symmetries as sampling in the discrete generative process introduces slight asymmetries in X,, enabling correlations between node and edge representations to emerge. Each denoising step can leverage these small asymmetries from previous steps, slowly building node identity information. any-order
As T with absorbing diffusion, the model eventually becomes equivalent autoregressive model (Hoogeboom et al., 2022a), thus exemplifying gradual symmetry-breaking by changing one element at time. However, this process remains inefficient compared to approaches that directly address the symmetry problem.
3.2 SOLUTION: ALIGNED PERMUTATION EQUIVARIANCE
Given the failure of permutation equivariant denoisers to effectively copy graphs, we propose to relax the permutation equivariance constraint in a controlled way. We observe that it is sufficient to have permutation equivariance in the sense that if we permute X and/or Y, and accordingly permute the node mapping PY PY-X such that the matching berween Y and X remains, the model output should be the same. We call this aligned permutation equivariance. Formally, we use the node- mode-mapping permutation matrix pY-X as an input the denoiser and consider denoisers that
5

---

Published as a conference paper at ICLR 2025
satisfy the following constraint: DR QY, RPY-XQ RD Y, YYPY))), where R and Q are permutation matrices of shapes (Nx x Nx) and (Ny x Ny), respectively.
With PY-X and Y input, an unconstrained denoiser can output the ground-truth permutation pY-XY for the identity data task. Restricting the function to the aligned equivariant class does not clash with this, as can be seen by writing out the optimal solution and the equivariance condition for a permuted input R.X, QY, RPY-XQXX
(Identity data)
D& QY, RPY RPX-XQT) R QY = DRRYP-X- (9) DR QY, RPY RPP-XXQ] R., Y,Px-X) RPX-XXX
(Aligned Equivariance)
(10)
Note that the permutation invariance of (Xo) does not apply as it does for fully permutation cquivariant models. However, we show that a generalized form of distribution invariance holds.
Theorem 2. (Aligned denoisers induce aligned permutation invariant distributions) If the de- noiser function Do has the aligned equivariance property and the prior p(Xr) ) is permutation in- variant, then the generative distribution po Po(Xo Y. Y.PY -Y has the corresponding property for any permutation matrices R and Q:
po (RXo QY, RRX-XXXXXX = po (X. Y,PX-X).
(11)
proof is given in App. A.4, while Figure showcases the denoising output of an aligned equivari- ant model with given PY-X Informally, the theorem states that the graph pairing has the same probability for all isomorphisms of the input and target graphs, as long as the node mapping is not reassigned to different nodes. This means that during training, in the data and be confident that the model generalizes to other permutations. During sampling, only have access to Y without node mapping information. The theorem ensures we can assign node mappings arbitrarily and still obtain effectively the same distribution over the targets. We show this mathematically in App. A.4. Lastly, we note that use of node mapping here is a constraint: it is merely an additional input which allows the denoiser to use the structure of the conditional graph efficiently, and as such benefit even from partially correct node mapping.
use the permutations present
Reactant
Product
3.3 METHODS FOR ALIGNMENT IN GNNS
We have established how constraining permutation equiv- ariance to aligned permutation equivariance is a key com- ponent for the success of the denoising model. In this sec- tion, we discuss multiple methods to induce alignment between the input and target graphs. We visualize these methods in Fig. A9 and prove that each belongs to the class of aligned equivariant models in App. A.5.
N
C CON
CON
0.25
0 0
0
Perm. Eq. GNN
0
0
0 1
1 0
10 0
0 0
1
00 1
D
CON 100 D 010
CONL
DACL 0003
Node-mapped positional encodings Graph positional encodings are a standard tool to break permutation sym- metry in GNNs, making them natural candidate for in- ducing aligned permutation equivariance. We consider uniquely identifying pairs of nodes matched via the node- mapping matrix in both X, and Y by adding a posi- tional encoding vector to each unique node pair. In prac- tice, we generate set of distinct vectors & E Ry xd such that yp = y(Y) for each of the input graph nodes Y. with y function generating the encodings based on Y. In this work, we use Laplacian positional en- codings (Dwivedi et al., 2023), but in principle any po- sitional encoding scheme is applicable. We then map the vectors ø to the corresponding inputs for the noisy graph nodes X,. Formally, we get Do (X,, Y, PY-+X)
Aligned GNN
100 001
(X.... C.X..YP'.. X.X.Y.YYYXX.... -XW
Figure 4: Comparing the optimal per- mutation equivariant denoiser to an aligned denoiser. As per Theorem I, the permutation equivariant model (top) outputs the marginal distribution over node types whereas the aligned model (bottom) reconstructs the correct reac- tant nodes in a specific permutation -
[o[[x"' pY-X X.F Y sp), Y²), where So is the neural network that takes as inputs the aug- mented node features and regular edge features and [XW pY-X xp) E R +dp) corresponds

---

Published as a conference paper at ICLR 2025
to concatenation along the feature dimension. The only change required for the neural network is to increase the initial linear layer size for the node inputs.
Direct skip connection Next, motivated by the identity function analysis in Sec. 3.1, propose an alignment method that solves the identity task in a minimal way. In particular, modify the network include direct connection from the condition to the target output: (X,, Y, YYPY at the last layer of the neural network for the nodes and edges, À is learnable parameter and The sum is possible because Y is in one-hot format and the dimensionalities of the denoiser output and PY-Xy are the same. It is easy to see that when dominates in the softmax.
softmax far (X,, Y) + XPY where f (X,, Y) are the logits
pY-XY=
Doy-Xyyl pY-PP--X y (PY-x))
c0, (x, Y, T,PY-X) pY-Xy because the direct connection from Y
Aligning Y in the input As a natural next step from aligning the graphs at the output via skip connections, we propose to align them in the input. This allows more expressivity from the network, since it can process the aligned graphs in the layers of the network before out- putting the denoising solution. In practice, we align Y by concatenating X and pY-*X along the feature dimension before passing it to the neural network. Thus, we can drop out the ex- plicit Y graph entirely, and only process the X graph augmented with Y: DX Y, Y.PY+X) fo([X PY-Xy-1 [XP P (PYxxx(PPX)))) (I)- where [-] means concatenation along the feature dimension.
3.4 ADDING POST-TRAINING CONDITIONING FOR DISCRETE DIFFUSION MODELS
To fully realize the potential of diffusion models for graph-to-graph translation, derive novel reconstruction guidance-like method for discrete models (Ho et al., 2022; Chung et al., 2023: Song al., 2023a). This approach differs from the classifier-uuidance used by Vignac et al. (2023) in their conditional generation. We use the denoiser output to evaluate the likelihood p(y Xx) of the condition y (e-g.. probability of synthesizability) given the reactants Xp. and backpropagate to get an adjustment of the backward update step:
log Pa (X.11 Xr, Y, x Vx, log(Ep, xyp(y Xo)) + log P X,, Y).
(12)
The derivation starts with similar steps as the classifier guidance method in Vignac et al. (2023). We provide more details in App. F. The method can be implemented with a few lines of code and works as long as ply Xo) is differentiable. For the retrosynthesis application, we experiment with toy model of synthesizability based on the count of atoms in the reaction (Ertl & Schuffenhauer, 2009),
a concept also known as atom economy (Trost, 1991).
EXPERIMENTS
We assess the effect of aligned denoisers on the performance of regular discrete diffusion models. To this effect, we first demonstrate our model on a toy example: copying simple graphs We then evaluate our method more rigorously on the real-world task of retrosynthesis, the task of defining precursors for given compound. We also show how diffusion enables number of downstream tasks within retrosynthesis, including inpainting and property-guides generation.
Original
Aligned
Equivariant
4.1 COPYING GRAPHS
We evaluate the model's ability to copy a graph as it is. To do so, we define a simple dataset made x grids. We define an aligned and an equivariant denoiser, both with the same architecture: a 2-layer GNN with 16 nodes the hidden dimension. We train both models on the same 100 samples with a batch size of 32 for 10 epochs. Fig- ure 5 shows that our aligned denoiser recovers the original graph almost perfectly, while a fully equivariant denoiser outputs only a fraction of the same components.
Figure 5: Showcasing performance on graph copying of grids.
7

---

Published as a conference paper at ICLR 2025
4.2 RETROSYNTHESIS
Retrosynthesis is crucial step in the drug discovery pipeline, it provides a concrete plan to create an identified compound. Single-step retrosynthesis in particular seeks to define the reactants in single chemical reaction, which can then be chained to create more comprchensive synthesis plan. The task lends itself naturally to graph translation models, since it seeks to generate a target graph (reactants) given input graph (product). In this experiment, compare the performance of the aligned denoiser to that of an equivariant denoiser using real-world chemical reactions.
Experimental setup We use the benchmark dataset USPTO-50k for experiments. The dataset consists of 50000 chemical reactions, in SMILES format (Weininger, 1988), curated by Schnei- der et al. (2016) from an original 2 million reactions extracted through text mining by Lowe (2012). More information on the benchmark dataset USPTO and its various subsets can be found in App. C.2. We use the graph transformer architecture introduced by Dwivedi & Bresson (2021) and used by Vignac et al. (2023). The network architecture and hyperparameters are detailed in App. C.6. For each product, we generate 100 reactant set, we deduplicate them and rank the unique reactants from most-to- leatst likely, as judged by the model, using the likelihood lower bound and du- plicate counts as a proxy. Based on the types alignment discussed in Sec. 3.3, we experiment with (1) a permutation equivariant model (Unaligned), (2) model augmented with atom-mapped po- sitional encodings, calculated from the graph Laplacian eigendecomposition (DiffAlign-PE)) (3) A model with Laplacian positional encodings as well as the skip connection (DiffAlign-PE+stl model where X and Y are aligned at the input as well as in the output (DiffAliig--in) iippgnn--iuut alignment). We use T 100. We trained the models for 400-600 epochs and chose the best checkpoint based on the Mean Reciprocal Rank (MRR, (Liu et al., 2009)) score with = 10 of the validation set. For more details on our experimental setup, see App. C.6.
(4) A
Evaluation We follow previous practices and use the accuracy of obtaining the ground-truth reac- tants as our main metric. This is measured by top-k accuracy, which counts the number of ground- truth matches in the deduplicated and ranked samples among the top-k generated reactions. We also report Mean Reciprocal Rank (MRR, (Liu et al., 2009)) as used by Maziarz et (2023). We focus here on a comparison between aligned and equivariant denoisers. In order to place our models in the context of the available literature, we also compare to existing baselines in App.
Results We visualize the aligned denoiser and the permutation equivariant denoiser in Fig. 6a. As predicted by Theorem 1, a sample from the permutation equivariant denoising distribution at high levels of noise pxXoo XT, Y) has no information about the structure of the product. This is direct consequence of the symmetry limitation: without alignment, the equivariant denoiser cannot distinguish between different possible node mappings, producing identical outputs for all nodes and effectively averaging across all possible configurations. In contrast, an aligned denoiser is able to copy the product structure, and the initial denoising output is of much higher quality.
Table 1: Top-k accuracy and MRR on the USPTO-50k test data set. Aligned models outperform the unaligned one with the combined PE+skip model reaching the highest results. For a comparison to other retrosynthetic baselines see App. D.
Method
3
MRR f
Unaligned
4.1
6.5
7.8
9.8
0.056
DiffAlign-inpot
44.1
65.9
72.2
78.7
0.554
DiffAlign-PE.
70.7
76.6
81.8
0.601
DiffAlign Pf+tkii
54.7
73.3
77.8
81.1
0.639
We report results for the different aligned models and the unaligned model for top-k and the com- bined MRR score on the USPTO-50k test set in Table 1. The model without alignment performs worst, while combining the input alignment with positional encodings achieves the highest top- I < 10 and MRR scores across all models, making our diffusion based model competitive with SOTA retrosynthesis. Specifically, Table shows our top-k accuracy and MRR score compared to other baselines. We also outperform template-based models in all top-k scores (see note on the evaluation of (Igashov et al., 2024) to understand why its results are not comparable), and outper- form all non- non-rretr models 1op- While we a large value for T during training in our best models, we also highlight that the performance of the aligned model does not degrade significantly when reducing the count of sampling steps to fraction of T 100. See Fig. 6 for an ablation study
8

---

Published as a conference paper at ICLR 2025
Aligned Denoiser
Unaligued Denoiser
Sumple with AM
Product
Sample
Product
100%
Aligned model
Pormuuaaiin cquivariant model
80%
Top-k
Sample from Po Xx X Y, px-xx
Sample (x Xy.
= blank
= absorbing
atom mapping
100
Sampling step count
(a) Aligned vs unaligned denoiser
(b) Quality of samples vs. sampling speed
Figure 6: Top-k scores for sampling step counts T for our PE-skip model and a model with a standard permutation equivariant denoiser, using the first 10% of validation set reactions.
with the number of sampling steps for our best model with positional encodings and skip connec- tions. We also compare a permutation equivariant model and show that the top-k scores go zero at 10 steps, while the aligned model sometimes recovers the ground truth even with a single denoising step. We provide further ablations for different transition matrices, model using only the skip connections, and a model with matched Gaussian noise positional encodings in App. E. With the latter experiment, we show that the benefits brought by the positional encoding having the graph inductive bias due to the graph Laplacian eigenvector structure are not particularly significant, compared the inductive bias of alignment brought by matching the positional encodings across sides.
Table 2: An extended comparison with top-k accuracy and MRR on the USPTO-50K test data set. We include models with pretraining on larger data sets, and Retrobridge (Igashov et al., 2024), model whose evaluation done with relaxed metric that does not consider charges or stereochem- istry.
Method
k 11
k = 3
k= 5
k 10
MRR
Pre- minned
RSMILES (Zhong et al., 2022)
56.3
86.2
91.0
0.680
PMSR (Jiang al., 2023)
62.0
0.704
Temp.
Retrosym (Colcy et al., 2017b)
37.3
54.7
63.3
74.1
0.480
GLN (Dai al., 2019)
52.5
74.7
81.2
87.9
0.641
LocalRetro (Chen Jung, 2021)
90.6
Synthon
GraphRetro (Sommath et al., 2021)
53.7
68.3
72.2
75.5
0.611
RetroDiff (Wang et al., 2023)
52.6
71.2
81.0
83.3
0.629
MEGAN (Sacha et al., 2021)
48.0
70.9
78.1
85.4
0.601
G2G (Shi 2020s)
SCROP (Zheng al., 2019)
43.7
60.0
65.2
68.7
0,521
Tied Transformer (Kim al., 2021)
47.1
67.1
73.1
76.3
0.572
Template free
Aug. Transformer (Tetko et 2020)
48.3
73.4
77.4
0.569
Retrobridge (Igushov
2024)
GTA ang (Sen et al., 2021)
51.1
67.6
74.8
81.6
0.605
Graph25MIL.ES (Tu & Coley, 2022)
52.9
66.5
70,0
72.9
0.597
Retroformer (Wan et al., 2022)
53.2
71.1
76.6
82.1
0.626
DualTF aug (Sun al., 2021)
53.6
70.7
74.6
77.0
0.619
Unaligned
4.1
7.8
9.8
Ours
DiffAlign input
78.7
DiffAlign PE
49.0
70.7
76.6
81.8
0.601
DiffAlign-PE. PPeskip
54.7
73.3
77.8
81.1
0.639
4.3 BENEFITS OF DIFFUSION: GUIDED GENERATION AND INPAINTING
We study the use of an external function for guided generation and inpainting, thus demonstrating the advantages of graph-to-grapp diffusion in the field retrosynthesis as a concrete application domain. In retrosynthesis, bility of the generated reactants being synthesisable, using some pre- trained synthesisability model. To showcase the idea, use toy synthesisability model based the total count of atoms in
interesting use-case for crathoc-conditionnng is to increase the proba-

---

Published as a conference paper at ICLR 2025
does not correspond to ground truth
Objective: syathesize
the correct reaction (pussibly up to implicit present data) inpainted form
1. Carbonylation
Hydrogenation
3. Friedel-Crafts Acylation
Real (retm)synthesis path
Rasic generation
xOt X
yo+ Lo00
tOx = yo0 X
With inpainting
HOX oot Knowledge desired and guess ahout side reactants
YY LC yovr
Knowledge about type reaction (acylation)
Figure 7: Replicating BHC's green synthesis of Ibuprofen using our model interactively, and com- paring the known synthesis path.
Product
Produuc
Generate reaciant conditional another
High atom count guidance
Low alom count guidance
Generate conditional on destred substructure
(a) Atom guidance
(b) Inpainting
Figure 8: (a) Atom count guidance lets us specify if reactants should have many or few atoms, controlling the atom economy. (b) Examples of inpainting with our model. Parts highlighted in yellow are fixed by a practitioner reflect desired characteristics, and the diffusion model completes the reaction.
the reactants (Ertl Schuffenhauer, 2009)eFigure 8a shows an example where we nudge the model towards precursors with lower atom count with details of the procedure given in Alg. 3. More de- tailed comments on this procedure are available in App. F. To illustrate the benefits of inpainting, present hypothetical scenario of finding known synthesis pathway, in particular, BHC's green synthesis of lbuprofen (Cann & Connelly, 2000). Fig. compares the output of our model to the ground truth synthesis. We write the steps in detail in App. H.
CONCLUSION
In this work, we study an important aspect of conditional graph diffusion models: the equivariance of the denoiser. We show that permutation equivariant model converges to 'mean' distribu- tion for all graph components- fundamental limitation stemming from symmetry in the diffusion process. We propose aligned permutation equivariance as solution, forcing the model to only consider permutations that maintain alignment between conditioning and generated graphs. Our aligned denoisers achieve state-of-the-an results among template-free methods, reaching top-1 ac- curacy beyond template-basee methods. This approach unlocks the benefits graph diffusion in retrosynthesis, including flexible post-training conditioning and adjustable sampling steps during inference inference-vvllabllle properties for interactive applications and multi-step retrosynthesis planners.
A limitation is the requirement of some mapping information between conditional and generated graphs, though not fully mapped graphs. The model handles mapping errors with additional denois- ing steps. Another limitation is the high computational demand of iterative denoising. Advances in accelerated diffusion sampling methods (Hoogeboom et al., 2022a; Karras et al., 2022; Lu et al., 2022; Song et al., 2023b; Sauer et al., 2023; Shih al., 2023) are likely improve this aspect.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
We acknowledge funding from the Research Council of Finland (grants 339730, 362408, 334600, 342077) and the Jane and Aatos Erkko Foundation (grant 7031703.. We acknowledge CSC IT Center for Science, Finland, for awarding this project access to the LUMI supercomputer, owned by the EuroHPC Joint Undertaking, hosted by CSC (Finland) and the LUMI consortium through CSC. We acknowledge the computational resources provided by the Aalto Science-IT project.
REFERENCES
Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. Struc- tured denoising diffusion models in discrete state-spaces. In Advances Neural Information Processing Systems, volume 34, pp. 17981-17993, Curran Associates, Inc., 2021.
Benjamin Boys, Mark Girolami, Jakiw Pidstrigach, Sebastian Reich, Alan Mosca, and O Deniz Akyildiz. Tweedie moment projected diffusions for inverse problems. arXiv preprint arXiv:2310.06721, 2023.
M. Cann and M. Connelly. Synthesis of ibuprofen: A greener synthesis of ibuprofen which creates less waste and fewer byproducts. In Real World Cases in Green Chemistry, American Chemical Society, Washington, DC, 2000.
Shuan Chen and Yousung Jung. Deep retrosynthetic reaction prediction using local reactivity and global attention. JACS Au, 1(10):6112-120,, 2021.
Hyungjin Chung, Jeongsol Kim, Michael Thompson Mccann, Marc Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In International Conference on Learning Representations, 2023.
Connor W. Coley, Regina Barzilay, Tommi S. Jaakkola, William H. Green, and Klavs F. Jensen. Prediction of organic reaction outcomes using machine learning. ACS Central Science, 3(5): 443, 2017a.
Connor W. Coley, Luke Rogers, William H. Green, and Klavs F. Jensen. Computer-assisted ret- rosynthesis based on molecular similarity. ACS Central Science, 3(12):137-1225, 2017b.
Hanjun Dai, Chengtao Li, Connor Coley, Bo Dai, and Le Song. Retrosynthesis prediction with con- ditional graph logic network. In Advances in Neural Information Processing Systems, volume 32, pp. 8882.
Curran Associates, Inc., 2019.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis, Advances in neural information processing systems, 34:8780-8799 8794, 2021.
Zehao Dou and Yang Song. Diffusson posterior sampling for linear inverse problem solving: A filtering perspective. International Conference on Learning Representations, 2024.
Vijay Prakash Dwivedi and Xavier Bresson. A generalization of transformer networks to graphs. In AAAI Workshop on Deep Learning on Graphs, 2021.
Vijay Prakash Dwivedi, Chaitanya Joshi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Benchmarking graph neural networks. Journal of Machine Learning Research, 24(43):1-485 2023.
Peter Ertl and Ansgar Schuffenhauer. Estimation of synthetic accessibility score of drug-liks like molecules based on molecular complexity and fragment contributions. Journal of Cheminfor- matics, 1:1-11, 2009.
Mare Anton Finzi, Anudhyan Boral, Andrew Gordon Wilson, Fei Sha, and Leonardo Sepeda-Nunee User-defined event sampling and uncertainty quantification in diffusion models for physical dy- namical systems. In International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 10156-10152. PMLR, 2023.

---

Published as a conference paper at ICLR 2025
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, volume 33, pp. 6840-68511 Curran Associates, Inc., 2020.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633- 8646, 2022.
Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling, Argmax flows and multinomial diffusion: Learning categorical distributions. In Advances in Neural Information Processing Systems, volume 34, pp. 12454- 12465. Curran Associates, Inc., 2021.
Emiel Hoogeboom, Alexey Gritsenko, Jasmijn Bastings, Ben Poole, Rianne van den Berg, and Tim Salimans. Autoregressive diffusion models. In International Conference on Learning Rep- resentations, 2022a.
Emiel Hoogeboom, Víctor Gareia Satorras, Clément Vignac, and Max Welling. Equivariant diffu- sion for molecule generation in 3D. In International Conference Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 8867-8887. PMLR, 2022b.
H. Huang, L. Sun, B. Du, Y. Fu, and W. Lv. Graphgdp: Generative diffusion processes for permuta- tion invariant graph generation. In 2022 IEEE International Conference on Data Mining (ICDM), pp. 201-210. IEEE Computer Society, 2022.
Ilia Igashov, Arne Schneuing, Marwin Segler, Michael M Bronstein, and Bruno Correia. Retro- Bridge: Modeling retrosynthesis with Markov bridges. In Conference Learning Representa- fions, 2024.
John Ingraham, Vikas Garg, Regina Barzilay, and Tommi Jaakkola. Generative models for graph- based protein design. In Advances in Neural Information Processing Systems, 2019.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-sofiie softmax. In International Conference on Learning Representations, 2016.
Shaoxiong Ji, Shirui Pan, Erik Cambria, Pekka Marttinen, and Philip S. Yu. A survey on knowledge
graphs: Representation, acquisition, and applications. IEEE Transactions on Neural Networks and Learning Systems, 33(2), 2022. doi: 10.11099mmss22221.377000
Yinjie Jiang, WEI Ying, Fei Wu, Zhengxing Huang, Kun Kuang, and Zhihua Wang. Learning chemical rules of retrosynthesis with pre-training. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 5113-5121, 2023.
Wengong Jin, Connor Coley, Regina Barzilay, and Tommi Jaakkola. Predicting organic reaction out- comes with Weisfeiler-Lehman network. In Advances in Neural Information Processing Systems, volume 30, pp. 2607-2616. Curran Associates, Inc., 2017.
Wengong Jin, Kevin Yang. Regina Barzilay, and Tommi Jaakkola. Learning multimodal graph-to- graph translation for molecule optimization. In International Conference on Learning Represen- tations, 2019a
Wengong Jin, Kevin Yang. Regina Barzilay, and Tommi Jaakkola. Learning multimodal graph-to- graph translation for molecule optimization. In International Conference on Learning Represen- tations, 2019b.
Sékou-Qumar Kaba, Arnab Kumar Mondal, Yan Zhang, Yoshua Bengio, and Siamak Ravanbakhsh. Equivariance with learned canonicalization functions, 2023.
Tero Karras, Miik Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. In Advances in Neural Information Processing Systems, volume 35, pp. 26565-26577. Curran Associates, Inc., 2022.
Eunji Kim, Dongseon Lee, Youngchun Kwon, Min Sik Park, and Youn-Suk Choi. Valid, plausi- ble, and diverse retrosynthesis using tied two-way transformers with latent variables. Journal of Chemical Information and Modeling, (11::123--333, 2021.
12

---

Published as a conference paper at ICLR 2025
Hannah Lawrence, Vasco Portilheiro, Yan Zhang, and Sékou- Oumar Kaba. Improving equivariant networks with probabilistic symmetry breaking. ICML 2024 Workshop on Geometry grounded Representation Learning and Generative Modeling. 2024.
Zaiyun Lin, Shiqiu Yin, Lei Shi, Wenbiao Zhou, and Yingsheng John Zhang. G2gt: Retrosynthesis prediction with traph-to-graphe attention neural network and self toff-training. Journal of Chemical Information and Modeling, 63:18944 18:4199-9,, 2023.
Songtao Liu, Zhengkai Tu, Minkai Xu, Zuobai Zhang, Lu Lin, Rex Ying, Jian Tang, Peilin Zhao, and Dinghao Wu. FusionRetro: Molecule representation fusion via in-context learning for retrosyn- thetic planning. In International Conference Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 22028-203 -2241. PMLR, 2023.
Tie-Yan Liu et al. Learning to rank for information retrieval. Foundations and Trends@ in Informa- fion Retrieval, 3(3):225-311, 2009.
Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion language modeling by estimating the ratios of the data distribution. 2024.
Daniel Mark Lowe. Extraction of Chemical Structures and Reactions from the Literature. PhD thesis, University of Cambridge, 2012.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan LI, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around steps. In Advances in Neural Information Processing Systems, volume 35, pp. 5775-5787. Curran Associates, Inc., 2022.
Krzysztof Maziarz, Austin Tripp. Guoqing Liu, Megan Stanley, Shufang Xie, Piotr Gaiński, Philipp Seidl, and Marwin Segler. Re-evaluating retrosynthesis algorithms with syntheseus. In NeurlPS 2023 for Science Workshop, 2023.
Giangiacomo Mercatali, Yogesh Verma, Andre Freitas, and Vikas Garg. Diffusion twigs with loop guidance for conditional graph generation. In The eiirth- Annual Conference on Neural Information Processing Systems, 2024.
Chenhao NIu, Yang Song, Jiaming Song, Shengjia Zhao, Aditya Grover, and Stefano Ermon. Per- mutation invariant graph generation via score-based generative modeling. In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pp. 1474-448. PMLR, 2020.
Xinyu Peng. Ziyang Zheng, Wenrui Dai, Nuoqian Xiao, Chenglin Li, Junni Zou, and Hongkai Xiong. Improving diffusion models for inverse problems using optimal posterior covariance. arXiv preprint 442002240. 2024.
Emanuele Rossi, Ben Chamberlain, Fabrizio Frasca, Davide Eynard, Federico Monti, and Michael Bronstein. Temporal graph networks for deep learning on dynamic graphs. arXiv preprint 20XV::2006.10637, 2020.
Mikolaj Sacha, Mikolaj Blaz, Piotr Byrski, Pawet Sabrowski-Tumansee Mikolaj Chrominski, Rafat and Stanistaw Jastrzebski. Molecule edit graph attention network: modeling chemical reactions as sequences of graph edits. Journal of Chemical Infor- mation and Modeling, 61(7):3273-3284 2021.
Loska, Pawet
Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion dis- tillation. arXiv preprini arXiv:231 2023.
Nadine Schneider, Nikolaus Stiefl, and Gregory A Landrum. What's what: The (nearly) definitive guide to reaction role assignment. Journal of Chemical Information and Modeling, 56(12):2336 2346, 2016.
Philippe Schwaller, Theophile Gaudin, David Lanyi, Costas Bekas, and Teodoro Laino. "Found in Translation": predicting outcomes of complex organic chemistry reactions using neural sequence- to-sequence models. Chemical Science, 9(28):6091 16098, 2018.
13

---

Published as a conference paper at ICLR 2025
Philippe Schwaller, Teodoro Laino, Théophile Gaudin, Peter Bolgar, Christopher A Hunter, Costas Bekas, and Alpha A Lee. Molecular transformer: model for uncertainty-calibrat chemical reaction prediction. ACS central science, 5(9):1572-1583, 2019a.
Philippe Schwaller, Teodoro Laino, Théophile Gaudin, Peter Bolgar, Christopher A Hunter, Costas Bekas, and Alpha A Lee. Molecular transformer: A model for uncertainty-calibacce chemical reaction prediction. ACS Central Science, 5(9):1572-1583, 2019b.
Marwin HS Segler and Mark P Waller. Neural-symbolic machine learning for retrosynthesis and reaction prediction. Chemistry-A European Journal, 23(25):5966-597) 2017.
Seung-Wor Woo Seo, You Young Song, June Yong Yang, Scohui Bac, Hankook Lee, Jinwoo Shin, Sung Ju Hwang, and Eunho Yang. GTA: Graph truncated attention for retrosynthesis. Pro- ceedings of the AAAI Conference on Artificial Intelligence, 35(1):531-553 2021.
Chence Shi, Minkai Xu, Hongyu Guo, Ming Zhang, and Jian Tang. A graph to graphs framework for retrosynthesis prediction. In International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 8818-8822 PMLR, 2020a.
Chence Shi, Minkai Xu, Hongyu Guo, Ming Zhang, and Jian Tang. A graph to graphs framework for retrosynthesis prediction. In International conference on machine learning, pp. 8818-8827. PMLR, 2020b.
Andy Shih, Suneel Belkhale, Stefano Ermon, Dorsa Sadigh, and Nima Anari Parallel sampling of diffusion models. In Advances in Neural Information Processing Systems, pp. 4263-4276. Curran Associates, Inc., 2023.
Tess E. Smidt, Mario Geiger, and Benjamin Kurt Miller. Finding symmetry breaking order parame- ters with euclidean neural networks. Phys. Rev. Res., 3, 2021. doi: 0..1103/hyyReeeeeseach....3 L012002.
Jascha Sohi-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learn- ing, volume 37 of Proceedings of Machine Learning Research, pp. 2256 2265. PMLR, 2015.
Vignesh Ram Somnath, Charlotte Bunne, Connor Coley, Andreas Krause, and Regina Barzilay. Learning graph models for retrosynthesis prediction. In Advances in Neural Information Process- ing Systems, volume 34, pp. 9405-9415. Curran Associates, Inc., 2021.
Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz. Pseudoinverse-guiie diffusion models for inverse problems. In International Conference Learning Representations, 2023a.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Interna- tional Conference on Learning Representations, 2021.
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 32211-32252. PMLR, 2023b.
Balasubramaniam Srinivasan and Bruno Ribeiro. On the equivalence between positional node em- beddings and structural graph representations. In International Conference on Learning Repre- sentations, 2020.
Ruoxi Sun, Hanjun Dai, Li Li, Steven Kearnes, and Bo Dai. Towards understanding retrosynthesis by energy-based models. In Advances in Neural Information Processing Systems, volume 34, pp. 10186-10194, Curran Associates, Inc., 2021.
Igor V Tetko, Pavel Karpov, Ruud Van Deursen, and Guillaume Godin. Statc-of-the-an augmented NLP transformer models for direct and single-step retrosynthesis. Nature Communications, 11 (1):5575, 2020.
Barry M Trost. The atom economy a search for synthetic efficiency. Science, 254(5037): 1471- 1477, 1991.
14

---

Published as a conference paper at ICLR 2025
Zhengkai Tu and Connor W Coley. Permutation invariant Seuph-to-sequenee model for template- free retrosynthesis and reaction prediction. Journal of Chemical Information and Modeling, 62 (15):3503-3513, 2022.
Yogesh Verma, Markus Heinonen, and Vikas Garg. AbODE: Ab initio antibody design using con- joined ODEs. In International Conference on Machine Learning, 2023.
Clement Vignac, Igor Krawezuk, Antoine Siraudin, Bohan Wang, Volkan Cevher, and Pascal Frossard. DiGress: Discrete denoising diffusion for graph generation. In International Con- ference Learning Representations, 2023.
Yue Wan, Chang- Yu Hsieh, Ben Liao, and Shengyu Zhang. Retroformer: Pushing the limits of end- to-end retrosynthesis transformer. In International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 2-475-22490. PMLR, 2022.
Yiming Wang, Yuxuan Song, Minkai Xu, Rui Wang. Hao Zhou, and Weiying Ma. RetroDiff: Ret- rosynthesis as multi-stage distribution interpolation. arXiv preprint arXiv:2311.14077 2023.
David Weininger. Smiles, chemical language and information system. Journal of Chemical Infor- mation and Computer Sciences, 28(1):31-36 1988.
Jin- Wen Wu, Fei Yin, Yan-Ming Zhang, Xu- Yao Zhang, and Cheng Liu. Graph-to-graphe To- wards accurate and interpretable online handwritten mathematical expression recognition. Pro- ceedings of the AAAI Conference on Artificial Intelligence, 35(4), 2021. doi: 10. (6699/aai.vv351 16399.
Luhuan Wu, Brian Trippe, Christian Naesseth, David Blei, and John P Cunningham. Practical and asymptotically exact conditional sampling in diffusion models. In A. Oh, T. Naumann, A. Glober- son, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Sys- Curran Associates, Inc., 2023.
tems, volume 36, pp. 31372-314033
Shufang Xie, Rui Yan, Junliang Guo, Yingee Xia, Lijun Wu, and Tao Qin. Retrosynthesis prediction with local template retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 5330-5338, 2023.
YuQing Xie and Tess Smidt. Equivariant symmetry breaking sets, 2024.
Chaochao Yan, Qianggang Ding, Peilin Zhao, Shuangjia Zheng, JINYU YANG, Yang Yu, and Jun- zhou Huang. RetroXpert: Decompose retrosynthesis prediction like chemist. In Advances in Neural Information Processing Systems, volume 33, pp. 11248-1 12588. Curran Associates, Inc., 2020.
Qi Yan, Zhengyang Liang. Yang Song, Renjie Liao, and Lele Wang. SwinGNN: Rethinking per- mutation invariance in diffusion models for graph generation. arXiv preprint arXiv:2307.01646, 2023.
Yan Zhang, David W. Zhang, Simon Lacoste-Julien, Gertjan J. Burghouts, and Cees G. M. Snoek. Multiset-equivariant set prediction with approximate implicit differentiation, 2022.
Shuangjia Zheng, Jiahua Rao, Zhongyue Zhang, Jun Xu, and Yuedong Yang. Predicting retrosyn- thetic reactions using self-corrected transformer neural networks. Journal of Chemical Informa- tion and Modeling. 60(1):47-55, 2019.
Zipeng Zhong, Jie Song, Zunlei Feng, Tiantao Liu, Lingxiang Jia, Shaolun Yao, Min Wu, Tingjun Hou, and Mingli Song. Root-aligned SMILES: a tight representation for chemical reaction pre- diction. Chemical Science, 13031)49033-944, 2022.
15

---

Published as a conference paper at ICLR 2025
APPENDICES
This appendix is organized follows.
App. A presents our theoretical results on aligned permutation equivariance and accompanying proofs. App. B provides additional details the setup for conditional graph diffusion, includ- ing the transition matrices, noise schedule, and data encoding as graphs. App. C includes additional details to replicate experimental setup. App. D provides detailed comparsion between model and other retrosynthetic baselines. App. F develops method to apply arbitrary post-training con- ditioning with discrete diffusion models, and presents case studies showcasing the usefulness of post-training conditional inference in applications relevant to retrosynthesis. This includes generat- ing samples with desired properties and refining the generation interactively through inpainting.
A ALIGNED PERMUTATION EQUIVARIANCE
18
A.I VISUALIZING OUR ALIGNMENT METHODS
18
A.2 PROOF THAT PERMUTATION EQUIVARIANT DENOISERS Do NOT RECOVER THE IDENTITY DATA
18
A.3 PROOF THAT ALIGNED PERMUTATION EQUIVARIANT DENOISERS RECOVER THE IDENTITY DATA
20
A.4 PROOF OF THE GENERALIZED DISTRIRUTIONAL INVARIANCE WITH ALIGNED
EQUIVARIANCE
A.5 PROOPS THAAT OUR DENOISERS ARE ALIGNED PERMUTATION EQUIVARIANT
22
A.6 A SINGLE-LAYER GRAPH TRANSFORMER WITH ORTHOGONAL ATOM-MAPPED
POSITIONAL ENCODINGS ABLE TO IMPLEMENT THE IDENTITY DATA SOLU-
TION FOR NODES
23
A.7 EXTENDING THEOREM SMALLER TIME STEPS
25
A.8 ELABORATION ON WHY WE CAN CHOOSE ANY NODE MAPPING MATRIX DURING INFERENCE
26
DETAILS ON CONDITIONAL GRAPH DIFFUSION
C EXPERIMENTAL SETUP
29
C.I COPYING TASK ON THE GRID DATASET
29
C.2 DATA: USPTO DATA SETS
29
C.3 NOTES ON OUR SAMPLING AND RANKING PROCEDURES
30
C.4 DETAILS ON STEREOCHEMISTRY
30
C.5 DETAILS THE EVALUATION PROCEDURE
31
C.6 NEURAL NETWORK ARCHITECTURE, HYPERPARAMETERS, AND COMPUTE RE- SOURCES
31
D COMPARISION TO RETROSYNTHETIC BASELINES
32
E ADDITIONAL ABLATIONS
34
F ADDING CONDITIONING DISCRETE DIFFUSION MODELS
35
G HANDLING NOISY NODE MAPPINGS FOR THE TOY DATA
37
16

---

Published as a conference paper at ICLR 2025
H DETAILS FOR THE IBUPROFEN SYNTHESIS EXPERIMENT
37
17

---

Published as a conference paper at ICLR 2025
Do YY
PY-
X E
X-V
Do(X,
Matched positional encodings
GNN
Y
YW
DuXKY YY
X E
X
Do(X. Y)
Neural net predicts the difference to the product graph
GNN
YE
Y-M
PY Xyx pY- ( (p
Align the graphs in the input
Dx(X
X
X
GNN
Do(X, Y)
Y-V
F
Figure A9: Different ways to align the graphs in the architecture. All of them can be combined. The sign means concatenation along the feature dimension, and + is the standard addition. All of the methods can be combined together.
A ALIGNED PERMUTATION EQUIVARIANCE
A.1 VISUALIZING OUR ALIGNMENT METHODS
We visualize the various alignment methods we use in this work in Fig. A9.
A.2 PROOF THAT PERMUTATION EQUIVARIANT DENOISERS Do NOT RECOVER THE IDENTITY DATA
(X,, Y,, PY- where for all data points, X, = -Y--Xy, that is, both sides of the reactions are equivalent, up to some permutation, as defined in the atom mapping matrix PY It is always possible to preprocess the data such that the rows of Y, are permuted with Y, f- -YPPXXX Y, so that the resulting atom mapping between Y, and X,, is always identity. For simplicity, assume such preprocessed data set in this section.
Definitions Let us consider a data set
Let us assume that the one-step denoiser probability, po (X, Xr, Y), is parameterized by the neural network Dø Dr, Y) VNX K such that the probability factorises for the individual nodes and edges (so there is one output in the network for each node and each edge): po (X,) Xr. Y) = II: Ex X Do (Xr, Y) II. E. XE G. Do (XT, Y)
18

---

Published as a conference paper at ICLR 2025
The ideal denoiser The correct one-step denoiser Dø (X Y) Y. This can be shown with the Bayes' rule by ((Xo (x, Y) x(Xr Xo.Y (og(Xo one X always matches with exactly one Y in the data, this is delta distribution g(o, YY) =
2 2
gXoo Y). Because
g(Xr Y)
g(Xr)
I. 1 where we define z0,; and Y/ as the value of the i:th node edge. It is also easy to see that Do(XT, Y) =
Y is the optimal solution for the cross-entropy loss:
Σ
((oo) log (Xo) XT, Y)
E log (Y XT, Y)
()
=
Σ log III
o. DX
Y B o.jjj. Dx( Y) figh
(13)
fig are maximized for each i,j and Y if Do (Xy, Y) Y. In this case, the loss goes zero.
All of the sums En Y. DXT,, Y and E. Y 0.
(x D.
D.i.A.
The following theorem states that if the neural net is permutation equivariant, it will converge to a 'mean' solution, where the output for each node and each edge is the marginal distribution of nodes and edges in the conditioning product molecule, instead of the global optimum Dx Y) =Y
Theorem 3. The optimal permutation equivariant denoiser Let D& Dx Y) be permutation equiv- ariant s.t. DPXT Y) = PDs(Xr, Y), and let q(X)) be permutation invariant. The optimal solution with respect to the cross-entropy loss with the identity data is, for all nodes and j
Dx YY = y = Y. EE E.A
(14)
(X = -f =A. YISH! Edi Yigkt.
=
where Sh and yh are the marginal distributions of node and edge values in Y.
Proof. Nodes. The cross-entropy denoising loss for the nodes can be written
CE
Σ
Eg(X
Σ
x i log Do Do Y)
(15)
Xx)
(Xo,Y)
i.k
Σ Ex(XT)
Σ
x-V. log Do (xr, Y)i
(16)
(Xo.Y)
1,k
--EBA E (x()
Σ
Y. log Do (XT, Y)
(17)
where the first equality is due to qxT X,) containing no information about X, at the end of the forward process, and the second equality due to X = Y the data. Since ((T)) is permu- tation invariant, that is, all permuted versions PX7 of X are equally probable, we can split the expectation into two parts Ex(x))l-1 & Ey(X4) Ep(:) where X/, contain only graphs in distinct isomorphism classes, and Ep sums over all permutation matrices of size N x N.
CE α - Σ Ey(x4))
ΣΣ Y. log DP Y) 1,k
(18)
Due to the permutation equivariance, (PX,, Y PDo Y) and DPX, Y) (X Y)N (1),k= where ri) denotes the index the index is mapped to in the permutation P. Thus,
CE x - Σ By(X,,) Y
ΣΣ K
&A log Do Y)
(19)
Bx(xe,)
Y
Co. log Do (X Y)
(20)
where the equality is due to all permutations being in a symmetric position: What matters is the rela- tive permutation between Y and Da XXr, Y). Now, Σ, YN
1()) Σ= YN
*(), C Y.V
19

---

Published as a conference paper at ICLR 2025
because for each node index i, all the nodes in Y are included equally often due to symmetry. This is proportional to the marginal distribution 5-V up to some constant, and thus we have:
CE B
St log Do Y)
(21)
i,k
The optimal value for each node output is the empirical marginal distribution D Y) = (g-V)..
Edges With the exact same steps, we can get the equivalent of Eq. (18) for the edges:
CE x
By(
Σ
ΣΣYE
Y ii.k log Do Y)
(22)
P 1.
The permutation equivariance property for the edges is now written as Do Y) (X4 Y) and Da Y) = ( () Thus,
=
CE x - Σ B ΣΣ Y log D 11
(23)
-E
Ex()) ΣΣ 10.- log DX Y)
(24)
with the equality holding again due symmetry. Now, for any pair of node indices and j. the set of all permutations contains all pairs of node indices (x (()) equally often due to symme- try. These pairs corresoond to edges Y and thus En Y (-)
Σn Y a(),
D Ex, ki where D is constant that counts how many times each edge pair appeared in the set of all permutations. This is again proportional to the marginal distribution over the edges y
CE
E
(X
y log DX Y).... ΣSE
(25)
Again, the optimal value for each edge output (i, is Do Y) = (YB)T.
A.3 PROOF THAT ALIGNED PERMUTATION EQUIVARIANT DENOISERS RECOVER THE IDENTITY DATA
Here, we detail the steps in the proof of Theorem 1, but with aligned permutation eqivariance intead of regular permutation equivariance, and highlight the principal differences both calculations. We start with versions of Eq. (17) rewritten with atom mapping explicitly, and PY PY-X conditioning in the denoiser:
CE= Σ
Eg(Xr (x)) Σ
ExM. log Do Y, PV )
(26)
(X.,Y,PY-x)
= Σ
Ex(xx)
)i, log Do (x Y, PY + Ji.k
(27)
(X)
i.
Versions of Eq. (18) to Eq. (20) with atom mapping conditioning and utilizing aligned permutation cquivariance (instead of permutation equivariance):
CF
Σ
Bxxx))
ΣΣ('
E(PP
( )a log Do Y, PY Vi
(28)
Xx x
P 1.k
Σ
By(x7)
ΣΣ( (PY MXX-XXyN )I. log Do( Y, p-1pp-p-- -X
+X N
(29)
(()))
-X
i.k
Σ (x(
Σ Σ (PX-XX ) log Do (X Y. P pp +X )i.
(30)
-XP
i.k
Σ Bxx23) Σ Σ
(P---p--X ) log Do(X Y, p-1 p-1py-X1
(31)
Y,PY sx
P
20

---

Published as a conference paper at ICLR 2025
We could assume again for notational simplicity (not necessary) that pY I in the data. In any case, we have cross-entropy loss where we try to predict permutation of Y. P pplpp-aa ' po'+Xy- Both p-1py-X and Y are given as explicit information to the neural network, for each P, the optimal output is p-1 p-1pY-u py-pyyyy Y, which a permutation of the original Y.
The primary difference in these two proofs:
With the version with regular permutation equivariance, only the data term Y is dependent on the permutation P in Eq. (20). This makes it possible to factor out the sum Σn Y x (0) leading to Eq. (21). This factorization also why the optimal neural net output is not dependent on P.
With aligned penmutation equivariance, the denoiser term retains dependence on P and the maximal factorization Ep(P "y ),, log Do(X;, (X,, Y, P.,P) ") Thus, for each com- bination of P and Y, the neural net will have different optimal output.
A.4 PROOF OF THE GENERALIZED DISTRIBUTIONAL INVARIANCE WITH ALIGNED EQUIVARIANCE
We start by proving useful lemma, and then continue and continue to the proof of the main theorem.
Lemma 1. (An aligned denoiser induces aligned distribution equivariance for a single reverse step)
the denoiser function Da has the aligned equivariance property Da RRX. QY RRY-XXQ))
RDD (X, Y, PX-X)) then the conditional reverse distribution pa (X, X, Y, VYPY- has the property po (RX,-1 RX;; QY,RYY RXPY-XQTT (X, Xt, Y PY-X).
Proof. First, let us denote the transition probabilities from L to L with Fo(X,, Y Y.-x))), where formally FX,,, Y, Y,PY PY-PYXXA )
iik Σx ((N 1-1ik X.N X Gi,ks 0,2 Da Y, pY--XX--P like
and (X,, Y, V.PY-XjE A.j.k Ev q( X.E L.i.j.k ( Do(o, (D,,, Y,PY-XJE i.j.k' Since
the values of Fg depend only pointwise on the values of Da. Fa is aligned permutation equivariant as well.
We continue by directly deriving the connection:
Pa (RX, RX, QY, RPT)
(32)
I
RX. RX.
W QY, RP- )(.
RXX, Po QY XPX-QQT lig.
(33)
1 h
Y,PY-X (1).A
II X. (X- Fo Y. P 1 (1)
(34)
where in the last line we used the aligned permutation equivariance definition, and the the effect of the permutation matrix R on index was denoted as (i). Now, regardless of the permutation, the products contain all possible values and pairs i. exactly once. Thus, the expression remains equal if we replace TT)) with just i:
Pe P8( - RX, QY, RRY XPX-XQ))
(35)
II X. NY Po Y, -Y--X )ii
)i,
I
x, D2 Y, YY-Y-X
)
Fij.k
(36)
&
Po (X.-1 X,, Y, YY-XXX).
(37)
which concludes the proof.
Theorem 4. Aligned denoisers induce aligned permutation invariant distributions If the denoiser
function Dn has the aligned equivariance property and the prior p(X)) is permutation invariant,
21

---

Published as a conference paper at ICLR 2025
then the generative distribution (Xu Y, PY-X) has the corresponding property for any permu- tation matrices R and Q:
po (RXo QY, RPY Q = PX Y, PY-XX
(38)
Proof. Let assume that the result holds for some noisy data level t: pe (RX; RX,RP RPP-XXQ)) Px Y. YYPx-x). We will then show that the same will
hold for X1-1, which we can use to inductively show that the property holds for Xo. We begin as follows:
Po PRX, QY, RPY-XQQ) = Σ pa (RX, Xx, QY, TRPY-XO, Q )o (x, QY, RP RPX-=X""
X
(39)
Σ X,
pa (X Y, PYPPXXX + Doo -x, Y, PY-X)
(40)
where on the second line we used Lem. and the assumption that the result holds for noise level t. The sum over X, contains all possible graphs and all of their permutations. Thus, the exact value of R does not affect the value of the final sum, as we simply go through the same permutations in different order, and aggregate the permutations with the sum. Thus,
po (RX, QY RPX-R-X-O XXX-XXQ
po Po (x-- X. Y, Px Y.P Σ X
po (X. (Y
(41)
PX 1 XXPX-
(42)
showing that if the result holds for level f, then it also holds for level - 1. We only need to show that it holds for level X to start the inductive chain:
Po 1IXX--IQQ RPY-R--- RP Σ
RX,, XT. QY, RRY Q )((T))
(43)
XT
Σ
po (
R R- Y, PY-X
)R(R (RR)
(44)
where on the second line we again used Lem. and the permutation invariance of p(Xr). Again, the exact value of R does not matter for the sum, since the sum goes through all possible permutations
in any case. Thus we have
Po (RXT QY, RPY-XQU)
Σ Po Xy. Y. Y.PY-X )((T)
(45)
Xx
(XT-1 Y Y.P.XXX
(46)
Thus, since the property holds for - it also holds for X -2...... until X,. This concludes the proof.
A.5 PROOFS THAT OUR DENOISERS ARE ALIGNED PERMUTATION EQUIVARIANT
In this section, show for each of the three alignment methods that the corresponding denoisers do indeed fall within the aligned permutation equivariance function class. Fig. A9 summarizes the different alignment methods.
mtom- positional encodings cquivariance condition, D QY XPY-XXQ ) for this particular function class, and directly show that it equals PDe X,, Y.P. -X).
We start by writing out one side of the aligned permutation
(RX, RY -X-XXX TT
fx RXX RPY-XQ Q RX [QY
Q+l Q
(47)
= (RXA RPY-X
R
QQ Q+) QY*QT),
(48)
22

---

Published as a conference paper at ICLR 2025
where fo itself is function that is permutation equivariant for the combined X and Y graph as input. This means that the neural net itself gives an output for the entire combined graph, but we only consider the X subgraph as the denoiser output, denoted here as fø For clarity, can combine the reactant and product node features and adjacency matrices in the notation, and use the permutation equivariance property of our GNN:
RXV QY
RPY-X
R O
fo(
Qi
QY
(49)
R
0
X.N
PY-X
x
0 YE
fo
Y
(permutation equivariance of base NN)
0
Q
0
(50)
Taking only the X part of the output and reverting to Da notation, we directly arrive at the result that (RX, QY, R (X, Y, Y,PY-X).
Directly adding to the output We again start by writing out an aligned equivariant input to the denoiser, with fe again denoting network that is permutation equivariant with respect to the combined X and Y graph:
(RX (RX,, QY QY.RPX softmax(f) (RX, QY) + RPY RPY-XQTQQY
(51)
softmax RRfo (X, Y) + -xp)
(permutation equivariance of base denoiser)
(52)
R softmax f (X, Y) +pX-Xy) (X, RD,,(X,Y, PX-Y)
(53)
(54)
where we were able to move the permutation outside the softmax since the softmax is applied on each node and edge separately.
Aligning Y and X at the input to the model Let's denote by [ PY-XY) concatenation along the feature dimension for both the nodes and edges of the graphs Recall then that the definition of aligning the graphs in the input is Do(X, Y, (Y permutation equivariant denoiser. Writing out the aligned equivariance condition
fe([X pY-XX where fo is
DR(RX, QY, RP Jo[RX QT = Sa([RX RPY-XY))
(55)
= fo(R[X PY-XY = Rfx( PX-XX1)
(56)
Do Y,YY-XX
(57)
which shows that this method results in aligned equivariance as well.
A.6 A SINGLE-LLAER GRAPH TRANSFOOMMER WITH ORTHOGONAL. ATOM-MAPPED POSITIONAL. ENCODINGS ABLE TO IMPLEMENT THE IDENTITY DATA SOLUTION FOR NODES
Here, we show that a single-layer Graph Transformer neural net can model the identity data for the nodes given orthogonal atom-mapped positional encodings (e-g.. the graph Laplacian eigenvector- based ones). In particular it is possible to find 0 such that D*(X,, Y, Y.pY-XjM Y.V. We hope that this section can serve as an intuitive motivation for why matched positional encodings help in copying over the structure from the product the reactant side.
Recall that we have atoms on both sides of the reaction. Let us map the atom mapping indices to basis vectors in orthogonal basis y (2. pPN] In practice, the node input to the neural net on the reactant side is now X; [ pp.. and the node input on the product side is Y YY-V sp).
23

---

Published as a conference paper at ICLR 2025
Now, single-layer Graph Transformer looks follows (as defined in the Vignae et al. (2023)
codebase):
N = (X
)T, (Y**)TT
(Concatenate rows)
(58)
X E
0
E
0
Y
(Create joined graph)
(59)
1 = MLPN (N)
(Applied with respect to last dimension)
(60)
E 2 (MLP MLPE(E)) (E) (Symmetrize the input)
(61)
I1 MT..((1)
(62)
Q K, V = Wi,I, WRN,, W.N, (One attention head for simplicity)
(63)
1 QK
(64)
A = Ai * (WE. E + +1) + E
(65)
A3 softmax(As)
(66)
N₂ AV/ /d (dy is the embedding dim of V)
(67)
N3 = N₂ * (Wat + 1) + W
(68)
E2 W(A (We, t1 + 1) + Wi,t1)
(69)
44 = .MPP, (Wt2 + W[min(Ns). max( (os)
W[minn (Ns), max(Ns), mean (N3) st:(
(70)
E3 = MLPPE (E2) + E
(71)
Eout (EE Eg))
(Symmetrize the output)
(72)
Nout = M MLP (Na) +
(73)
fout
14 +
(74)
Now, for purposes of illustration,
define most linear layers to be zero layers:
MLPE, ML,, W WE,, WE,, We W Wt,, Wt, = 0. In addition to this, we define MLPN to be identity transform. Wq and Wk are both chosen as picking out the U columns of N1, with additional overall scaling by some constant a. Wy is chosen to pick out the product node la-
bels: WvN1 = Y Now, we can easily see how the Graph Transformer can obtain the optimal
denoising solution for the nodes. Consider an input N = [(X;)) (I) where the reactant side is permuted. The output of the network should be PY. Focusing on the parts of the network that compute the node features:
Pyp P Pp
Pip pp
I P
P I
A a²
P
T
=
=
(75)
A A
(76)
I
P
Ag 2 softmax (A2)
(If >> 1),
(77)
2
P
I
A
I
P
0
/(2/d)) = PYN Yn ((2/d))
N2
ddp
PT
I
YN
(78)
Here, we used the fact that we chose the positional embeddings to be an orthogonal basis, and pppT= II as well as the fact that PPT = I for any permutation matrices. The term 1 in the third equation came from the fact that each row of A₁ contains two non-zero values that are also equal. The probability gets divided between the two of them in the softmax if the logits are scaled large enough, and the approximation becomes arbitrarily accurate.
From now on, since we are interested the denoising output only for the reactant side, we drop out the reactant side Y and only focus on the PYW part. We choose the final M MLPN, to scale the output by some factor 3 > 1:
Nout N + +N
(79)
(PX; Y softmax 3PY + N) PY
(80)
24

---

Published as a conference paper at ICLR 2025
Here, the approximation be made arbitrarily accurate by scaling to higher value, since the logits will become more and more peaked towards the values where PY equals one instead of zero. This showcases how the attention mechanism in the Graph Transformer pairs with atom mapping-based orthogonal positional encodings to achieve the identity function from products to reactants.
A.7 EXTENDING THEOREM 1 SMALLER TIME STEPS
We prove the following result characterizing the optimal denoiser output for a more restricted setup of absorbing-stee diffusion and on an identity data with no edges (i.e., just set of nodes):
(Yi,s if X is not in the absorbing state
ΣyEM, YiA where Dt = EyEM, Ee Y9,
Do Y),
if Xe, is in the absorbing state
where Mr refers the of nodes the mask state. We will include the proof as an additional comment below. In words, the optimal output for the masked nodes is the marginal distribution of the nodes that have not been de-masked yet, that is, the mean of all possible placements of the remaining nodes.
The analysis for graphs with edges becomes more involved, but we postulate that similar analysis would show that the output would converge to a mean of all possible permutations of Y that are in agreement with the current set of subgraphs X₁.
To recap, we have the simplified scenario where we generate set of nodes X conditioned on another set of nodes Y. without edges. The goal is to characterize the optimal permutation equivariant denoiser for this process at any noise level.
Definitions. Let Xo E 13 be the initial set of N nodes, each represented by a one-hot vector of dimension K. We condition on Y E 1) where X, Y in our data. The forward process is an absorbing state diffusion, defined as follows for E 0, ..::
o(Xe,; MIXo,)) 1/T
(81)
g(x,, = Xo.) = 1-t/T
(82)
where M represents the additional "mask" state. We denote the denoiser as (x Y), which is assumed to be permutation equivariant with respect to X.,
N: M(X)): Set of indices of non-masked nodes 1/. Marginal distribution of node values in Y for the node indices that masked in X
Notation For a given X,, we define: M(X)) cI...., Set of indices of masked nodes U(X,) = 1,...,N \
,(())
MXX.J EieM(x)) Yi,;:
Result
Theorem 5. The optimal permutation equivariant denoiser Dx foror the absorbing state diffi-- sion process at any time and for any particular X, is given by: Dx(X,, Y)i, = Ye.. if E UX,)
DX Y)s,k = Ub
M(X)) if M (X.)
Proof. The cross-entropy loss for the denoiser at time t is:
N
CE.- ExxxxxXX
Σ Xo,t,k log Do (Xe, Y),,
(83)
Σ iEU(X.)
Σ Nasa log Dx Y)
Σ (EM(X))
Xa.. log (X,,,, Y)
(84)
We consider the optimal denoiser output for non-masked and masked nodes separately:
Non-masked nodes (i E U(X))) For these nodes, the optimal output is D,(X,, Y) = Y.
since Yú,- Xo,t.K-
25

---

Published as a conference paper at ICLR 2025
Masked nodes (i E (X)) Let's focus on a particular instantiation of Xt. Due to the permutation equivariance of Dg. for any permutation P that only permutes indices within M (X2), we have: Dp(P.., PD. X,, Y.. Moreover, because all nodes in (X, are in the same mask state, all such permutations of X, are identical: Px,,,,, Y) = Dx(X, Y) Combining these facts, we conclude that for any i M(x)): (X Y): Dx Y); Thus, the denoiser must output the same value for all masked nodes in a given X₁. Now, consider the part of the cross-entropy loss corresponding to the masked nodes for this particular Xx:
(X.) - Σ
Σ
No.s. log (x Y)
(85)
(xM(
Σ
No
log (X,, M)m,k
(86)
(EM(X))
where m is any index in M (X.). Observe that ΣiEM(X,) Xu,i. is proportional to the marginal distribution of the masked nodes in Xo (which is equal to Y). Using our defined notation, we can rewrite the cross-entropy as: CRM (X.) x -IM(X,) V/
(x()))) log Do(X,, Y)m,k: The optimal
(X m), that minimizes this expression is 8/. non-masked nodes yields the stated optimal denoiser.
((x)) Combining the results for masked and
A.8 ELABORATION ON WIYY WE CAN CHOOSE ANY NODE MAPPING MATRIX DURING INFERENCE
We note that all node mapping matrices can be characterized by a base permutation matrix P left multiplied by different permutations R. Applying Theorem 2: (Xol Y, RPY-X) XX Y, PY-+)), where R-1 is just another permutation matrix, shows that the distribu- tion equals pa (Xo Y.PY-X), up to some permutation. In fact, by sampling the node mapping matrix randomly, the distribution pe (Xo Y) = Epy-x PX Y, PY-P--Xp (ppp))(PYYX)) --X becomes invariant to any permutation of X, and Y, although this is not strictly necessary in our context.
B DETAILS ON CONDITIONAL GRAPH DIFFUSION
Our transition matrices To define Q and Qf, we adopt the absorbing-state formulation from Austin et al. (2021), where nodes and edges gradually transfer to the absorbing state L. Formally, give the generic form of the transition matrix Q for node input X E NNxNx!
Q. (1 B + 3,1eI,
(87)
where B, defines the diffusion schedule and e is one-hot on the absorbing state 1. For complete- ness, we list the other two common transitions relevant to our application. The first is the uniform transition as proposed by Hoogeboom et al. (2021)
11
Q = -1- 3
(88)
K
where Bt, I are before and K is the number of clement (edge node) types, i.e., the number of input features for both nodes and edges. Vignac et al. (2023) also proposed a marginal transition matrix
= 3'1 ( and OF -1-
8'1 (m)
(89)
which they argued leads to faster convergence. In this case, n R. and m R row vectors representing the marginal distributions for node and edge types respectively. We tested all three types of transition matrices in early experiments and noted the absorbing state model to be slightly better than the others. The marginal (X, Xn) and conditional posterior q(X, X,, Xo) also have a closed form for all of these transition matrices.
'The only difference between Q and Q for the absorbing state and uniform transitions is the dimensions of 1 and the value of K. We therefore give generic form for both and imply choosing the right dimensions for each case.
26

---

Published as a conference paper at ICLR 2025
Noise schedule which leads to
We use the mutual information noise schedule proposed by Austin et al. (2021),
1 T(X,;XX) HXx,, X,) - H(X.) Exoxx,(Xx))pe Xo) log otX, Xx)
Exg p(X, (
x,)
T
H(Xu)
H(Xo)
Ex, ((x)) log p( (Xn)
(90)
For absorbing state diffusion, these equations lead to B = 7-2 Similarly, the total transition probability to the absorbing state at time t has simple form: g(X, =1 (Xo) f
Forward process posterior For transition matrices that factorize over dimensions, we have
X,Q, Xa.i.. Qr-1
gX,, Xea. Xo,..)
(91)
Xo,s. Q.X
where X is the one-hot encoding of 1th node/edge of the graph, in row vector format.
Variational lower-bound loss Diffusion models commonly trained by minimizing the negative variational lower-bound on the model's likelihood (Ho et al., 2020). Austin et al. (2021) discuss the difference between optimizing the ELBO and cross-entropy losses and show that the two losses are equivalent for the absorbing transition. We choose to use cross-entropy, similar to Vignac et al. (2023), due to faster convergence during training. We include the formula for the ELBO Eq. (92) for completeness.
Cvb = Exg
KL [L( Xo) ll p(X))
Er
T
Ex Ex Xx) KL [g (xx X, X,) (a X,)]
KL
f--1
E EX, (x)) log (Xo x1)
(92)
We also note that use this quantity as part of the scoring function mentioned in Sec.
Data encoding and atom-mapping We illustrate our graph encoding using atom-mapping and permutation matrices in Fig. A10.
Connection between the denoising parameterization p X₂ X,, Y) and p(x. X,, Y). Here, we elaborate on how does p(Xo X,, Y) connect with p (X X,, Y).
Recall that the reverse transition is factorized with respect to nodes and edges:
Pa X X,, Y)
II
Nx pa X X., Y) Nx pa X 1-1
Edi
X.. Y).
(93)
We parameterize the transition as follows:
Po (X, X., Y) = Ex, (X.-1 X,, Xo) Xn Xx, Y).
(94)
The connection between this parameterization and the probabilities pe (X+V No noting that both (X,, X,, Xo) and Po (Xo) X,, Y) factorize over dimensions:
Xx, Y)
obtained by
Nx
Nx Edd
a(x,1 X X,)
II
II
g
Ex
x
x
B.d.
x B.d)
(95)
Nx
Nx
No (Xo X₁, Y) = 11
6o x
Ni
X Y)
II
po
(x
E E.h.d
X,, Y)
(96)
i.j
27

---

Published as a conference paper at ICLR 2025
Figure A10: Illustrating graph encoding using atom-mapping.
Reactants
Product
+
N
N
Atom mapping from the nodes of Y to the nodes of X
CONF
CONF
CONF
1000
1030
1000
Nx
X
0
NY
0100
00000
10000
1000
Ncc
0000
nupped
0001
1000
Y-V
XW
PY-XYV PY-X
Atom mapping from the edges of Y to the edges of X
O
U
U
0
0
(PY+X)
F
PY-X
X
PY- XX-XY"(PX-XX)
Plugging these to Eq. (94) and expanding the sum, we get
(M xx
Nx
Pa DX X,, Y)= = ΣΣ Σ E
x
30 Nil Xe. Y)
x
00
X
x
Nx
Nx
Fig
x
Fiid
x
F.i
pe
(x Frinj
X,, x)
i
(97)
28

---

Published as a conference paper at ICLR 2025
Gathering terms together, we get
Σ X
X
X x.
)0 (x, |x,, x)
(98)
xx X..Y)
Σ
(x
x Ni1 X po
) (X X x)
(99)
x
=po (X..Y)
(100)
Σ
N.N.N
X Nx..Nx X. NNx po (x Nx.NN X x)
(101)
1-1
X Xxxx
-x (x,,8)
Nx
Nx
po (
Va
X Y)
Po (X Y).
(102)
C EXPERIMENTAL SETUP
C.I COPYING TASK ON THE GRID DATASET
Data generation We generate 5 x 5 fully connected grids then noise them by flipping fixed portion of edges chosen at random. We set the portion of edges to be flipped to 5% of the total number of edges (i.e., for 5 5 grids, 31 edges get flipped).
Neural network We use the same neural network architecture for both the aligned and unaligned denoisers. We use graph transformer Dwivedi & Bresson (2021) with layers, attention head, and dimension 16 for the hidden layer. The activation function used is ReLu, and dropout rate of 0.1.
Training hyperparameters We train both models for 10 epochs, using Adam optimizer with a learning rate of 0.01. We set the batch size 32.
C.2 DATA: USPTO DATA SETS
All open-source data sets available for reaction modelling are derived in some form from the patent mining work of Lowe (2012). We distinguish subsets used in previous work: 15k, 50k, MIT, Sterco, and full (original data set). Table A3 provides key information about the subsets.
Table A3: UPSTO-50K subsets used in retrosynthesis
Subset
Introduced by
# of reactions
Preprocessing & Data split (script)
Full
Lowe (2012)
808 938
Dai et al. (2019)
Sterco
Schwaller et al. (2018)
002
Schwaller et al. (2019b)
MIT
Jin al. (2017)
479 035
50k
Schneider et al. (2016)
50
Dai et al. (2019)
is proposed by Coley et al. (2017a). The subset includes reactions covered by the 1.7k most common templates. All molecules appearing in the reaction are included to model the involvement of reagents and solvents despite not contributing with atoms to the product.
15k
50k is preprocessed by Schneider et al. (2016). The goal the analysis is to assign roles (reactant, reagent, solvent) to different participants in reaction through atom mapping. This effort led the
29

---

Published as a conference paper at ICLR 2025
creation of an atom- diom-mapped and classified subset of around 50k reactions, which is used nowadays as a benchmark for retrosynthesis tasks. It not clear how said subset was selected.
MIT is used by Jin et al. (2017). The preprocessing is described 'removing duplicate and erroneous reactions' with no further explanation of what qualifies as an erroneous reaction. The output of this filtering data set of 49k reactions (from original set of 1.8M reactions).
Stereo is proposed by Schwaller et al. (2018). The authors apply a more flexible filtering strategy compared to USPTO-MIT. Their data set only discards 800k reactions from the original data set because they are duplicates or they could not be canonicalized by RDKit. In addition, the data set only considers single-product reactions (92% of the full data set), opposed splitting multi- product reactions. The preprocessing steps include removing reagents (molecules with no atoms appearing in the product), removing hydrogen atoms from molecules, discarding atom-mapping information and canonicalizing molecules. In addition, since the original method applied to this subset a language model, tokenization is performed on the atoms.
Full is preprocessed by Dai et al. (2019). The processing includes removing duplicate reactions, splitting reactions with multiple products into multiple reactions with one product, removing reactant molecules appearing unchanged on the product side, removing all reactions with bad atom-mapping (i.e., when the sorted mapping between products and reactants is not one-to-one), and removing bad products (missing mapping, not parsed by Rdkit).
Our choice Similar to many other works on retrosynthesis, we use 50k as the main data set to evaluate our method.
C.3 NOTES ON OUR SAMPLING AND RANKING PROCEDURES
Duplicate removal Removing duplicates from the set of generated precursors is common method- ology in retrosynthesis, albeit often not discussed explicitly in papers. The benefit of duplicate re- moval is to ensure that an incorrect molecule that is nevertheless judged as the best one according to the ranking scheme does not fill up all of the top-k positions after ranking. While this does not affect t scores, not removing the duplicates would degrade the other top-k scores significantly.
Choice of scoring function Specifically, we use the following formula for approximating the likelihood of the sample under the model.
count X
cllbn(X)
s sXX (1 (1- (1 Ex count(X') +
(103)
Σx enlho(X')'
where count(.) returns the number of occurrences of sample X in the set of generated samples (by default, 100), clbo(.) computes the variational lower bound of the specific sample under the model, and is a weighting hyperparameter. The sums are taken over the set of generated samples. Intuitively, the idea is to provide an estimate of the likelihood from two routes. The ELBO is an estimate (a lower bound) log pa (X), and exponentiating and normalizing gives an estimate the probability distribution. Since the same reactants are often repeated in set of 100 samples, the counts can be used as a more direct proxy, although they inherently require relatively large amount of samples to limit the variance of this estimate.
We set the value of À to be 0.9, although we find that the top-k scores are not at all sensitive to variation in the exact value, long as it is below and the count information is used. Thus, the counts seem more important than the ELBO, which may be due to the lower bound nature of the ELBO or stochasticity in estimating its value. More accurate likelihood estimation schemes for diffusion models, such as exact likelihood values using the probability flow ODE (Song et al., 2021), could be a valuable direction for future research in the context of retrosynthetic diffusion models.
C.4 DETAILS ON STEREOCHEMMISRY
Our model does not explicitly consider changes in stereochemistry in the reaction, but instead, we use the atom mappings implicitly assigned to the samples by the model to transfer the chiral tags from the products
the reactants. The initial choice of PY--X at the start of sampling can be considered to be the atom mapping of the generated reactants, given that the model has been trained on correct atom mappings.
30

---

Published as a conference paper at ICLR 2025
For the chiral tags, we take the ground-truth SMILES for the product molecules from the dataset and assign the corresponding chiral tag to the corresponding atom mapping on the generated reactants. For cis/trans isomerism, use the Chem. rdchem. BondDir bond field in rdkit molecules and transfer them to the reactant side based on the atom mapping of the pair of atoms at the start and end of the bond.
Note that when using rdkit, transferring chirality requires some special www.rahhrrreeeeeee. CAI_TETRAHEDRAL are defined in the context of the order in
care: The chiral tags Chem. Typee CAI_TETRAHEERRLL..... which the bonds are attached the chiral atom in the molecule data structure. Thus, the chiral tag sometimes has to be flipped to retain the correct stereochemistry, based on whether the order of the bonds is different on the reactant molecule data structure and the product molecule data structure.
and
C.5 DETAILS OF THE EVALUATION PROCEDURE
Top-k scores We evaluate the top-k scores by ranking the list of generated and deduplicated samples and calculating the percentage of products for which the ground- -tuth reactants are in the first k elements in the list.
Mean reciprocal rank We formally define the MRR as MRR = [p (r- Verbally, it is the expected value of the inverse of the amount of reactant suggestions that the model makes before encountering the ground truth, and as such measures how early on is the correct reactant encountered in the ranked samples. It also incorporates the intuition that the difference between obtaining the correct reactants in, say, the 9th or the 10th position, is not as significant as the difference between the 1st and 2nd positions.
While we do not have direct access to the entire p(r) just based on the top-k scores, we can estimate with a uniform distribution assumption on r within the different top-k ranges. Formally, we define four sets S1, S2, Sz, S4 {1}. f2, 3), (4. 5), (6. 7, 8, 9, 10} in which p(r) is assumed to be uniform, and sr)) E {1, 2, 4} denotes the group that rank r belongs to. Top-k denoted as topa. where k E {1,3,5 5, 10) in our case. Note that they are also equal to the cumulative distribution
of p(r) until k. We thus define p(r) "Pmaa(( "0Pmae(G and p(1) top For the case
G.(oll
where the ground truth was not in the 10, we assume it is not recovered and place the rest of the probability mass p(oc). Our MRR estimate is then defined as
1 MRR- Σ p(r)
(104)
In cases where we do not have top-k values for all (1,3,510) (such as the Augmented Transformer in Table 2 for top-3), we assume that p(r) is constant in the wider interval between the preceding and following top-ks (2-5 in the case of the top-3 missing).
C.6 NEURAL NETWORK ARCHITECTURE, HYPERPARAMETEAS AND COMPUTE RESOURCES
We discuss here the neural network architecture and seper-parameters we choose. Our denoiser is implemented Graph Transformer (Dwivedi & Bresson, 2021), based on the implementation of Vignac al. (2023) with additional graph-level level features added to the input of the model. See Vignac al. (2023) for an in-depth discussion of the neural network.
In all our models, we use Graph Transformer layers. When using Laplacian positional encod- ings, we get the 20 eigenvectors of the Graph Laplacian matrix with the largest eigenvalues and assign to each node a 20-dimensional feature vector.
We a maximum of 15 'blank' nodes, in practice meaning that the models have the capacity to add 15 additional atoms on the reactant side. another detail, following Vignac et al. (2023), we weigh the edge components in the cross-entropy loss by factor of compared to the node components.
We used a batch size of 16 for the models where the expanded graph containing X and Y as sub- graphs is given input. These models were trained for approximately 600 epochs with a single A A100/V AMDDAMD MI250x GPU. For the model where aligmment is done by concatenating Y along the feature dimension in the input, the attention map sizes were smaller and could fit larger
31

---

Published as a conference paper at ICLR 2025
A reaction where bonds are broken and reorganised
Reactant suggestions from model
OH
Figure All: Left: An example reaction from USPTO-50k where the reactant is not possible to predict with a synthon-based model. Right: example reactants generated
by our model (includes the ground-truth one). The SMILES-string for the reaction is
ON=CICCc2oc3cacc3cco htc.:/cccccceaooccccccccccocc...
batch of 32 with single V100 GPU. This model was trained for 600 epochs. The training time for all of our models was approximately three days. In early experiments and developing the model, we trained or partially trained multiple models that did not make it to the main paper. Sampling 100 samples for one product with T = 100 from the model takes roughly 60 seconds with the current version of our code with an AMD MI250x GPU, and 100 samples with T = 10 takes correspond- ingly about seconds. is likely that the inference could be optimized, increasing the sample throughput.
The reported models were chosen based on evaluating different checkpoints with 10 diffusion steps on the validation set for different checkpoints and chose the best checkpoint based the MRR score.
D COMPARISION TO RETROSYNTHETIC BASELINES
Overview of retrosynthetic baselines There are three main types of retrosynthesis models (Liu et al., 2023). Template-based models depend the availability and quality of hard-coded chem- ical rules (Segler Waller, 2017: Xie et al., 2023). Synthon-based models are limited by their definition of reaction center, which does not necessarily hold for complex reactions (Yan et al., 2020; Shi et al., 2020a; Wang et al., 2023). See Fig. All for an example for synthon-based models but which scalable since they do not use any chemical assumptions in their design but perform suboptimally compared to template-basee methods on benchmarks like UPSTO-50k (Wan et al., 2022: Sco al., 2021). Efforts to bridge the gap between the template-based and template-free paradigms include methods investigating pretraining (Zhong al., 2022; Jiang al., 2023). Note that our method is in between template-foe free and template-basse based methods, since it uses stom-mapping information but not an explicit (and thus limited) set of templates. We put our models in their own category in the results table.
reaction impossible model gets correctly. Template-free methods the most
Methods with a different evaluation procedure Despite Igashov al. (2024)'s RetroBridge being a closely related to a diffusion model, we cannot include it in straightforward comparison because discards atom charges from the ground truth smiles during evaluation. Specifically, the model uses only atom types node features and compares the generated samples to the smiles reconstructed from the ground truth data through the same encoding (i.e., without charges too) 2. To compare uncharged atom-labels, and model to RetroBridge fairly, we trained our Dikk evaluated it without considering atom charge nor stereochemistry. The results are shown in App. D. Many recently published methods for retrosynthesis have capped out their top-1 scores in the 49%- 53% range (see our Table 2), indicating that the increase we achieve over RetroBridge's top-1 score (of over 5%) quite significant. Furthermore, the results we achieve are with 5 times less denoising steps (100 instead of 500). For closer look at the performance at different step counts, we invite the reader to compare Figure 5 in (Igashov et al., 2024) to our Figure Fig. The comparison shows that Retrobridge's 's top-1 score decreases to almost 0 at a step count more than 10. In contrast, our top-1 is close to 50% even at steps, and higher than Retrobridge at 100 steps. We attribute this improvement to our more careful analysis of alignment, and utilising multiple alignment techniques.
Thii can be seen in the code shared by Igashov et al. (2024): https: / /gg chub.comm/gaaheee
RetroBri ódg
32

---

Published as a conference paper at ICLR 2025
Table A4:
Comparison of top-k accuracy and MRR on the USPTO-50K test set without charges and without stereochemical information, following the evaluation setup of Igashov et al. (2024)
Model
k
k
3
k
5
k
10
MRR
RetroBridge (T=500)
50.8
74.1
80.6
85.6
0.622
DillAlign Pl=+Skip (T=100)
56.3
75.2
80.4
84.0
0.658
Table A5: Round-trip coverage and accuracy for different baselines. The methods are categorized as: template-based (TB), cnn-template-baser (NTB), and models performing their evaluation without taking into account formal charges (NC) nor stereochemistry (NS), We achieve the highest coverage and top-1 I accuracy among non-template-bes teaseate-baaeed methods. Our lower top-k>> accuracy may be due to our model generating a higher number of unique predictions compared to competitors, visualized in Figure Fig. A12.
Coverage (1)
Accuracy (t)
Model
k
1
k = 3
TB
GLN (Dai al., 2019)
82.5
92.0
94.0
82.5
71.0
66.2
MEGAN (Sacha et al., 2021)
78.1
88.6
91.3
78.1
67.3
61.7
Graph2SMILES (Tu & Coley, 2022)
76.7
56.0
46.4
NTB
Retroformet.ug (Wan et al., 2022)
78.6
71.8
67.1
DiffAliignPEE+skip (ours)
81.6
90.0
91.8
81.6
62.8
53.3
NS
LocalRetro (Chen & Jung, 2021)
82.1
92.3
94.7
82.1
71.0
66.7
RetroBridge (Igashov et al., 2024)
85.1
95.7
97.1
85.1
73.6
67.8
NS-NC
(ours)
87.6
96.4
97.6
87.6
69.3
59.1
Methods with pretraining Zhong et al. (2022) and Jiang et al. (2023) pre-train their models on the USPTO-FiU Full and Pistachio data sets, respectively, and as such the results are not directly comparable to models trained on the standard USPTO-50k benchmark. Pretraining with diffusion models is interesting direction for future research, but consider it outside the scope of work. Furthermore, comparison between models with different pretraining datasets and pretraining strategies has the danger of complicating comparisons, given that relative increases in performance could be explained by the model, the pretraining strategy, or the pretraining dataset. As such, we believe that standardized benchmarks like USPTO-50K are necessary when researching modelling strategies.
Another commonly used metric used evaluate retrosynthesis model is round-trip accuracy, in which a forward prediction model is used to evaluate whether our samples can indeed produce the input product. Coverage considers a sample correct if matches exactly the ground truth or deemed suitable by the forward prediction oracle. Accuracy counts the percentage of valid precur- sors over all generated samples. We use the same prediction model as previous work, i.c., Molecular Transformer (Schwaller et al., 2019a). As can be seen from Table AS our model outperforms all non-template-bant based baselines on all thresholds. We also highlight a known tradeoff between diver- sity and accuracy of generation in Fig. A12, which explains partly why accuracy is lower than other baselines (we compare to samples from RetroBridge (Igashov et al., 2024) particular as an illustration).
33

---

Published as a conference paper at ICLR 2025
DiffAllinn-E+sskip*
2500
Retrobridge
2000
# of products
1500
1000
500
20
60
100
120
of unique predictions per product
Figure A12: We compare the diversity (number of unique predictions per product) of our samples to RetroBridge (Igashov et al., 2024). is important for a retrosynthetic model to generate diverse precursors, thus offering the practitioner with multiple synthesis strategies, especially given the limited accuracy of forward prediction models.
E ADDITIONAL ABLATIONS
Disentangling the effect of the inductive biases due to graph positional encodings vs. the induc- tive bias of alignment. To disentangle the effect the graph inductive bias brought by the graph Laplacian positional encodings and the alignment inductive bias brought by matching the positional encodings for the input and output graph, we trained two models:
1. A model where the only alignment method is the matched PE where the PE is gener- ated from the graph Laplacian eigenvectors the conditioning graph Y (our default PE method).
2. A model where the only alignment method is matched PEs where the PE is generated by sampling random Gaussian noise vectors on the conditioning graph nodes Y and placing these same vectors the X side.
Both contain aligmment information, but the latter does not contain the additional inductive bias brought by the graph positional encoding. We show the results at roughly 10% of the full training budget for the results in Table 2. The results are listed in Table A6. The results are not signifi- cantly different, indicating that the inductive bias brought by the graph positional encodings is not as significant as the inductive bias alignment.
Different transitions. (Vignac et al., 2023) did not originally consider the absorbing state transi- tions we use. Instead, they used the uniform and marginal transitions. With the uniform transitions, each step provides non-zero chance of transitioning from state any of the others. This verges to an uniform distribution over the node and edge states at Xr. With the marginal transitions, these transitions distribution of the node and edge types in the original data. Since there are much more no-edges than edges in the data, this encodes sparsity to the graphs in p(Xx).
biased such that the distribution of nodes and edges in p(X,) is the marginal
To provide a fair comparison, we compare our absorbing state model and these models on roughly 10% of the full training budget (80 epochs), and list the results for 512 samples from the test set in
34

---

Published as a conference paper at ICLR 2025
Model
Top-1
Top-3
Top-5
Top-10
Top-50
MRR
DiffAlignnPE-Gaussii 45.51
68.55
73.24
75.78
83.20
0.5649
DiffAAiign-PE-Laalacian
46.09
68.75
72.85
75.98
83.98
0.5686
Table A6: Performance comparison of the model with the only alignment method being the matched Gaussian noise encoding versus a model with the only alignment method being the matched Lapla- cian eigenvector- based positional encoding. Both contain alignment information, but the first does not contain additional inductive biases about the structure of the conditioning graph Y. Results are obtained with 512 samples form the USPTO-50k test set and 10% of the full training budget used in Table 2.
Model
Top-1
Top-3
Top-5
Top-10 10
Top-50
MRR
Uniform- Uniform-PE+Skip
43.75
58.98
62.89
67.38
72.66
0.5156
Marginal-PE+SSe
44.73
60.94
63.28
66.60
69.92
0.5244
49.22
70.90
74.61
77.93
84.96
0.5952
Table A7: Performance comparison of the PE+Skip models with different transition matrices, eval-
uated with 512 samples form the USPTO-50k test set and 10% of the full training budget.
Table A7. The absorbing-state model performs the best, whereas the marginal and uniform models perform slightly worse. The marginal transitions somewhat outperform the uniform transitions.
Comparison to only using the skip connections. Here, we provide a comparison to model that only uses the skip connections, and other alignment method. The results are again shown for roughly 10% of the training budget (epoch (80) for the PE+Skip model and for the Skip-only model in Table A8. It is evident that only having the skip connection does not work well, although is better than an entirely unaligned model (see Table 2). This is likely due to the skip-only model having limited expressivity: The neural net does not get the alignment information in the input all, and thus is unable to use it apart from the skip connection, which enables literally copying the Y graph from input output.
F ADDING POST-TRAINING CONDITIONING TO DISCRETE DIFFUSION MODELS
In this section, we show method to add additional controls and conditions on the used discrete diffusion model post-training. Note that this approach differs from (Vignac et al., 2023)'s in that our method is an adaptation of reconstruction guidance (Ho et al., 2022; Chung et al., 2023; Song al., 2023a), while they adapt classifier guuidaneerr (Sohl-Dickstein et al., 2015; Dhariwal & Nichol, 2021; Song et al., 2021) for their conditional model. While the notation is from the point of view of our graph-to-graph translation, the method here applies in general any discrete diffusion model. We start out by following (Vignac et al., 2023) and (Sohl-Dickstedd et al., 2015; Dhariwal & Nichol, 2021), and then make the novel connection to reconstruction guidance. We write Bayes' rule for an additional condition (e.g., specified level of drug-likeness or synthesizability, or an inpainting mask)
Po X X,, Y, y) a p(U X,,1, (X, X,, Y)
(105)
ply X,-1, (X, (X,-1 X, Y)
(106)
where the second equation was due to the Markovian structure of the generative process (X,-1 d-separates and X,). Now, we can take the log and interpret the probabilities as tensors
Model
Top-1
Top-3
Top-5
Top-10 10
Top-50
MRR
Diif--lignSSipp
9.77
17.38
24.61
29.30
32,23
0.1517
DiffAllig-PE+SSee (Laplacian)
49.22
70.90
74.61
77.93
84.96
0.5952
Table A8: Performance comparison for model with the only alignment method being skip connec- tions and a model where we also use the Laplacian positional encodings, evaluated with 512 samples form the USPTO-50k test set and 10% of the full training budget
35

---

Published as a conference paper at ICLR 2025
Po X, Y) and Po(X, X2, Y) defined in the same space as the one-hot valued tensors X(-1 and X,. We get:
log Po X Y, y) log Ply Xx.1, Y) + log Pe PX X Y)
(107)
Similarly to (Vignac et al., 2023), we can now Taylor expand log Po(y X Y) around X with
P(y X,-1, Y) ~ log P(y X., Vx log P(y X,, Y) Ix (X, - X,)
(108)
Given that we are interested in the distribution w.r.t. X,-1, the X terms are constant when we plug them in to Eq. (107), resulting in
log Po (X,X, X,, ,) x Vx; log P(y X xI X, + log Po(X,-1(X, Y)
(109)
Simplifying the notation to assume taking the gradient at X,, we can also write
log Po X,, Y, v) x Vx, Po(y X,, Y)X,- + log Px X,, Y)
(110)
In practice, the equation means that given log Po X, Y), we get pa (X, X,, Y. y) by adding the input gradient of log po (y) X,, Y) to the logits given by the regular reverse transition and re-
normalizing.
would possible to approximate log (y X,, Y) by training an additional classifier, leading to classifier guidance (Sohl-Dickstein et al., 2015; Dhariwal & Nichol, 2021; Song et al., 2021) and the exact method presented in (Vignac et al., 2023). We go further by adapting these formulas to reconstruction guidance (Ho et al., 2022: Chung et al., 2023; Song al., 2023a). These methods and more advanced versions (Dou & Song, 2024; Finzi et al., 2023; Boys et al., 2023; Wu et al., 2023; Peng et al., 2024) provide different levels of approximations of the true conditional distribution. Here, we show an approach particularly similar to (Chung et al., 2023), by approximating the true denoising distribution p(Xa|Xx |X,, Y) directly with the denoiser output
p(u X, Y) = Σ
P(Xo X,, Y)p(y Xx)
(111)
X
(Xo Xx, Y Y)p(y Xo).
(112)
X
Here p(Xo X., Y) is the true, intractable distribution given by going through the entire sampling process and p(X, X,, Y) is the factorized distribution that is given by the denoiser output, and 'jumps to Xo directly. This results in the following update step:
log (X X., y, Vx, log (Epa(XX) (X. X))(l Xo)) X.-1 + log Po( Xe, Y).
(113)
Summing over all possible graphs Xo is prohibitive, however. Instead, we could sample X. from (Xo Xx, Y) with the Gumbel-Softmax trick (Jang et al., 2016) and evaluate log p(y) Xo). As long log ply Xo) is differentiable,
then just use automatic differentiation to get timate of V x. log Po(y X,, Y). Another, more simplified approach that avoids sampling from (X,, X, Y) is relax the definition of the likelihood function to directly condition the continuous- valued probability vector Pe(X, Xo Xx, Y) instead of the discrete-valued Xg. For sim- plicity, we adopted this approach, but the full method with more difficult to implement. The algorithm for sampling is shown in Alg. 3.
is not significantly
Toy Synthesisability Model: Controlling Atom Economy
The model of synthesizability that we use is
X. = po Xx, Y), ply synthesizable Xo) a( Eies Xo.sd
(114)
y
(115)
b
where is the set of non-atom- maroped nodes, a, and y are constants that define the synthesizability model and d refers to the dummy node index. The intuition is that the more nodes are classified as dummy nodes (non-atoms), the fewer atoms we have in total, leaving the atom economy higher. Note that Eies Xo the expected amount of dummy nodes from po (Xo) X,, Y). We set
36

---

Published as a conference paper at ICLR 2025
Algorithm 3 Sampling with atom count guidance
Input: Product Y
Choose: PY -X E IR Ny Xxr p(X)))
for f T to do
X. Dx(X,, Y, PY
Denoising output bo Regular reverse transition probabilities
X: - - Ex (X;- X X
x softmax (log X + Xvx log o Eies Son. Xona--))
> Renormalize
Return X.
to half the amount of dummy nodes and b one-quarter of the amount of dummy nodes. It turns out that this leaves as a useful parameter to tune the sharpness the conditioning. The gradient estimate is then given by
Vx, log Po(y X, Y) VV,, log o Eies Xo,id )0
(116)
which can be directly calculated with automatic differentiation.
G HANDLING NOISY NODE MAPPINGS FOR THE TOY DATA
Sec. 4.1 introduced the graph copying toy data set. Here, extend it to analyse the effect of imperfect node mapping during training. In particular, we consider scheme for adding 'noise" to the node mapping by swapping them with each other on either the conditioning, or equivalently, the other side. We do it as follows:
1. Sample the number of pair swaps: f ~~ Poisson(A)
2. For each of the swaps, randomly select two nodes i, in the graph A and swap their mappings
The Poisson parameter controls the expected number of swaps, allowing us to tnne the level of noise in the node mapping. We trained small models for 1000 steps of training, and show the results with different noise levels in Fig. A13. Small errors the node mapping are evidently not significant issue, but larger À does start affecting performance, at least in this early training phase. We also include a measure of similarity to the target graph: The precision is the fraction of edges that were correctly inferred, compared to the ground-tree truth target.
H DETAILS FOR THE IBUPROFEN SYNTHESIS EXPERIMENT
Below we explain the synthesis steps visualized in Fig. 7 in detail:
1. The retrosynthesis begins with carbonylation, adding a 'CO' structure. Initial basic gen- eration yields unpromising results, the practitioner suggests partial reactant structure, leading to more viable path.
2. The model then proposes hydrogenation, logical next step. While H2 molecules aren't explicitly represented in the data, they're re inferred from the context of the reaction.
3. For the third step. the practitioner identifies an opportunity for acylation (involving the C (-0) group), potentially leading readily available reactants. Given this (-O) structure, the model successfully completes the reaction. These steps align with the syn- thesis plan proposed by BHC.
4. The retrosynthesis path starts with carbonylation, a well-known synthetic reaction which adds 'CO' structure notices that the suggested reactant not promising. They then suggest a partial structure of the reactants which leads to more sensible path.
compound. The practitioner tries basic generation but then
37

---

Published as a conference paper at ICLR 2025
Original target graph
Adjacency Matrix
10-
15
20-
10
11
20
Precision: 0.931
Adjacency Matrix
Precislon:
Adjacency
10
20
20
Polsson Adjecency
Adjacency
Figure A13: Results from training an aligned model for 1000 steps in the graph copying task presented in Sec. 4.1, with different levels of noise in the atom mappings. À parameterizes a Poisson distribution that defines the level of mistakes the node mapping. On lower noise levels, the results do not significantly change. However, there seems to be limit in which the neural net training dynamics changes to produce suboptimal results.
38

---

Published as a conference paper at ICLR 2025
0.9
0.0
0.6
0.5
Poisson A=0.0
Poisson 2-0.01 0.01
0.3
Poisson A=0.015
Poisson 2-0.02
0.2
10
20 25
Figure A14: The precision of the adjacency matrix estimates from aligned model across training, with different levels of node mapping noise A. Going above À > 0.02 nudges the training dynamics perform poorly this task.
5. Next, our model proposes hydrogenation, which is sensible suggestion in this case. The data does not handle explicitly individual H2 molecules, but they are inferred from the context.
6. In the third step, the practitioner notices that an acylation reaction (a reaction with the C (=0) group) might lead to reactants that readily available. The model is able to complete the rest of the reaction after knowing that C(=O)C is present. These steps match the synthesis plan proposed by BHC.
39


Reviews:
[{"review_id": "SYJjKtePE2", "reviewer": "Reviewer_uWes", "summary": "This paper presents a novel approach to graph-to-graph translation using diffusion models. The authors propose methods to address the inherent limitations of equivariant models in aligning nodes between input and output graphs. The paper introduces several strategies for breaking equivariance and empirically validates their approach on retrosynthesis tasks. While the paper is innovative in addressing this important problem, certain theoretical justifications and experimental evaluations could be strengthened.", "questions": "Please refer to  the weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper addresses a novel and relevant problem in graph diffusion literature, particularly in breaking symmetries for graph-to-graph translation, which has not been explored in depth.\n- The idea of using equivariant models and tackling their limitations is well-motivated, and the theoretical underpinning is clear and compelling.\n- The alignment techniques proposed for mapping nodes between input and output graphs are intuitive and demonstrate promising preliminary results, especially compared to unaligned versions.", "weaknesses": "- The justification for certain design choices, such as using the absorbing state distribution instead of marginal distributions (as in DiGress), is insufficiently explained. A more thorough motivation or comparison would be beneficial.\n- Some key claims, particularly in Section 3.2, are vague or lack detailed explanation. For example, the statement regarding the model's capability to handle graph transformations requires clearer elaboration.\n- The experimental validation, while promising, lacks sufficient quantitative comparisons to existing methods (e.g., RetroBridge). More thorough benchmarking and analysis would strengthen the empirical contribution of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0EPgkQ3HnO", "reviewer": "Reviewer_Gt4F", "summary": "The central objective of this paper is to design a discrete denoising diffusion model for graph-to-graph translation, where the task is to train a model to predict a target graph given a source graph and, perhaps surprisingly, a matrix of node mappings between them. The authors use [1] as their design template, though the state transition matrix of the forward process relies on the absorbing state formulation from [2]. In this model, the backward process relies on the denoiser network realized by an equivariant transformer architecture, which, on its own, can take only the target graph as input. The authors' main challenge is augmenting the denoiser by also taking the source graph as input. The authors demonstrate theoretically and empirically that a simple augmentation of the denoiser's input by the source graph is not sufficient even for simply copying graphs, claiming that the equivariance of the denoiser is the main reason for this issue. Therefore, the authors propose to partially relax the equivariance by further extending the denoiser's input with the node mapping matrix, proposing the aligned permutation equivariance. The key idea is to design the underlying transformer architecture such that if the source and target graphs are permuted (by two separate permutation matrices) and the node mapping matrix is permuted accordingly, then the output of the denoiser is equivariant only under the permutations of the target graph. In other words, the output remains sensitive only to the permutation of the target graph. The authors compare the performance of their model with many baseline methods on the task of chemical retrosynthesis, achieving state-of-the-art top-1 accuracy and mean reciprocal rank on the USPTO-50k dataset. Finally, they also show the suitability of their model in the context of the guided and conditional generation of molecular graphs.", "questions": "Do the authors need the node mapping matrix for this task?\n\nCan the authors explain why the left part of (6) should hold (in light of the above concerns with the output of the denoising network $D_\\theta$ and the one-hot encoding of $X$)?\n\nWhy do the authors present Theorem 1 for fixed t=T? Is the result for $X_T$ not rather obvious (by definition)?\n\nIf the authors use $P^{Y\\rightarrow X}$ and $Y$ as input into the denoiser, then these two terms already form the full information about $X$. The authors seem to train the denoiser that always has direct access to $X\\_0$ through $P^{Y\\rightarrow X}Y$. The consequence of this is that $\\tilde{p}\\_\\theta$ in (3) contains $P^{Y\\rightarrow X}$ in the condition, i.e., $\\tilde{p}\\_\\theta(X\\_0|X\\_t,Y,P^{Y\\rightarrow X})=\\tilde{p}_\\theta(X\\_0|X\\_t,X\\_0)$. This design choice is unsound. Please, can the authors clarify?\n\nCan the authors please provide a more comprehensive explanation of the following comment (starting in line 227): *``This makes it impossible for the model ... some random permutation of $Y$.''*?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper presents a comprehensive set of experiments that demonstrate the benefits and the practical utility of the proposed model in the context of guided and conditional generation of molecular graphs. The model achieves state-of-the-art results in chemical retrosynthesis.\n\nThe paper concerns an interesting problem, and its central message is important to the community.\n\nNice and clear illustrations accompany the ideas introduced in the paper.", "weaknesses": "The writing deserves improvement. Certain parts of the paper need to be communicated more clearly. The notation sometimes needs to be clarified. These concerns are detailed in the following comments.\n\nMajor:\n\nThe content from the start of Section 3 up to Section 3.2 qualifies more like a background section. The material is heavily inspired by [1]. Please note that [1] also uses conditioning, though it is not presented in the equations in such an explicit way. It takes more work to recognize the details behind the authors' contributions in the current form. Can the authors clearly delineate which parts are background and which are novel contributions?\n\nIn line 124 the authors define $X=P^{Y\\rightarrow X}Y$ as $(X^N,X^E)=(P^{Y\\rightarrow X}Y^N,P^{Y\\rightarrow X}Y^E(P^{Y\\rightarrow X})^T)$, and in line 151 define $D_\\theta(X,Y)=(D_\\theta(X,Y)^N,D_\\theta(X,Y)^E)$, saying that $D_\\theta(X,Y)^N$ and $D_\\theta(X,Y)^E$ output a probability vector for each node and edge, respectively. Now, in line 107, the authors say that $X^N$ and $X^E$ are one-hot encoded (again, it is not suitable to use $\\mathbb{R}$ in lines 104 and 105 to define this), then how is it possible that $D_\\theta(X,Y)=P^{Y\\rightarrow X}Y$? There is no mention that the output of $D_\\theta(X,Y)$ is implicitly one-hot encoded. This confusion repeats throughout the text, and clear definitions should stabilize it. The authors should\n\n- clarify when and where $X^N$ and $X^E$ are one-hot encoded or real-valued matrices/tensors,\n- explicitly define the output space of $D_\\theta(X,Y)$,\n- explain how the equality $D_\\theta(X,Y)=P^{Y\\rightarrow X}Y$ is possible given the different encodings,\n- use consistent notation throughout the paper for the spaces these variables belong to.\n\nMinor:\n\nThe introduction should better motivate why graph-to-graph translation is important in practical applications.\n\nLine 50: *``a solution aligned''* -> *``a solution: aligned''*\n\nFigure 1: *``w.r.t to''* -> *``w.r.t.''*\n\nIn Section 3.3, the authors use phrases such as *``should be possible''*, *``should have learned''*, *``should be the same''*. Why do not use a more certain language? Can the authors please clarify what is or is not true instead of something that should be?\n\nThe definition of graph-to-graph translation task deserves improvements. Specifically, if the authors define that the node feature matrix and edge feature tensor belong to the set of reals, why do they redefine them as one-hot vectors in line 107? If they are one-hot, then they are only a subset of reals.\n\nThere should be $Y$ on the left side of (2). Consequently, how does this density differ from (3)?\n\nLine 153: *``such that we have a probability vector for each node and edge.''* Probability is not the whole set of reals. It only ranges from 0 to 1.\n\nLine 217: The use of *`respectively'* does not fit the sentence's construction.\n\nLine 243: $\\mathbf{X}=\\mathbf{P}^{\\mathbf{Y}\\rightarrow \\mathbf{X}}Y$ -> $\\mathbf{X}=\\mathbf{P}^{\\mathbf{Y}\\rightarrow \\mathbf{X}}\\mathbf{Y}$\n\nLine 323: What is DPS?\n\nLine 374: What is MRR? It is first defined in line 402.\n\nThe explanation of retrosynthesis in lines 337 and 358 is nearly identical.\n\nLine 337: *``3 parts\"* -> *``three parts''*\n\nTable A3 should be in the main text (e.g., Figure 8 can be moved to the appendix).\n\nFigure 3 demonstrates the case for $X_T$, yet there is $D_\\theta(X_t,Y)$.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5jjVfUX1i7", "reviewer": "Reviewer_gPTC", "summary": "The paper presents a novel method to perform graph-to-graph translation using graph diffusion models. The authors show why equivariant models are inherently limited in their ability to perform this task when nodes from the prior distribution need to be aligned with nodes from the output. To that purpose, they demonstrate that an ideal equivariant denoiser cannot correctly copy a graph. Then, they propose several ways to break equivariance for the nodes that should be aligned between the input and the output, and empirically validate their method on the retrosynthesis task.", "questions": "- What is the difference between $D$ and $\\tilde p_{\\theta}$ ?\n- How is $\\psi$ generated in section 3.5 ?\n- How is Y passed to the denoiser ? I feel it's not clearly outlined anywhere except in the section \"Aligning Y in the input\"\n- I don't get how you build the dataset for retrosynthesis. If it is standard benchmark, could you please provide more explanations or point out to a reference", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- To the best of my knowledge, the topic of breaking symmetries to perform graph-to-graph translation has not yet been addressed in the graph diffusion literature.\n- The problem of using equivariant models is clearly identified and backed by theoritical evidences.\n- The paper proposed several simple ways to solve this problem.", "weaknesses": "- Why using the aborbing state distribution and not the marginal distributions introduced in DiGress (Vignac et al., 2022) ?\n\n- I would avoid statements like \"it is easy to see\" l304 or \"Clearly\" l242 and provide instead clear motivations and explanations\n\n- Some claims or design choices lack motivation or justification :\n- For example, the claims in section 3.2 \"a model that is capable of copying graphs from one side of the reaction\nto the other should also be extendable to modulations of this task\". Do you mean that it's a prerequisite or that is implies that the model has the ability to peform \"the modoluations\". Overall, I fell that this subsection conveys very little information.\n- The proposed aligning methods are insufficiently motivated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ymWQGqckmo", "reviewer": "Reviewer_HVWS", "summary": "This paper proposes an aligned permutation equivariant denoiser for a discrete diffusion model to address the graph translation problem. To align the initial state and target, several techniques, including nodal positional encoding, skip connections, and input alignment, are introduced. The model was validated on retrosynthesis, guided generation, and inpainting tasks.", "questions": "1. The paper assumes that the prior distribution is absorbing state. Does the proposed model still effective even on the uniform distribution prior?\n\n2. What happens if the node-mapping matrix has uncertainty and is not a binary matrix?\n\n3. How does the performance compare when using only direct skip connections without positional encoding?\n\n4. How does the performance differ from graph translation methods in Related Works, such as G2gt?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Aligning initial and target to generate efficiently makes sense and the results are promising compared to unaligned version.", "weaknesses": "Since alignment itself is an empirical approach that has been used previously, the contribution may be limited to explaining why alignment is necessary specifically in a discrete diffusion model with an absorbing state as the prior. When compared to RetroBridge in the retrosynthesis task, it is difficult to find a significant performance gap between the proposed model and RetroBridge. In the cases of guided diffusion and inpainting, only a few figures are provided without numerical comparisons, making precise validation difficult.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)