WyckoffDiff -- A Generative Diffusion Model for Crystal Symmetry
Authors: Filip Ekström Kelvinius
Oskar B. Andersson
Abhijith S Parackal
Dong Qian
Rickard Armiento
Fredrik Lindsten
Venue: ICML 2025 poster (2025)
Keywords: generative modeling
materials science
materials generation
diffusion models
discrete diffusion models
Wyckoff
Primary area: applications->chemistry_physics_and_earth_sciences
TL;DR: A generative diffusion model for generation of symmetry descriptions of materials

Abstract: Crystalline materials often exhibit a high level of symmetry. However, most generative models do not account for symmetry, but rather model each atom without any constraints on its position or element. We propose a generative model, Wyckoff Diffusion (WyckoffDiff), which generates symmetry-based descriptions of crystals. This is enabled by considering a crystal structure representation that encodes all symmetry, and we design a novel neural network architecture which enables using this representation inside a discrete generative model framework. In addition to respecting symmetry by construction, the discrete nature of our model enables fast generation. We additionally present a new metric, Fréchet Wrenformer Distance, which captures the symmetry aspects of the materials generated, and we benchmark WyckoffDiff against recently proposed generative models for crystal generation. As a proof-of-concept study, we use WyckoffDiff to find new materials below the convex hull of thermodynamical stability.

WyckoffDiff - -A Generative Diffusion Model for Crystal Symmetry
Filip Ekström Kelvinius'
Oskar B. Andersson 2 Abhijith S. Parackal 2 Dong Qian Rickard Armiento 2
Fredrik Lindsten
Abstract
This is an aspect that only recently has been built into gener- ative models (Jiao et al., 2024; Zhu et al., 2024; Levy et al., 2025). Instead, models without any built-in mechanisms that ensure symmetry in materials have and are still being developed (Xie et al., 2022; Jiao et al., 2023; Merchant et al., 2023; Zeni et al., 2025). As demonstrated by several works (Levy et al., 2025; Cheetham & Seshadri, 2024; Zeni et al., 2025), materials generated from methods without these ex- plicit constraints often lack the symmetrical characteristics of materials found databases. For example, Cheetham & Seshadri (2024) find that roughly 34 % of the materials generated by the GNoME model (Merchant et al., 2023) be- long to four different space groups of which only one exists in the Inorganic Crystal Structure Database (Belsky et al., 2002) where it makes up only 1 %, and Zeni et al. (2025) mention that their MatterGen model tends to generate less symmetric structures than are present in the training data.
Crystalline materials often exhibit a high level of symmetry. However, most generative models do not account for symmetry, but rather model each atom without any constraints on its position or element. We propose a generative model, Wyck- off Diffusion (WYCKOFDIIFF) which generates symmetry-based descriptions of erystals. This is enabled by considering a crystal structure rep- resentation that encodes all symmetry, and we design a novel neural network architecture which enables using this representation inside discrete generative model framework. In addition to re- specting symmetry by construction, the discrete nature of our model enables fast generation. We additionally present a new metric, Fréchet Wren- former Distance, which captures the symmetry aspects of the materials generated, and we bench- mark WYCKOFFDIFF against recently proposed generative models for crystal generation. As a proof-of-concept study, we use WYCKOFFDIFFF to find new materials below the convex hull of thermodynamical stability.
The symmetry of material can be encoded in protostruc- ture description (Parackal et al., 2024, see also Section 2.1), where elements occupy Wyckoff positions in crystal struc- tures categorized into space groups. This description avoids specifying the exact atomic coordinates, while maintain- ing the key structural information, which has been shown to be efficient for searching for novel stable materials by enabling initial step where candidate crystal structures with high likelihood of being stable are identified based on the symmetry description alone. This step avoids wasting computational resources on exact coordinate calculations across all possible materials (Goodall et al., 2022). Addi- tionally, the infinite space of continuous coordinates also opens the risk of generating degenerate materials or struc- tures outside of the symmetry proximity. Since materials of high symmetry are generally the interesting materials to ex- plore, generation of large sets of low symmetry materials is inefficient. Explicitly encoding symmetry could allow a gen- erative model to only generate within a space of interesting materials of higher symmetry, allowing a symmetry-ine innfuud generative model generate broader variety of relevant crystalline materials compared a generative model using exact coordinate representations.
1. Introduction
Materials science is a field of research that is essential for technological advancement. With machine learning seeing success in a variety of fields, materials science is no excep- tion. In the search for new materials, so called generative models are an attractive class of methods, and number of models that can generate new materials have been devel- oped (see, e.g., Park et al., 2024, for an overview), However, crystallime materials are often characterized by their specific symmetries, which are integral to their materials properties.
'Department of Computer and Information Science (IDA), Linköping University. Sweden "Department of Physics, Chem- istry and Biology (IFM), Linköping University. Sweden. Corre- spondence to: Filip Ekström Kelvinius Oskar B. Andersson @ liu.s,, askar..nderssonee liu.se>.
Explicitly enforcing knowledge about symmetry in gener- ative models for crystal structures is currently an underex- plored research direction. Our approach is different from
Proceedings of the 4nnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

WyckoffDiff- A Generative Diffusion Model for Crystal Symmetry
X
Po po (Xe)
0.1
0.1
0.0
0.05
0.0
n2-4a
62-4h
62-44
62 4b
3
1
0
0...
0.01
0.3
0.
0.0
0.33
0
0.2
0
0
62-44
62-R4
0
0,05
0.2
0.02
0.0
K2-Bd
0.1
0.4
0
0.12
0.1
0.1
0.5
0.2
0.05
0.01
Figure/ 1. Illustration of the (graph) representation of a material used in our generative model. A material of space group 62 has four Wyckoff Positions (a, b, e, d). Two them (a and b, dark blue) has the constraint that at most one alom can occupy the position, and we hence model that as single variable indicating which atom type that occupies the corresponding position (2 denoting no atom). For the other two positions (c and d, light blue), any number of atoms can occupy the position, and we hence model this as a set of variables, one for each atom type, which indicates how many of the respective atom types that occupying the position. To the left is the state of the material at some sampling time t, and to the right is the prediction of the "clean" material Xa made by the neural network. For all variables, there is corresponding row in the figure, corresponding to probability vectors, and all rows hence sum to 1.
previous works in how we specifically target the genera- tion of protostructures using representation that enables the use of generative models for discrete data to generate new materials. Our method shows competitive performance against other methods on various quantitative metrics. The generated protostructures can be used as part of a machine- learning based workflow for materials discovery to find new stable crystal structures. As a proof of concept, we realize subset of the generated protostructures into crystal struc- tures and from this set we highlight some examples with interesting and varied chemistries (CsSnFe. NaNbO2, and CapPI) which are on or below the currently known convex hull of thermodynamically stable compounds. Data and code is available online'.
ement from the periodic table of elements, can then occupy called Wyckoff position in the crystal structure, which represents sets of points which the symmetry operators act in a specific way. Hence, if an atom is specified to sit at a specific Wyckoff position, depending on the nature of that Wyckoff position, this declares it to reside exactly at a specific point; anywhere along a line; in a plane; or in a volume, and the symmetry operators then imply that equiva- lent atoms sit at a number (the nultiplicity) of other points in the unit cell, called the orbit. These different Wyckoff positions are labeled using a letter from the Latin alphabet (a. b, c, etc.). The space group completely determines which Wyckoff positions that are available, as tabulated by The Volume of International Tables for Crystallography (IUCr, 2002).
2. Background
In this work, we use the term prototype as defined for AFLOW prototype labels (Mehl et al., 2017), i.e., the combi- nation of the spacegroup and how the Wyckoff positions are occupied by unspecified but distinct elements, without addi- tional information about the remaining degrees of freedom for those occupied positions. In more detail, the AFLOW prototype label ABC6_hR24 122..666.8 a b h specifies first the 66
2.1. Representing Crystals
An ideal crystalline material is commonly represented by its crystal structure as an infinitely repeating set of unit cells with atoms of specified chemical elements placed at specific atomic positions. In the unit cell, the M atoms are specified by their positions X E M and elements E ZMM and the geometry of the unit cell can be specified by three lattice vectors L 3x333, As an alternative, one can separately specify the symmetry of the atomic positions, and then specify the atomic coordinates only by precise values for the remaining degrees of freedom. This representation is discussed in the following.
anonymous composition AB6C (i.e., AB&C), then the Pear- son symbol hR24, followed by the spacegroup number 66, and a list of Wyckoff labels for the positions occu- pied by the distinct elements in the anonymous formula, a b. (i.e., positions a, h, and b). Furthermore, following Parackal et al. (2024) we use the term protostructure to refer to a prototype where specific chemical elements are assigned to the Wyckoff positions (but where the degrees of freedom of the structure remains unspecified). Proto- structures can be labeled by extended AFLOW prototype labels, e.g., AB6C hB6C_hR24 166..66 6_.a.ll h b.b. Cs-F-Sn, to indi- cate that the previously anonymous elements A, B, and C
Protostructures All possible combinations of symme- tries of crystal structures can be categorized into 230 space groups (Müller et al., 2013). The atoms, each chemical el-
"https://giil
2

---

Wyckof[Diff- - A Generative Diffusion Model for Crystal Symmetry
are Cs-F-Sn (Cs, F, Sn), which occupy the spacegroup 166 Wyckoff positions a h b (a, h, b)².
randomly assigned according the decoded composition. The diffusion process then consists of denoising the posi- tions and elements, conditioned on 2, while keeping L fixed during the full process. The positions and atoms are updated without any explicit or built-in constraints with respect to symmetries.
2.2. Diffusion Models
Diffusion models (Sohl-Dickstein et al., 2015; Ho et al., 2020; Song et al., 2021) are type of generative models that have received tremendous interest lately. In essence, they are based on the idea of starting from a pure noise sample Xr, which is iteratively "denoised" to end up with a "clean" sample Xg. This denoising is enabled by viewing the data-to-noise (forward) process as a fixed Markov chain
DP and DifTCSP++ DifICSP (Jiao et al., 2023) builds upon CDVAE by replacing the VAE with a dif- fusion model that jointly learns the lattice and coordi- nates, enabling more precise modeling of crystal geometry. DifICSP++ (Jiao et al., 2024) further incorporates space group symmetry by leveraging pre-defined structural tem- plates from the training data to learn atomic types and coor- dinates aligned with these templates. However, this might limit the diversity and novelty of the generated materials.
T-1
g(xo:T) = ((x)) II (xx+1)xx), 1=0
(1)
where ((xo) is the data distribution and the transitions ((xe+1 (xx) are designed such that, for large T, q(xT) con- verges to distribution p(xr) from which we can easily sample, like a Gaussian distribution in case of continuous variables. The reverse process is then parametrized as
SymmCD To address this limitation, SymmCD (Levy et al., 2025) introduces a thysically-motivatee representa- tion of symmetries binary matrices, enabling efficient information-sharing and generalization across both crystal and site symmetries. SymmCD is related to our work in the sense that it also generates Wyckoff positions, but the approach is conceptually different: it start by sampling a number M of "representative" orbits, and then the element and the aforementioned binary representations of these are generated. We, on the other hand, will "start" from all Wyck- off positions and then generate which and how many of each element (if any) occupy each position.
T-1
(x(x)))) = p(XT) II po (x++)))
where po(x (x+1) are fitted such that po (x F q(x, (x+4)). ). Sampling according to the reverse process will then give (approximate) samples from the data distribution ((xo).
While most diffusion models have been developed for con- tinuous data, there are also several methods designed for the discrete case (e.g., Hoogeboom et al., 2021: Austin et al., 2021; Campbell et al., 2022; Sun et al., 2023; Lou et al., 2024). Conceptually, the idea is the same, but the transitions (both in the forward and backward directions) operate on discrete state-spaces and the limiting distribution p(xr) is typically chosen to factorize over the components of xT to enable easy sampling. In this work we make explicit use of the method D3PM by Austin et al. (2021), which we explain in more detail in the context of our model in Section 3.
WyCryst A similar work to ours is WyCryst (Zhu et al., 2024), which also generates only a Wyckoff- descrip- tion (and assign exact coordinates in a later step). However, this is based on a different representation than ours, and their study focuses on generation of strictly ternrry materi- als while we put no such restrictions on the materials.
3. Wyckoff Diffusion
3.1. Representing a Protostructure
Given a space group s E G= (1..., 230}. we denote the set of all possible Wyckoff positions as L(s)². To represent a protostructure, we partition the set of Wyckoff positions into the positions without degrees of freedom (i.e., an atom occupying the position limited to fixed point in space) and the positions with degrees of freedom (i.e., atom oc- cupying the position can be positioned anywhere on line, in a plane, or in a volume). We call these constrained and unconstrained positions, and use the notation Lo C L(s) and L (s) C L(s) for the respective sets. Although uncon- strained Wyckoff positions can virtually be occupied by any number of atoms, in our modeling, a maximum of P atoms
2.3. Related Work
CDVAE The Crystal Diffusion Variational Autoencoder (CDVAE) (Xie et al., 2022) is a generative model for crystal structures that combines a variational autoencoder (VAE) with diffusson model. Generation from CDVAE starts with sampling from the VAž: a vector (o, 1) is sampled from which the lattice vectors L, the number of atoms M, and the initial composition are decoded. The positions of the M atoms are randomly initialized, and the elements are
Note that the canonicalization of the protostructure compared the prototype is different, due protostructures heing canoni- calized based on alphabetical element order.
'lll possible Wyckoff positions can be found in IUCr (2002).
3

---

vector in Rd, and we then use a Wyckof[Diff - A Generative Diffusion Model for Crystal Symmetry
of each type can occupy an unconstrained Wyckoff position (which means the unit cell has P times the multiplicity of that Wyckoff position of such atoms). We denote Na as the largest atomic number under consideration. Both Na and P can be determined from training data. Conditionally on the space group s, the unconstrained positions can then
variable x² at sampling time 4, the Markov forward process (ef. the general description in Section 2.2) can be written as
q(x++, (xx) = Categorical (1 |p Q(+t+1), (5)
with Q++1 being a transition matrix, and g(xx+1|xx) = Ie= ((x(+1 |xe). The matrices Q++1 are chosen so that the stationary distribution (g(xf) ) for large T) is a simple distri- bution (we discuss this choice in Section 3.5). The variables x are assumed conditionally independent given X(+1 in the backward process, i.c., Pe (x(xxxx++)) = II Pe(x xx+1), and as the backward distribution q(x; X x5) can be computed exactly, the backward process (x xx41) is
be represented by ~ E Max = {0, I..... ( i... each element (E.j). {0, ,.....,),, is the number of atoms of type j occupying the unconstrained Wyckoff po- sition i. A constrained position, however, can only be oc- cupied by 0 or 1 atoms (as the positions are restricted to a fixed point in space). Therefore, we represent the ele- ments of the atoms occupying each of these positions as z E M, = (O....,
. No. [La(,)), where the value 0 corre- sponds to no atom occupying the position. To summarize, protostructure can be described as the tuple*
parametrized as a marginalization over all possible xi:
po (o(x)(xxta) = Σ
(6)
z (,)) M x Mo.
(3)
In other words, to use this framework, it is necessary to de- termine a suitable noise process (i.e., choosing the matrices 1x+1), and construct and train model which can predict the "clean" variable xñ. given a noisy sample X++1 (i.e., the model po (x8 x++1)).
3.2. Model Overview
Given our representation of protostructure in Equation (3), we now aim to sample from the (unknown) distribution (s z², ) Since the space group determines the num- ber of Wyckoff positions, we propose to first sample a space group s, and then sampling the remaining variables condi- tioned on s. Using the representation (s, z0, 200) ensures that we sample a valid material where constrained positions occupied by at most one atom. As an estimation of the distribution of we can use the empirical training data dis- tribution (s), and write our model of Psata(s, z", ))
3.4. WyckofIGNN - Neural Network Backbone
For the parametrization of (x(x)x++4) we design a novel neural architecture, WyckoffGNN, that takes a "noisy" data point X as input, and outputs 1) different probabil- ity vectors, where D is the number of variables. This means that for the Wyckoff representation in Equation (3), the neural network needs to predict the probabilities for D = 1 (s)] x N. [s)) different categorical distri- butions. To do this, we view each Wyckoff position in L(s) as node in a fully connected graph. As different space groups have different number of Wyckoff' positions, using the graph representation and processing this with a graph neural network (GNN) gives us the ficxibility to utilize a single model for all space groups. The GNN used to encode each position neural network to decode the vectors into the corresponding probability distributions. An illustration of this can be found in Figure 1.
as
po po 2,, 200 = Pddatas))pp(( 2z (s),
(4)
where pe(z", (=)) is a diffusion model. We will in the next sections describe how we design po(2", 200 (s), and when doing so, we will for simplicity use the notation as the concatenation (z", )) as well as keeping the conditioning on 8 implicit. Algorithm 1 outlines the full generation of a material using WYCKOFFDIFF.
3.3. Discrete Diffusion
Encoding Wyckoff' Positions
The encoding of Wyckoff MMx)]
positions starts with initial set of vectors (h]).4 -_)]) one for each Wyckoff position. These encode the atoms occu- pying the respective positions, i.e., d-dimensional vector embeddings of the atom types on the positions in Lo, and the number of each element on the positions Lox (see more details in Appendix A). Additionally, we have set of static vectors {h, pos [s(ss] which encode information about the po- sition like the Wyckoff letter and the number of degrees of freedom, but also the space group s and the sampling time t (again, the form of high-dimensional embedding vectors,
As both z and x0 are discrete variables, we will use the Discrete Denoising Diffusson Model (D3PM) (Austin et al., 2021) as our underlying diffusion model. In this framework, a datapoint is denoted as x = (x' (x'......:)) P)) where each variable x² is discrete variable, and "noise" is added inde- discrete Markov chain. By denoting x as one-hot encoding of the kith
pendently to each variable according
*For ease of notation, have omitted the dependence of M.. and M, on 8.
4

---

WyckoffDiff- - A Generative Diffusion Model for Crystal Symmetry
Algorithm 1 WYCKOFFDIFF
over which atom type (if any) that is occupying the position. For the unconstrained Wyckoff positions Loo. it instead corresponds to, for each atom type, the probabilities over the number of atoms of the corresponding atom type that occupies this position. As the output differs between these two types of positions, we use two different MLPs for the decoding. For the constrained positions, an MLP takes as input the representation hς" and outputs a single vector of probabilities over atomic numbers, where we use 0 "no atom" and only consider the atomic numbers I to N, = 100, as there are no training data points involving higher atomic numbers. For the unconstrained positions, an MLP instead outputs Na different probability vectors over number of atoms, one for each atom type. Again, we use a truncated range of 0 to P === 54 based on training data. An algorithm outlining the full forward-pass of the neural network can be found in Algorithm 2 in the appendix, together with more details in Appendix A.
We use the notation = =2], 2,,,2,²) ), In the for- loop over k; if k is an unconstrained position, xij consists of Naa variables and MLP(h)) outputs Na different probability vectors (sampled independently)
Note:
Sample 8 ~ Pdata (s)
Sample Xy ~ po (o(x (s) {Prirr distribution, e.g., assign
all variables to }eros}
for t in T - ......... do
Encode material as (h+ [[( k= = GN,(s, xx+1)
for k in 1: [[{) do
po (xx) ;)+1; 8) Cat p= Mhe((
Compute pa [x s) according to Equation (6)
Sample x ~ po (xx) xx)xx++1.8)
end for
end for
return 8, Xo
see Appendix A). We then design the 1:th update of the vec- tors as first concatenating h! with its corresponding h and then one layer of a pesssage-passing neural network (Gilmer al., 2017) where first, for each Wyckoff position, a message m is computed as m = Σ)= Mi(W,, wy). where w; and w, are the aforementioned concatenation of vectors. The message m is hence an aggregation of messages sent between pairs of Wyckoff positions, and the purpose is to propagate information about the full material. As we do not have an inherent graph but rather assume a complete graph, we construct a message function -r(---) inspired by Bronstein et al. (2021, chapter 5.4) where we use two multilayer perceptrons (MLPs, or fully connected neu- ral networks). One MLP takes in the neighboring vector w and outputs new vector w/ MLPP,, (W;). while the other takes input concatenation of w; and w, and outputs scalar aig = MLPP(caa((e) (w;, w,)), which is multiplied with w' i.e.,
Training To train our neural network, we start by sam- pling a time t from the discrete uniform distribution Uniform([,...., TT). Then, to sample x; ~ q(x, Xx),, we sample x ~ q(x/1xx)) = Categorical(p = x Q,) independently for each k E {,, ...., D). where Q. Q 2I... Qt. and the choice of Q, is described in Sec- tion 3.5. The neural network takes as imput this noisy sample X,, and as in DiGress (Vignac et al., 2022), we optimize the cross-entropy between the true sample xo and the predicted distribution px (x). J. We also tried the variational objec- tive by Austin et al. (2021), but the large state spaces made it unfeasible to fit into GPU-memory.
3.5. The Choice of Qt
Austin et al. (2021) proposes a few different choices of Qu- In our work, we use a matrix of the form
= (1- + 3,1m"'
(8)
M((w,,,) = 0s.j (W;, Wj) (w)))(W)).
(7)
where 8y is given by some user-defined schedule, is a vector of ones, and m is vector of probabilities. With this transition matrix, variable stays in its current state with probability 1 3r, and with probability 3r it transitions to new state sampled from a Categorical(p
The message m is hence a linear combination of transfor- mations of the neighbor vectors zy. This message is then added to the current vector, so that the updated vector rep- resentation becomes hy = h 11 + m Performing such updates N times (i.e., a neural network with N layers), we obtain our encoded positions as the vector representations [[())]
m) distribu- tion. This is general form for which the choice m = L/D gives rise to D3PM-uniform by Austin et al. (2021). In this general form, for large T, the limiting distribution qf (x)) becomes Categorical(p m). and sampling from D3PM hence starts by sampling each variable x from this dis- tribution. Although using the uniform distribution could work, in case the data is very "sparse", for example in our case where most of the elements in the matrix representation in Section 2.1 are O, using the uniform distribution as the limiting distribution could require many generation steps just to find the correct level of "sparseness". Vignae et al.
(ha) J==1 Algorithms describing the GNN layer and the message function together with more details on hyperparam- eter choices can be found in Appendix A.
Decoding the Probabilities When we have obtained the encodings (h ) of the Wyckoff positions, we need to decode these into vectors of probabilities. For constrained Wyckoff positions, Lop this corresponds to probabilities
5

---

WyckoffDifT - A Generative Diffusion Model for Crystal Symmetry
Table 1. Results on the material generation task. All metrics are computed for 10000 samples, and we present averages and standard deviations for three models trained with different seeds. To compute FWD, the training set was subsampled to contain an equal number of samples. the case of novel materials, 10000 novel materials have been generated. The different options for WYCKOFFDIFR indicates the different prior (limiting) distributions. *Models trained only for 100 instead of I 000 epochs. **SymmCD is somewhat unstable and produces materials with NaN values (~~ 4% of the materials), while WYCKoFF-DFF-unifor produces a few materials with 0 atoms (S 0.05%), and therefore discard these, meaning the numbers for these models are slightly biased (however, the numbers are still computed on 10k samples).
NOVEL
Nov. 1 (%)
UNIQ. (%)
UNIQ t (%)
MODEL
FWD 1
FWD 1
NOV./MIN.
CDVAE
41.9 + 2.69
99.4 + 0.06
99.9 = 0.00
41.8 + 2.60
99.9 0.00
71
DIFFCS+++
0.83 + 0.14
48.4 + 0.56
98.4 1 0.12
5.15
98.7 + 0.06
46
SYMMCD** *
1.47 + 0.29
52.3 + 1.21
98.1 # 0.12
4.53 + 0.65
98.9 + 0.06
115
UYCKOFFDIFF-UNIFORM
2.29 0.15
40.2
0
13.71
98.0 0.12
159
OURS
1.65 0.07
55.7 +
98.6 0.21
+
98.9 + 0.06
WYCKOFFDITF-MARRGIMA
0.55 0.05
31.4 1.46
98.0 + 0.12
1.57+
97.6 + 0.23
125
WYCKORPDIT--ZEROS
1.03 + 0.24
54.9 + 2.54
98.8 0.17
5.39
99.3 + 0.15
WYCKORFDUF--ZEROS
0.48 + 0.02
30.2
0.97
98.1 0.23
4.36
98.1 0.15
119
(2022) propose to use the empirical marginal distribution instead of the uniform distribution as m. As we show in the experiments section, we find that using a marginal distri-
4. Numerical Evaluations
4.1. FWD, Novelty, and Uniqueness
bution, or Dirac distribution at zero for all variables (i.e., starting from a material without any atoms at all), greatly improves the performance compared with using the uniform distribution.
The quantitative evaluation of our models uses the WBM dataset (Wang et al., 2021) created by substitution of chem- ical elements in the crystal structures available from the Materials Project (MP) (Jain et al., 2013) to generate a to- tal of 257k materials. We set aside 10k+10k materials as validation and test sets. We start by comparing WYCKOFFD- IFF with CDVAE (Xie al., 2022), DifICSP++ (Jiao et al., 2024), and SymmCD (Levy et al., 2025) they constitute examples of models that to different degrees model crystal symmetry. Implementation details of these baseline meth- ods can be found in Appendix B. It should be noted that we encountered some numerical issues during generation with SymmCD, resulting in NaN values, and we chose to discard these failed materials (~ 4% of samples, see more details in Appendix B). We also found that using WYCK- OFFDIFF with uniform initialization can produce small amount (S 0.05%) of "void" materials with 0 atoms, which we also discarded.
3.6. Evaluation Metric Fréchet Wrenformer Distance
To evaluate generative model, strive to find way of projecting materials into some lower-dimensional space, and draw conclusions about the difference between gener- ated materials and real materials in this space. To do this, we take inspiration from the Fréchet Inception distance used for image generation (Heusel et al., 2017), and propose the metric Fréchet Wrenformer distance (FWD). This metric computes the Wasserstein distance between Gaussian dis- tributions fit with embeddings of the generated materials and training set, respectively, extracted from the pretrained Wrenformer (Riebesell et al., 2024), which adapts the GNN- based model by Goodall et al. (2022) to a Transformer architecture (Vaswani et al., 2017) and is distributed with the aviary software The FWD metric aims to capture the similarities of the generated materials with the train- ing materials, while being invariant to exact geometry the Wrenformer only takes into account the protostructure of the material. Similar developments have been done for chemical (Fréchet ChemNet distance, FCD (Preuer et al., 2018)) and biological (Fréchet Biological distance, FBD (Stark et al., 2024)) applications.
As the focus of our work is on the generation of protostruc- tures and the compared methods all generate full geometries, we convert these materials AFLOW protostructures (Mehl et al., 2017) using aviary², with default tolerance param- eters. For all methods, we generate 10 000 protostructures and compute the FWD, novelty (Nov., the fraction of gen- erated protostructures not present in the training set), and uniqueness (Uniq., fraction of unique protostructures among the generated). We discuss validity in Appendix D. The re-
'https: / // thub. com///ooppRyy///vatarr//tetee
'We provide an experiment on Carbon24 (Pickard, 2020) in Appendix E
main
6

---

WyckoffDifT- A Generative Diffusion Model for Crystal Symmetry
Table The number of unique and novel prototypes among 10000 novel protostructures.
WBM-train
Q10=-2.326
0.8
Q50=-0.552
Q90=-0.033
# UNIQUE & NOVEL PROTOTYPES
Gen-novel
MODEL
0.6
010=-2.308
Densigy
Q50=-0.495
Q90=0.043
CDVAE
2083 + 61
DIFFCS+++
527 + 39
0.4
Gen-unfiltered
Q10=-2.272
SYMMCD
780 + 49
-50m80
WYCKOFFDITF-UNIE
1214 + 32
Q90=-0.036
0.2
WYCKOFFDITF-MARG
733 + 11
WYCKOFFDIFF-ZERSS 1175 + 80
0.0 5
-3
-2 -1
Formation energy [eV/atom]
sults are presented in Table 1. It should be noted that, in this discrete setting, we do not expect the novelty to be even for a "perfect model"". However, in a practical materi- als discovery setting we are mainly interested in the novel materials and, since FWD is a metric that benefits from sampling materials from the training set, we also compute FWD and uniqueness among only novel materials. To do this, we generate enough materials so that we have obtained 10 10000 novel protostructures from all methods. To simulate the computational cost if applying the postprocessing step of filtering out novel materials, we provide the number of novel materials per minute (nov./min).
Figure 2. Distribution of formation energies predicted by Wren for WYCKOFFDIFF->enss generated (unfiltered) protostructures and novel protostructures, relative the training Q10, a50,and Q90 are the 10th, 50th, and 90th percentiles respectively.
this "novelty filer" as part of the generative procedure. Ad- ditionally, when computing FWD on only novel materials, WYCKOFFDuFF outperforms all baselines, indicating that even if the protostructures are novel, they are to larger extent faithful to the training distribution.
From Table 1, we first conclude that CDVAE, which does not incorporate any knowledge about symmetry, generates materials that are very dissimilar to the training distribution, as indicated by the very high FWD. As FWD measures similarity based on protostructures, the high value is likely to be due to the inability to capture the symmetry properties of materials. By examining the distribution of space groups, find that 36% of the materials generated by CDVAE are from space group 1, and >90% are in space gropus 1-15, while the corresponding numbers for WBM are 0.3% (SG 1) and 13% (SG 1-15). Similar results were found by (Levy et al., 2025).
4.2. Prototype Uniqueness
In Section 4.1, materials were classified as different if their protostructures were different. Now, we consider only the prototypes to evaluate the models' abilities to generate struc- tural novelty. Among the 10000 novel protostructures, we count the number of unique and novel prototypes and present this Table We see that our model indeed gener- ates new prototypes, which highlights that it not merely learning "substitution-algoriin already know structural template (i.e., the prototype) and just replace the elements. We also see that only CDVAF performs better in this regard, but as CDVAE has no restric- tions in its generation, this is expected. However, when comparing to DiffCSP++ and SymmCD which do take sym- metry into account, WYCKOFFDIFE produces significantly higher number of unique and novel prototypes, showing its promise as a general generative model for crystal structures.
where it learns to
We also notice that the choice of initial distribution in DYCKOFFDIFF makes a big difference, and using the uni- form distribution severely underperforms compared to ini- tializing from the marginal distribution, with completely empty materials. This highlights that even if the model is supposed to "denoise"" starting from something that is closer to the actual data plays a big role. Compared to the baselines, we notice that the novelty for WYCKOFF DIFF is somewhat lower, which seems be connected with training time: numbers for models trained with only 10 % of the number of steps shows higher novelty, indicating that the model is "memorizing" the training distribution. However, looking at sampling speed, WYCKOFFDIFF is much faster as it does not generate full geometries, and hence, even if the novelty is lower, we produce more novel materials with the same amount of computation time, and we could view
4.3. Wren Energies
To further investigate the protostructures generated by WYckoFFDIFF and get sense of their usefulness, compare the formation energies (i.e., the energy required to form material from the pure elements, see Appendix F for more details) of the generated protostructures with those of the training set. To compute the formation energies, we rely on the same pretrained Wrenformer model as used for FWD

---

WyckoffDiff A Generative Diffusion Model for Crystal Symmetry
(a) 11224_16_a___ a_h b:Cs-F-Sn
(b) ABC,_hP8 1e__194_______ a__---Na-
(c) A,BC_hR12 166 1_6_______________ b:Ca-IP
22
CsSnF; g
NaNbO;
Ca Enuil 0.0
Enul -0.032
Euul -0.001
Cs
Sn Na
Nb Ca
Figure 3. Selection of three examples of WYCKOFFDIFF generated crystal structures close or below the convex hull of WBM and Materials Project (MP). Displaying the energy above hull Exau [eV] relative to the convex hull of WBM and MP combined. (a) has a formation energy of Eform = -2.610 the resulting in ExUit being negative distinctly below hull. In comparison with the convex hull structure (a) is indeed below the hull, highlighted with the green star in the phase diagram. (b) has formation energy of Eform = 2.537, 1.422 which makes the Exaut approximately zero. Comparing (b) and (c) with the convex hull shows that the structures are on the hull, indicated by the smaller stars.
resulting in a negative Enall but insignificantly far from the hull. (c) has a formation energy of Eform
(see Section 3.6), which can predict the formation energy only given a protostructure. Figure 2 shows histograms of formation energies of protostructures generated by the zeros- initialization model. We see that the materials in general follow the same distribution as the training set, where the novel materials have a slight shift towards higher energies. A possible explanation is that the training data, ultimately derived from structures seen in experiments, samples the lowest energy structures thoroughly enough that the filter- ing on novel materials rejects more lower energy structures than higher energy ones. This further suggests the ability of DIFKOFFDIFF to generate protostructures that are also physically plausible. We see overall the same results for the distributions for the other versions of WYCKOFFDIFF, and present those in Appendix C.
use the interatomic potential MACE² (Batatia al., 2023) to perform constrained relaxation where the energy is min- imized while the symmetries set by the protostructure are retained. We repeat this process of realizing and relaxing crystal structures until the two lowest energies seen lies within small cutoff of 0.01 eV eV/atom. The lowest energy found taken as our computationally predicted energy of the material generated by WYCKOFFDIFF. As is common in materials science, this energy is converted into forma- tion energy by for each atom subtracting the corresponding energy per atom from a representative elemental solid.
Low formation energies are only indirectly related to stabil- ity; the thermodynamically stable material at a composition is the one with the lowest formation energy compared to all alternative competing phases and linear combinations of phases, which spans the so called convex hull of ther- modynamical stability (see, e.g., Bartel et al. (2020) and Appendix F for more details). However, given the indi- rect relationship, we selected 200 structures with the lowest formation energies to investigate further. We used the high- throughput toolkit (httk) (Armiento, 2020) recalculate them with density functional theory (DFT) using the VASP clectronic-structue software (Kresse & Hafner, 1994) and evaluated their stability relative to the known convex hull from all materials in the MP (Jain et al., 2013) and WBM (Wang et al., 2021) databases (further details in G.2).
5. Materials Discovery Using DYCK
We now demonstrate how DYC fits into a materi- als discovery pipeline. Starting with a generation of 20 000 novel crystal structures, 10000 from each of two WYCK- OFFD models (WYCKoFDDFF-Zeros and previous iter- ation of W WYYCKOFFDIFF--ARE marginal;; see Appendix G.3), we extract structures with chemical elements that are not noble gasses and where the underlying computational methods used for the training data are known to be more reliable, i.e., elements from the s-, p-, and d-blocks of the periodic table of elements.
Out of the 200 selected materials, we highlight three hand- picked examples with interesting chemistries (CsSnFg. NaNbO2, and CaşPI), shown in Figure 3 in their respective composition phase diagrams generated using pymat gen (Ong et al., 2013). The DFT results for these generated ma- terials confirm them to be stable; one is distinctly below, and
We then realize the resulting 12 12650 protostructures into crystal structures by a process where we first semi-randomly assign values to the degrees of freedom of the Wyckoff po- sitions using the Pyxtal library (Fredericks et al., 2021) using the implementation in aviary?. Subsequently, we
//tt:////gttttb/t www Coes/ACEsuit.... t/acee-eeeeeeaeeeeeeeeeee mpp re releases/cag http///aceeeaaeaeeeeeeeeeeeeaaee
8

---

WyckoffDiff- - A Generative Diffusion Model for Crystal Symmetry
the other two are on, the convex hull. Hence, the generated structure for CsSnF& is clearly a new predicted material not present in MP or WBM. The other two materials, 2a and CazPI, already exist in MP (i.e., they are part of the known convex hull and therefore on it), and can be traced to experimental works (Roth et al., 1993; Hadenfeldt & Herdejürgen, 1988). These are thus explicit examples of WYCKOFFDIFF recreating materials outside of its training set (WBM), which are experimentally confirmed to exist. These results substantiate the ability of the model to gener- ate materials that are physically reasonable. Furthermore, our investigation of the 200 selected materials finds seven other fluorides confirmed by DFT to be distinctly below the known convex hull from WBM and MP (details presented in Appendix G.I. Table 6). The over-representation of new stable fluorides in this set of 200 materials is likely due to that proof-of-conce smallest, i.e., most negative, formation energies may bias towards this chemistry, rather than being feature of the model.
Center Linköping-Lund in Information Technology (EL- LITT). R.A, O.B.A, and A.S.P acknowledge support from the Swedish Research Council (VR) grant no. 2020-05402 and the Swedish e-Science Centre (SeRC).
Parts of the computations were enabled by the Berzelius resource provided by the Knut and Alice Wallenberg Foun- dation at the National Supercomputer Centre (NSC). Other computations performed at NSC and Chalmers Centre for Computational Science and Engineering (C3SE) were en- abled by resources provided by the National Academic In- frastructure for Supercomputing in Sweden (NAISS), par- tially funded by the Swedish Research Council through grant agreement no. 2022-06725.
CRediT Authorship Contribution Statement
methodology of extracting the
Data curation, Formal analysis, Investigation, Methodology, Project administration, Soft- ware, Validation, Visualization, Writing original draft, Writing - review & editing Oskar B. Andersson: Data cura- tion, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing- -aaainngo-rrgaaee original draft, Writing review & editing Abhijith S. Parackal: Data curation, Formal analysis, Investigation, Methodology, Software, Val- idation, Visualization, Writing original draft, Writing- review & editing Dong Qian: Formal analysis. Investi- gation, Methodology, Software, Validation, Visualization, Writing- - original draft, Writing - review & editing Rickard Armiento: Conceptualizaaion, Funding acquisition, Super- vision, Validation, Writing- - original draft, Writing - review & editing Fredrik Lindsten; Conceptualization, Funding acquisition, Supervision, Writing original draft, Writing- review & editing
Filip Ekström Kelvinius:
6. Discussion& & Conclusions
In this paper we propose WYCKOFFDIFF,, novel gener- ative model which leverages new representation of the symmetrical aspects of materials together with novel neu- ral network architecture and discrete diffusion to generate new protostructures. Although obtaining the full material requires extra steps, viewing the protostructure and the full geometry as separate processes opens up the possibility of using models tailored for each respective task, and use of computational effort where it is most needed. As we high- light with our proof-of-concee materials discovery pipeline in Section 5, the precise geometry can be uncovered via a pretrained generally applicable interatomic potential such as MACE, only for the most promising materials. WYCK- OFFDIFF shows competitive performance compared to the current state-of-the-ar both in terms of novel generated ma- terials/min, structural novelty, and agreement with the data distribution based on the newly proposed Fréchet Wren- former Distance.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There many potential societal consequences of our work, none which we feel must be specifically highlighted here.
References
Acknowledgments
Armiento, R. Database-driven high-throughput calculations and machine learning models for materials design. Ma- chine Learning Meets Quantum Physics, pp. 377-395, 2020.
This work was partially supported by the Knut and Alice Wallenberg Foundation (KAW) via the Wallenberg AI, Au- tonomous Systems and Software Program (WASP) and the Wallenberg Initiative Material Science for Sustainability (WISE) through the joint WASP-WISE project Generative Al models for property to structure materials prediction.
Austin, J., Johnson, D. D., Ho, J., Tarlow, D., and van den Berg, R. Structured Denoising Diffusson Models in Dis- crete State-Spaces. In Advances in Neural Information Processing Systems, November 2021.
F.E.K., D.Q., and F.L. further acknowledge support from the Swedish Research Council (VR) grant no. 2020-04122, 2024-05011, 1, KAW project 0033, and the Excellence
Bartel, C. J., Trewartha, A., Wang, Q., Dunn, A., Jain, A.,

---

WyckoffDiff- A Generative Diffusion Model for Crystal Symmetry
and Ceder, G. A critical examination of compound stabil- ity predictions from machine-leared formation energies. npj Computational Materials, 6(1):97, July 2020. ISSN 2057-3960. doi: 10. (0088//411544-02000030000
and symmetry analysis. Computer Physics Commu- nications, 261:107810, 2021. ISSN 0010-4655. doi:
Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O., and Dahl, G. E. Neural Message Passing for Quantum Chem- istry. In International Conference on Machine Learning, pp. 1263-12777 1033-122. PMLR, July 2017. 1.. ISSN: 2640-3498.
Batatia, I., Benner, P., Chiang, Y., Elena, A. M., Kovács, D. P., Riebesell, J., Advincula, X. R., Asta, M., Baldwin, W.J., Bernstein, N., Bhowmik, A., Blau, S. M., Carare, V., Darby, J. P.. De, S., Pia, F. D., Deringer, V. L., Elijošius, R., El-Machachi, Z., Fako, E., Ferrari, A. C., Genreith- Schriever, A., George, J., Goodall, R. E. A., Grey, C. P., Han, S., Handley W., Heenen, H. H., Hermansson, K., Holm, C., Jaafar, J., Hofmann, S., Jakob, K. S., Jung, H., Kapil, V., Kaplan, A. D., Karimitari, N., Kroupa, N., Kullgren, J., Kuner, M. C.,, Kuryla, D., Liepuoniute, G., Margraf, J. T., Magdău, I.-B., Michaelides, A., Moore, J. H.. Naik, A. A., Niblett, S. P., Norwood, S. W. O'Neill, N., Ortner, C., Persson, K. A., Reuter, K., Rosen, A. S., Schaaf, L. L., Schran, C., Sivonxay, E., Stenczel, T. K., Svahn, V., Sutton, C., van der Oord, C.,, Varga-Umbrich, E., Vegge, T., Vondrák, M., Wang. Y., Witt, W. C., Zills, F., and Csányi, G. A foundation model for atomistic materials chemistry. MrXiv:2401.000% 2023.
Goodall, R. E. A., Parackal, A. S., Faber, F. A., Armiento, R., and Lee, A. A. Rapid discovery of stable materials by coordinate coarse graining. Science Advances, (30) 17, 2022. doi: 10.1 1266/ciadv..abn4111
Hadenfeldt, C. and Herdejürgen, H. Darstellung und kristall- struktur der calciumpnictidiodide ca2ni, ca2pl und ca2asl. Zeitschrift für anorganische und allgemeine Chemie, 558(1):35 40, March 1988. ISSN 1521-3749. doi: 10.1002/2aacc11988555811411
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vish- wanathan, S., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 30. Curran As- sociates, Inc., 2017.
Belsky, A., Hellenbrandt, M., Karen, V. L., and Luksch, P. New developments in the inorganic crystal structure database (icsd): accessibility in support of materials re- search and design. Acta Crystallographica Section B Structural Science, 58(3):364-36, May 2002. ISSN 0108-7681, doi: 10.1 102107//00077681200006666
Ho, J., Jain, A., and Abbeel, P. Denoising Diffusion Prob- abilistic Models. In Advances in Neural Information Processing Systems, volume 33, pp. 6840-6851. Curran Associates, Inc., 2020.
Bronstein, M. M., Bruna, J., Cohen, T., and Veličković, P. Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges. arXiv:2104.13478 [cs, stat), May 2021. arXiv: 2104.13478.
Hoogeboom, E., Nielsen, D., Jaini, P., Forré, P., and Welling, M. Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions. In Advances in Neural Infor- mation Processing Systems, volume 34, pp. 12454 12465. Curran Associates, Inc., 2021.
Campbell, A., Benton, J., Bortoli, V. D., Rainforth, T., Deli- giannidis, G., and Doucet, A. A Continuous Time Frame- work for Discrete Denoising Models. In Advances in Neural Information Processing Systems, May 2022.
TUCr. International Tables for Crystallography, Volume A: Space Group Symmetry. International Tables for Crys- tallography. Kluwer Academic Publishers, Dordrecht, Boston, London, 5. revised edition edition, 2002.
Cheetham, A. and Seshadri, Artificial Intelligence Driving Materials Discovery? Perspective on the Article: Scaling Deep Learning for Materials Discovery. Chem- istry of Materials, 0897-4756. doi: 10. 1/ac022lacc.cc wwwmmmaer...000644..... Pub- lisher: American Chemical Society.
April 2024. ISSN
Jain, A., Hautier, G., Ong, S. P., Moore, C. J., Fischer, C. C., Persson, K. A., and Ceder, G. Formation enthalpies by mixing GGA and GGA S+$ SUS calculations. Phys. Rev. B, 84(4):045115, July 2011. doi: 10.1 10.1103/PysRevvB..s 045115.
Davies, D. W., Butler, K. T., Jackson, A. J., Skelton, J. M., Morita, K., and Walsh, A. Smact: Semiconduct- ing materials by analogy and chemical theory. Jour- nal of Open Source Software, 4(38):1361, 2019, doi: 10.21 10210000osss.113111 URL https: doi org/10. 21105/joss. 01361.
Jain, A., Ong, S. P., Hautier, G., Chen, W., Richards, W. D., Dacek, S., Cholia, S., Gunter, D., Skinner, D., Ceder, G., and Persson, K. A. Commentary: The Materials Project: A materials genome approach to accelerating materials innovation. APL Materials, 1(1):011002, July 2013. ISSN 2166-532X. doi: 10. 01066/1.48122999
Fredericks, S.. Parrish, K., Sayre, D., and Zhu, Q. Pyx- tal: A python library for crystal structure generation
10

---

WyckoffDiff - A Generative Diffusion Model for Crystal Symmetry
Jiao, R., Huang, W., Lin, P., Han, J., Chen, P., Lu, Y., and Liu, Y. Crystal Structure Prediction by Joint Equivari- ant Diffusion. In Thirty-Seventh Conference Neural Information Processing Systems, November 2023.
Parackal, A. S., Goodall, R. E. A., Faber, F. A., and Armiento, R. Identifying crystal structures beyond known prototypes from X-ray powder diffraction spectra. Physi- cal Review Materials, 8 8(10): 103001 October 2024. ISSN 2475-9953. doi: 10. 103/PhysRee 10.300
Jiao, R., Huang, W., Liu, Y., Zhao, D., and Liu, Y. Space Group Constrained Crystal Generation. In The Twelfth International Conference on Learning Representations, 2024.
Park, H., Li, Z., and Walsh, A. Has generative artificial intelligence solved inverse materials design? Matter, 7 (7):2355 July 2024. ISSN 2590-2385. doi: 10.
1016//.mat.2024..5..11
Kresse, G. and Hafner, J. Ab initio molccular-dynamice simulation of the transition in germanium. Physical Review B, 49(20); 14251, 1994.
Pickard, C. J. Airss data for carbon at 10gpa and the c+nth+o system at Igpa, 2020. URL https://archive, htteo::/aacciiennttttiaaaa..oo www.emaaaaaiiio..oooommm record/2020 002.6/200002/.. 0026/vl.
Levy, D., Panigrahi, S. S., Kaba, S.-O., Zhu, Q. Lee, K. K., Galkin, M., Miret, S., and Ravanbakhsh, S. crystal generation with diffusion models. In The Thirteenth International Con- ference Learning Representations (ICLR), 2025.
Preuer, K., Renz, P., Unterthiner, T., Hochreiter, S., and Klambauer, G. Fréchet chemnet distance: metric for generative models for molecules drug discovery. Jour- nal of Chemical Information and Modeling, 58(9):1736- 1741, 2018. doi: (a.1021/acs..cimm.8000 301 18593.
SymmCD:
PMID:
Loshchilov, I. and Hutter, F. Decoupled Weight Decay Regularization. In International Conference on Learning Representations, 2019.
Ramachandran, P., Zoph, B., and Le, Q. V. Searching for Activation Functions, October 2017. arXiv:1710.059. [cs].
Lou, A., Meng. C., and Ermon, S. Discrete Diffusion Mod- eling by Estimating the Ratios of the Data Distribution. In Proceedings of the 41st International Conference on Machine Learning, pp. 32819-32844 PMLR, July 2024. ISSN: 2640-3498.
Riebesell, J., Goodall, R. E. A., Benner, P., Chiang, Y., Deng, B., Ceder, G., Asta, M., Lee, A. A., Jain, A., and Persson, K. A. Matbench discovery a framework to evaluate machine learning crystal stability predictions, 2024.
Mehl, M. J., Hicks, D., Toher, C., Levy, O.,, Hanson, R. M., Hart, G., and Curtarolo, S. The AFLOW Library Crys- tallographic Prototypes: Part 1. Computational Materials Science, 136:S1-S828, August 2017. ISSN 09270256. doi: 10. 10.10166/.commmitti 100.6f..oommatii22111..00..0
Roth, H., Meyer, G., Hu, Z., and Kaindl, G. Cheminform abstract: Synthesis, structure, and x-ray absorption spec- tra of lixnbo2 and naxnbo2 (x Σ 1). Chemlnform, 24 (42):1369 1373, October 1993. ISSN 1522-2667. doi: 10. 10.0/Occhin 199342004.
Merchant, A., Batzner, S., Schoenholz, S. S., Aykol, M., Cheon, G., and Cubuk, E. D. Scaling deep learn- ing for materials discovery. Nature, 624(7990):80-85, December 2023. ISSN 1476-4687. doi: 10.1038/ 9411586-033-06335-9 Publisher: Nature Publishing Group.
Sohl-DDckstein, J., Weiss, E., Maheswaranathan, N., and Ganguli, S. Deep Unsupervised Learning using Nonequi- librium Thermodynamies. In Proceedings of the 32nd International Conference on Machine Learning, pp. 2256 2265. PMLR, June 2015.
Müller, U., Wondratschek,, H., and Bärnighausen, H. Sym- metry Relationships between Crystal Structures: Applica- tions of Crystallographic Group Theory in Crystal Chem- istry. Number 18 in IUCr Texts on Crystallography. Ox- ford university press, Oxford, 2013. ISBN 978-0-19- 966995-0.
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Er- mon, S., and Poole, B. Score-Basee Generative Modeling through Stochastic Differential Equations. In Interna- tional Conference on Learning Representations, 2021.
Stark, H., Jing, B., Wang, C., Corso, G., Berger, B., Barzi- lay, R., and Jaakkola, T. Dirichlet flow matching with applications to DNA sequence design. In Forty-first In- ternational Conference on Machine Learning, 2024.
Ong, S. P., Richards, W. D., Jain, A., Hautier, G., Kocher, M., Cholia, S., Gunter, D., Chevrier, V. L., Persson, K. A., and Ceder, G. Python materials genomics (pymatgen): A robust, open-source python library for materials analysis. Computational Materials Science, 68:314-319 February 2013. ISSN 0927-0256. doi: 10.1016fj.oommtsci..00 commatsci.2012. 10.028.
Sun, H., Yu, L., Dai, B., Schuurmans, D., and Dai, H. Score- based Continuous-tinn time Discrete Diffusion Models. In International Conference on Learning Representations, February 2023.

---

WyckoffDiff A Generative Diffusion Model for Crystal Symmetry
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, I.., and Polosukhin, I. Atten- tion is All you Need. Advances in Neural Information Processing Systems, 30, 2017.
Vignac, C., Krawczuk, I., Siraudin, A., Wang, B., Cevher, V., and Frossard, P. DiGress: Discrete Denoising diffusion for graph generation. In The Eleventh International Con- ference on Learning Representations, September 2022.
Wang, H.-C., Botti, S., and Marques, M. A. L. Predicting stable crystalline compounds using chemical similarity. npj Computational Materials, 7(1):1-9, January 2021. ISSN 2057-3960. doi: 10. 1038//41552 000-552---00000 020-00481-6. Publisher: Nature Publishing Group.
Xie, T., Fu, X., Ganea, O.-E., Barzilay, R., and Jaakkola, S. Crystal diffusion variational autoencoder for peri- odic material generation. In International Conference Learning Representations, 2022.
Zeni, C., Pinsler, R., Zügner, D., Fowler, A., Horton, M., Fu, X., Wang, Z., Shysheya, A., Crabbé, J., Ueda, S., Sordillo, R., Sun, I.., Smith, J., Nguyen, B., Schulz, H., Lewis, S., Huang, C.-W., Lu, Z., Zhou, Y., Yang, H., Hao, H., Li, J., Yang. C.,, Li, W., Tomioka, R., and Xie, T. generative model for inorganic materials design. Nature, pp. 1-3, January 2025. ISSN 1476-4687. doi: 10. 1031:41336-025-08828-5 Publisher: Nature Publishing Group.
Zhu, R., Nong, W., Yamazaki, S., and Hippalgaonkar, K. WyCryst: Wyckoff inorganic crystal generator framework. Matter, 7100:3469-3488 October 2024. ISSN 2590- 2385. doi: mat0116..aat.22244.000
12

---

WyckoffDiff A Generative Diffusion Model for Crystal Symmetry
A. WyckoffGNN Details
A.I. Architecture
Here we give some more details on our neural network backbone, WyckofIGNN, As mentioned the main text, is based on the peasaag--asssing neural network framework (Gilmer et al., 2017), where each node in graph is represented by vector h/. and each layer corresponds to update of this representation according to
m,
1+1
Σ
M((h/,h))
(9a)
(EN())
h +1 = U, )
(9b)
Algorithm 2 describes the full pass through the network. It makes use Embedding() layers which maps discrete features, like the atom types number of atoms of certain atom type, vectors in some vector space Rdd and Linear() which are Wx + b. The embedding of the number of atoms embeds the
affine maps of vectors in iR to Rde i.e. Linear(x) number of atoms of each atom type in nd into scalar which are concatenated and then processed by linear layer such that all initial representations h of all Wyckoff positions are of the same dimension.
Algorithm describes the update of the hidden representations as Equation (9). As we are working on a fully connected graph, the sum over the neighbors over positions. our case, the input to Mr is not the hidden representations h and h/. but concatenations of the hidden representations and its corresponding position vector h which contains some general information of the Wyckoff position like the number of degrees of freedom, the letter, but also the space group and sampling timestep I. Algorithm 4 outlines how M2 is computed.
A.2. Choice of 3g
As scheduler for 3e, we used the cosine scheduler by Hoogeboom et al. (2021). By defining a, 1- and II Us. we choose Br such that
11/T +8
0 COS
(10)
1 2
with 0.008.
Algorithm 2 Full GNN forward pass
Input: Spacegroup s, positions with no constraints 20 E {0, 1...... (s)x N., positions with no degrees of freedom zd (O...., N₂ Lo(s), number of DOFs Xdof E ,.....,3)
Wyckoff letters Xiener E, timestep t where A, is the n-simplex
Output: Probability vectors 2 E A Los N. and pd E A Lo No
< stack (Embeddingnng("" Linear (Bmbedddin))
hpss f Embedding(Xaur) + Embedding(Xiuue) + Embedding(s) + Embedding(t)
for ayer in GNN ayers do
h layer(h, hpos) (Algorithm 3}
h + activation(h)
end for
xd < MLPo (Xao # 0) pd + (MLPs(h[x.. = 0))
return p'.pa
13

---

WyckoffDiff A Generative Diffusion Model for Crystal Symmetry
Algorithm 3 GNN layer forward pass. All operations are for 1.... L()) where [(s]
the number of Wyckoff
positions for the spacegroup
Input: Node features h' (h), h (o0) position specific embeddings hpss
Output: Updated features h
f- cat(h, hoss)
m 1+1 f Σ
2==1 [[()]] Mi(w;, w,) {M Algoritum 4. Complete graph, hence sum over all other positions.)
h f h + m
1+1 {Us, a simple skip connection)
return h
Algorithm& 4 GNN message, MiWw, w;) in Equation (9)
Input: Node features Wi, W
Output: Message mij = M (w;, w,)
ViJ - cat(w,, Wj)
as MLP5(vi.) {Scalar)
as < softmax, (as.g) (Will depend on other features, so cannot do this before computing 04,5 for all j}
m.j < 0t, MLP (W,)
return mij
Table 3. Hyperparameters used for WyckofIDiff'
PARAMETER
VALUE
MAX. TIMESTEP T
1000
GENERAL
MAX. ATOM
No
100
MAX. NUM ATOMS OF AN ELEMENT P
54
NUMBER OF GNN LAYERS,
DIMENSION OF h DIMENSION tos
256
GNN
16
ACTIVATION FUNCTION
SILU (SEE EQUATION (11))
NUMBER HIDDEN LAYERS
2
MLPs, GENERAL
ACTIVATION
SILU
MLPs IN M.
HIDDEN DIMENSION
2(DIM(h)) + DIM(h 544
PROBABILITY DECODING MLPS
HIDDEN DIMENSION
2DIM(h!) = 512
OPTIMIZER
ADAMW (LOSHCHILOV HUTTER, 2019)
LEARNING RATE
2. 10
TRAINING
BATCH SIZE
256
NUMBER OF EPOCHS
1000
A.3. Hyperparameters and Training Details
Training of model required approximately 38 hours single NVIDIA A100. Hyperparameters for WYCKOFFDUF and its training can be found in Table 3. The activation function SiLU (Ramachandran et al., 2017) is given by
exp(x)
Si)U(x) = x 1 + exx((x)
(11)
We did not perform any hyperparameter search.
"See also, e.g., https::/ppttorc...o
htmll
14

---

WyckoffDiff
A Generative Diffusion Model for Crystal Symmetry
A.4. A Note on Scalability
bottleneck our method is that we are operating on complete graphs, meaning that for space groups with many positions, the number of edges in the graph increases quickly. On the other hand the data dimensionality is fixed for certain space group, and more atoms in the unit cell does not change that. E.g., in Figure 1, the number of Cs atoms occupying the "c" position is represented by an integer, so increasing this from 0 to, say, 4, doesn't tt affect the dimensionality of the data. Increasing the size of the set of elements in the materials (e.g.. increasing N₂) and increasing the maximum number of atoms occupying an unconstrained position (i.c., P) will add additional computational overhead as, c.g., the backward transition requires summing over all possible values variable.
B. Implementation Details of Compared Methods
For all methods, used the official public implementatioons1101 and we train all methods for 1 000 epochs, We specify further details below.
B.I. COVAE
For CDVAE, we used the hyperparameters used for the MP20 dataset by the original authors, except for the learning rate which we lowered to 2- 10 as the default value led to instabilities in the training.
B.2. DiffCSP++
For DifCCS+++, we used the hyperparameters specified by the original authors for the MP20 dataset.
B.3. SymmCD
For SymmCD, used the hyperparameters specified by the original authors for the MP20 dataset, except for the number of training epochs and batch size, which we reduced to 100 and 256, respectively, to ensure fair comparisons.
When generating materials using SymmCD, we encountered an issue where the length and angle matrices contained NaN, Inf, or extremely small values. To facilitate subsequent evaluation, we filtered out those invalid materials.
C. Wren Energy Histograms
We show the similarities of generated material distrubution across all model versions W WYCKOHDDr--agginn WYCK- OFFDIIFTunifoom, and WyckooDDiF--eroo, in Figure 4.
wBM-train
WBNtttrain
WBM-train
050--0.552
0.55-
03552
090=-0.033
090=00.033
09033-0003
0.6
2106-2.316
0.6
010m-1 999
0.6
910m 308
Denatyy
050= 0.486
Denaty
Q50=
Denaly
050= 0.495
090=0.055 055
090=0.11 1.1
090=0.00 0.043
0.4
Gen-unfiltered
910m22 135
0.4
010m-2 2511
Gen-unfitered
0.4
910m-2 2272
Gen-unfiltere
050- 0.564
050- 0.527
050-00.
080-0.008
0.0
4
-3
22
0.0
44
2 Formation energy (eVistom)
0
0.0
44
-2 Formation energy [eViatom)
0
Formation energy [eVatom)
(a)
(b)
(c)
Figure 4. Distribution of formation energies predicted by Wren for, (unfiltered) generated protostructures, novel generated protostructures, relative to the training set for the model. Protostructures are generated by (a) SYcckorDDeE--aaggnn (b) DiFF-uuornn (c) Dickor-Din--zeros, Q10, Q50,and Q90 are the 10th. 50th, and 90th percentiles respectively.
Phttps: htttps://githube wttp:/..oocoooieeeeeeee 93/ccaae "https: //gi thub. pttporrr////ffpp-pppppp FFCSP-PP/
"https://gtt thub. com htt.://iasmaaaaakkaammm ratpssmaark//y/aa com////bbammarrk///mmcn
15

---

WyckoffDiff- A Generative Diffusion Model for Crystal Symmetry
Table Results the Carbon24 dataset. Due overall very low novelty, we settled for only 000 protostrrcturss, and 000 novel protostructures, for computing statistics.
NOVEL
Nov. t (%)
UNIQ. (%)
UNIQ. (%)
MODEL
FWD 4
FWD 4
NOv./MIN. 1
CDVAE
110 5.62
5.30 + 1.45
8.4
91.6 7.5
16.7 2.10
3
D1FFCSP++
4.12 1.53
1.40 + 0.46
0.60
38.6 4.93
22.3
2
SYMMCD**
11.4 + 1.85
6.53 + 1.72
16.4
21.7 33.1
6
WYCKOFFDIFF-UNIFOR
0.78 = 0.14
0.87
5.69
8.39
23.8 2.93
14
OURS
WYCKOFFDIFF-MARIIA
0.78 + 028
0.47
16.4
2.70
29.0 3.42
14
WYCKOFFDIF--ZEROS
0.89 = 0.21
1.40
1.52
4..0
27.8 1.95
12
Table The number of unique and novel prototypes among 000 novel protostructures from models trained on the Carbon24 datasct. Due to overall very low novelty, we settled for only 000 novel protostructures for computing statistics.
# UNIQUE & NOVEL PROTOTYPES
MODEL
CDVAE
167
DIFFCSP++
223 + 15
SYMMCD
217 1 38
WYCKOFFDIFF-UND MYCKOFFDIFF-MARG WYCKORFDIR-ZZEROS
237 + 25
290 + 48
278 1 15
D. Validity of Materials
Other related works (e.g., CDVVE (Xie al., 2022) and subsequent works) present two metrics on "validity"" of materials.
Structural validity A material is determined to be structurally valid if the distance between two atoms is less than 0.5 A. As we are only concerned with protostructures (and thus do not consider the exact geometries), this metric not applicable in our study.
Compositional validity If a materials has an overall neutral charge according to SMACT (Davies et al., 2019), is determined to be compositionally valid, which is something that can be computed for protostructures. When computing this on the novel protostructures, this number is 81.8 + 0.3% for CDVAE, 87.1 0.51% for DiffCSP++, 86.3 = 0.28% for SymmCD and 85.9 + 1%, 87.7 + 0.4, and 86.1 + 0.3% for WYCKOFDIFF-niffmm WYCKOFF respectively. However, the term "validity" in this case should not be taken as prerequisite for real material, as some systems do not fulfill this (e.g., metals with diffuse non-local bonds). Indeed, the validity of the materials in WBM is 87%, and it is hence not expected (nor desirable) to have this number any higher.
WYCKoFFDiF--aargnna and
E. Results on Carbon24
As an additional experiment, we used Carbon24 (Pickard, 2020). We used the same training set as in the DiffCSS++ repository, and all baselines used hyperparameter configurations from their corresponding repositories (see above). We used the same hyperparameters for WYCKOFFDITF as for WBM, apart from training for 4 000 epochs match the baselines, and setting P 24.
However, all models strggled general to generate novel protostructures, probably due the dataset containing only single element, and novel protostructure hence needed to be a novel prototype. due to this low novelty, we limited the study to compute statistics on (000 protostructures, and 000 novel protostructures. We present the numbers in Tables 4 and
16

---

WyckoffDiff A Generative Diffusion Model for Crystal Symmetry
F. Novel Stable Materials
Low formation energies are only indirectly related to stability; the thermodynamically stable material at a composition is the one with the lowest formation energy compared all alternative competing phases and linear combinations of phases, which spans the called convex hull of thermodynamical stability (see, e.g., (Bartel al., 2020)), Le., in order do determine if novel material is stable, the formation energy needs to be compared with the convex hull. Deriving the formation energy of material and computing the convex hull is described below.
E.1. Formation Energy
Formation energy is calculated by taking the total energy of material and subtracting the sum of elemental solid energy for each element present in the material. A negative formation energy therefore implies a lower energy state of the material relative to its elemental components. In turn, the formation energy proves that the material will not decompose into its elemental components.
F.2. Convex Hull
Plotting the formation energies of the materials and its corresponding elemental solid energies in a diagram constructs phase diagram. Materials that holds the lowest formation energy the phase diagram forms convex hull. The convex hull constructs serves as the line stable materials, meaning: a new crystal structure is discovered but has higher formation energy comparison the convex hull, the new crystal structure will decompose into its closest stable neighbors on the convex hull; whereas if the new crystal structure has lower formation energy in comparison the convex hull, the new material is novel and stable. The novel stable material is then part of convex hull, redefining the line of stable materials.
G. Supplementary Details on Materials Discovery Demonstration
G.1. Additional Protostructures
As described Section we performed a selection of three chemically interesting materials, whereas was noted that there where a total of eight fluorides with distinetly below the convex hull. In Table we list the materials sorted on energy distance from the convex hull of WBM and Materials Project (MP), up the final selected structure.
Table 6. Listed structures to the final included selection of interesting chemistry. The top section is the eight fluorides with formation energy distinctly below the convex hull. Selection three examples with interesting chemistry out WYCKOFFDIFE generated crystal structures close to or below the convex hull of WBM and Materials Project (MP).
E form. [eV/atom]
E above hull [eV]]
Protostrucuture
-2.6103
-0.0322
A228CD.cF40.225.c..a...ooo
-2.043
0.0194
236c_hR24.148...f.b:B...
-3.2550
-0.0097
ABBC.oP99.225...aa::--1111
-2.4313
-0.0076
A633:mC22.11211jaa.d1--11
-3.122
0.0068
-26064
0.0038
A68C cF32 225 a:
-3.6294
0.0027
A68C mC1 12 12
33.484
0.0019
AHC._h+a_194.ac....a-----
2.5369
0.0009
225
33.977
0.0003
228137
-0.0003
AJD.hh224.16..ebbFFF-GG
22.5113
0.0002
2.77786
3..7 10-05
ABL.cP__22_.abbcaa-
33.112
6.16
10 10-05
www.hh...4..cd.a:11.
33.1550
6.20
10
A84C oC24 63 c fg c:Ca-0-$
-2.6801
8.25
10 06
A2BC hR12 a a:
-1.2222
0.0001
17

---

WyckoffDiff A Generative Diffusion Model for Crystal Symmetry
G.2. Density Functional Theory Supplementary Details
In order maintain compatibility with MP and WBM dataset, all DFT calculations and post-corrections (MaterialsPro wtt.ss/ooocetootttaaiiaiiie ibimityy (Jain et al., 2011) performed using INCAR settings, KPOINTS and pseudo-potentials defined by Pymatgen (Ong et al., 2013). Caleulations where converged to atleast le-4 eV in total energy in electronic steps.
G.3. Previous WyckonrDiF--arginal
The structure A2BC.hR12.116.cc
Ca-I-P (Ca2PI) found using previous iteration of the WyckoffGNN architechure where we did not use softmax in the message-function Mj, but instead used the raw outputs of the neural network (see Algorithm 4), and encoded the degrees of freedom of the position using a binary representation, i.c., constrained or unconstrained position instead of 1, 2, or degrees of freedom.
18


Reviews:
[{"review_id": "MbmuSo5c2a", "reviewer": "Reviewer_btpV", "summary": "This paper proposes a novel generative model, WyckoffDiff, for generating Wyckoff representations of crystal materials. By applying discrete diffusion models directly to Wyckoff representations, WyckoffDiff can generate diverse protostructures for crystal materials. Additionally, WyckoffDiff leverages GNN models to derive Wyckoff representations through message passing over a complete graph.\n\n## update after rebuttal\nAfter reviewing the rebuttal and considering the comments from other reviewers, this reviewer remains concerned about the limitations of the proposed approach. While generating protostructures is certainly a viable strategy, it necessitates additional postprocessing to obtain full-atom structures. Moreover, the paper would benefit from clearer articulation of its motivation, more thorough comparisons with existing methods, and additional details regarding certain implementation steps. As such, the reviewer currently leans toward maintaining the original score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their comments, and we are happy to see that the reviewer agrees that incorporating symmetries is crucial for efficient material generation, and that our approach of applying discrete diffusion for working with the Wyckoff representation is reasonable. Below, we address and try to clear out the questions and concerns raised.\n\n## We have compared with very competitive baseline methods for material generation, without modifications\nThe motivation for the paper is that materials typically exhibit high symmetry, and we develop a method which incorporates that explicitly. We have compared with very competitive material generation baselines which span not incorporating symmetry (CDVAE), respecting symmetry but relying on templates (DiffCSP++), and respecting symmetry without relying on any templates (SymmCD). While these do not tackle the generation of protostructures explicitly, any generated material will of course _have a protostructure_, and a good method for material generation should also be able to generate materials with proper protostructures. Note that we did not modify the baseline methods, but simply computed the protostructures from the generated crystal structures (line 296 col 2). The reviewer do not specify which other baselines they think should be compared with (they also say that _\"There are no critically essential related works missing\"_), but the two other baselines mentioned in other reviews are FlowMM and WyCryst. The former does not explicitly enforce symmetry and would thus fall in the same category as CDVAE, and while the latter is conceptually similar to our, it faces some issues related to handling positions with 0 degrees of freedom (which we discuss in the response to Co5n). Also, the public code does not include code for unconditional generation which is the task we are facing, and the paper does not describe the method well enough for us to implement it ourselves (for example, the underlying neural network is not described, and generation is not described how it is done in practice, only training). \n\n## We use various metrics in our evaluation, not only stable materials\nWe respectfully disagree with the reviewer that we primarily rely on showcasing newly generated samples that fall below the convex hull. The comparison of realized structures against the convex hull is a proof-of-concept study which highlights that out method can indeed generate practically useful structures. This is not part of the comparison against other methods. In sec 4, however, we have tried to showcase various metrics which quantitatively highlight different aspects of the generated protostructures, and we have now added additional (see response to j5npy) such metrics. It is difficult to evaluate discrete generative models, as for example a metric such as uniqueness will not be $100 \\%$ even for a \"perfect\" model. From a materials discovery perspective we would like to have high novelty, but as we also write in the response to pC7E, we can obtain 100\\% novelty for all models by applying a filter that removes all non-novel materials, and then it is the time per generation that becomes the important metric, which is why we include the nov./min. metric. We will extend the discussion related to Table 1, including these aspects, in a revised version of the manuscript.\n\nWe think that reporting variations in metrics is very important to show the sensitivity of the methods.\n\n## Generating protostructures is an intermediate step, and we demonstrate how full geometries can be obtained\nIt is true that generating protostructures is an intermediate step, but as we show in section 5, full geometries can be obtained from them. As we discuss in section 6, separating these processes opens up the possibility to both do additional filtering of the protostructures to focus on the most promising ones and hence use resources where they are most needed, but also use specialized methods for the different steps. Hence, this approach introduces more flexibility for using methods tailored for respective task (e.g., obtaining the positions by using a large pretrained ML potential for DFT relaxation (as we do in sec 5) instead of training a diffusion model. While full structures is the end goal, dividing the process into multiple steps is a novel design and a strength in our model that offers unique advantages compared to the other methods. By the successful demonstration of the method as useful for materials design in the proof-of-concept study in sec 5, we think this can inspire continued development and improvements also by others.", "claims_and_evidence": "Yes, the claims are well-supported by the empirical evidence.", "methods_and_evaluation_criteria": "The proposed method is well-suited for the crystal material generation problem, and the chosen evaluation metrics are appropriate.", "theoretical_claims": "This paper does not contain theoretical claims.", "experimental_designs_or_analyses": "From the perspective of protostructure generation, the empirical design choices appear reasonable. However, the experimental results primarily compare previous baselines in the context of protostructure generation rather than full geometry generation, even though the baseline methods were originally designed for the latter. This raises concerns about how these baselines were adapted for protostructure generation. Without proper adaptation, the comparison may be misleading or unfair.", "relation_to_broader_scientific_literature": "The key contributions of the paper are mainly related to discrete diffusion models and crystal material generation.", "essential_references_not_discussed": "There are no critically essential related works missing from this paper. While some may exist, they can be incorporated later.", "other_strengths_and_weaknesses": "Strengths:\n\n(1) The paper is well-written, and the motivation behind the idea is clearly articulated.\n\n(2) Incorporating symmetries is crucial for efficiently generating highly symmetric crystal materials.\n\n(3) Applying discrete diffusion for generating Wyckoff representations is a reasonable approach.\n\nWeaknesses:\n\n(1) The empirical comparisons are currently insufficient, as the selected baseline methods are limited. Additionally, the proposed method does not demonstrate clear superiority over the baselines. The reviewer suggests that the comparison should focus on generating complete crystal symmetries, as this is the ultimate goal. Evaluating only Wyckoff generation is not entirely convincing.\n\n(2) A major limitation of WyckoffDiff is that it cannot directly generate the final crystal geometry. Instead, it only produces intermediate Wyckoff representations, requiring additional steps to obtain the complete crystal structure.", "other_comments_or_suggestions": "(1) The reviewer suggests incorporating more appropriate baselines. While it is understandable that existing works rarely follow the same problem formulation, the current evaluation of the proposed method primarily relies on showcasing newly generated samples that fall below the convex hull (as measured by oracle models). However, this approach is not entirely reliable.\n\n(2) The current empirical results are not well-structured. In Table 1, WyckoffDiff does not demonstrate clear superiority in terms of novelty or uniqueness compared to the selected baselines. The reviewer suggests that the authors should at least highlight the best-performing model for each metric and consider whether all reported variations are necessary.", "questions_for_authors": "The primary limitation of the proposed approach appears to be its inability to generate the full geometry of crystal materials.\n\n(1) How are the baseline methods adapted for protostructure generation?\n\n(2) Is protostructure generation sufficient for material design? Given that the ultimate goal is to generate complete material geometries, it seems that the proposed WyckoffDiff model falls short in this aspect.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "L9X3ZWJqlr", "reviewer": "Reviewer_pC7E", "summary": "The paper introduces WYCKOFFDIFF, a novel framework for generating crystal structures using a discrete diffusion process that inherently preserves symmetry. By representing crystal protostructures based on Wyckoff positions, the method partitions these positions into constrained (fixed) and unconstrained (flexible) categories, effectively encoding the intrinsic symmetry of crystals. The framework adopts the D3PM generative process alongside a novel WyckoffGNN backbone to model the occupancy of each Wyckoff position, enabling both symmetry-aware generation and rapid sampling. Additionally, the paper proposes the Fréchet Wrenformer Distance, a symmetry-sensitive metric for evaluating the quality of the generated structures. The authors finally validate model generation results by realizing the crystal structure to actual materials in the lab.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are very happy to see that the reviewer thinks our method is both original and interesting. We address the concerns related to the comparison to CDVAE and the proposed ablation study below. \n\n## While generating many novel structures, CDVAE does not generate symmetrical materials, which are the most interesting\nFirst, as the reviewer rightly points out, attaining a high novelty in itself is not enough (nor difficult), unless the materials are also at least \"valid\" and have interesting properties. We discuss validity in a later section. Regarding what materials are interesting: a significant portion of known crystalline materials, such as metals, salts, intermetallics, and covalent solids like diamond, silicon, etc., are built from small unit cells with ordered arrangements of atoms in a high degree of symmetry. As discussed in the introduction of our paper, this is also usually reflected in databases of materials, where one often sees a high representation of materials with space groups of higher symmetry. Hence, it is ultimately a (not uncommon) choice in this work to focus on materials in this category for materials discovery (which is not saying that this is the _only_ interesting category). If one do not enforce high symmetry in a generative model, it has to learn this property from data, which makes it more difficult to appropriately reproduce this aspect. \n\nIn the initial submission, we evaluated the quality of the generated materials using FWD, where CDVAE has a much higher FWD than the alternative methods, also when **computed only on the novel materials**. This suggests that CDVAE does not generate materials that respect the symmetries of the training set. This can further be assessed by considering the space group distributions: 36 \\% of the materials generated by CDVAE fall into space group 1, i.e., no symmetry, and $>90$ \\% of the materials belong to spacegroups 1-15 (the lowest symmetry groups). The corresponding numbers for the WBM dataset (the training split used in our paper) are 0.3 \\% (SG 1) and 13 \\% (SG 1-15). As we sample from the empirical SG distribution, we will follow this distribution by construction. We appreciate that the reviewer brings this up, and we will add this discussion to line 315 col 2 to further elaborate on the numbers for CDVAE (apart from the FWD), and add the numbers related to space groups.\n\nAdditionally, in our opinion, it is a less important feature for these methods to have a very high novelty, as long as it is reasonably large. It is trivial to extend any generative method with a final “novelty filter” step (essentially taking a rejection sampling approach, only keeping novel materials). This is arguably the most reasonable approach in a materials discovery setting to not spend resources on duplicates of training data, and thus ensures a novelty of 100\\% by construction. Hence, the “raw novelty” is less relevant, and the more important metric is the novel materials/minute after a rejection sampling step. Using this criterion, we see that WyckoffDiff performs much better than CDVAE (119-159 vs 71 Nov/Min; see rightmost column in Table 1), in addition to having a much better FWD on the filtered set of (only novel) generated materials. See response to j5np regarding the novelty of WyckoffDiff.\n\n## Removing encoding Wyckoff positions would not generate protostructures\nThe task we are tackling is the generation of protostructures, which essentially contains information of space group, elements, and which Wyckoff position each element occupy. Performing an ablation where we use **D3PM without encoding the Wyckoff positions would hence result in only generating a set of atoms, from which it is not possible to construct a protostructure**. In other words, removing the Wyckoff positions from the representation changes the problem. We are therefore unsure what the reviewer means when suggesting this ablation, but are happy to answer any follow up comment.\n\n## We have now looked at validity\nWe took the advice from the reviewer about validity metrics, and computed the compositional validity metric based on SMACT used in DiffCSP++ on the novel materials. This number is 84-85 \\% for WyckoffDiff, while it is 81-82 \\% for CDVAE. However, this metric can be misleading if interpreted as a measure of the number of \"valid generations\" since it is based on a typical matching of the charge states of ions in a material which often, but not always, is fulfilled by actual systems (e.g., metals with diffuse non-local bonds). Indeed, the training set we use for our models, WBM, only has roughly 87\\% of this \"validity\". In other words, it is not expected (nor desirable) to have this number any higher. We are open to further suggestions of useful validity metrics to explore.", "claims_and_evidence": "The authors claim their method enjoys symmetry-aware generation and fast sampling. The claims are supported by the FWD score and Nov./Min respectively in Table 1.", "methods_and_evaluation_criteria": "The proposed method, which encodes crystal structures using Wyckoff positions and employs the WyckoffGNN, is specifically designed to capture the intrinsic symmetry of crystals, making it well-suited for the problem at hand. The authors evaluate their model on the WBM dataset, a well-established crystal structure dataset.", "theoretical_claims": "There is no proof.", "experimental_designs_or_analyses": "The experimental design, as presented in Tables 1 and 2, is generally well-structured. The authors evaluate the crystals generated by their model and compare them with existing methods using metrics such as FWD, novelty, uniqueness, and sampling speed. These evaluations provide insight into the quality of the generated structures and support the authors’ claims regarding symmetry-aware generation.\nHowever, some experimental results seem to challenge rather than reinforce the proposed framework. For instance, in Table 1, all variants of WyckoffDiff perform significantly worse than CDVAE in terms of novelty. The authors attempt to justify this by attributing it to training time and arguing that faster sampling compensates for the lower novelty, but this explanation lacks a rigorous analysis. Additionally, the authors define a high FWD score as an indication that the generated structures deviate significantly from the training distribution. While CDVAE has a high FWD, it also achieves a notably high novelty score, suggesting that it may be generating novel and valid crystals.\nTo strengthen their claims, the authors could incorporate a validity metric. If they can demonstrate that CDVAE produces novel but largely invalid structures, it would provide stronger support for WyckoffDiff’s advantage in generating physically meaningful crystal structures.\nAlso, the paper lacks an ablation experiment on Wyckoff position encoding. To showcase the benefits of encoding symmetry with Wyckoff positions, an experiment on generating crystals with D3PM alone without Wyckoff positions should be needed.", "relation_to_broader_scientific_literature": "Prior methods lack consideration of symmetry. This paper explicitly models symmetry with Wyckoff positions.", "essential_references_not_discussed": "In the reviewer’s knowledge, no.", "other_strengths_and_weaknesses": "Strengths: Encoding symmetry with Wyckoff positions is both original and interesting. \n\nWeakness: The benchmarking is not rigorous enough to convincingly demonstrate the proposed framework’s performance.", "other_comments_or_suggestions": "No.", "questions_for_authors": "I summarize my questions as below (See Experimental Designs or Analyses for more details):\n1. Do the authors have a more convincing explanation on the poor performance on novelty as in Table 1 and 2? \n2. Have the authors conducted an ablation experiment on encoding Wyckoff positions?\n\nIf both concerns are fully addressed, that is the authors show WyckoffDiff generates both meaningful and novel samples and the proposed Wyckoff method is indeed beneficial , I will change my evaluation.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "guYc5R1aGv", "reviewer": "Reviewer_j5np", "summary": "In this paper, the authors propose a symmetry-aware generative model for crystal generation. The proposed Wyckoff diffusion model generates a protype based on elements taking Wychoff positions instead of 3D positions defined in a unit cell like a few methods in literature. They show that using this with a discrete diffusion model and Frechet Wrenformer distance (FWD) for evaluation leads to a more expressive generative model generating diverse samples (wrt to symmetries) for crystal generation. Material Project dataset is used for benchmarking with other models on generation task.\n\n## post rebuttal\nI have read the rebuttal. The authors provide a satisfactory rebuttal and thus I keep my score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We greatly appreciate the comments from the reviewer that have helped improving the clarity of the paper and improved the numerical evaluation. We are happy that the reviewer agrees with us that using discrete diffusion for generation of materials based on Wyckoff positions is a reasonable and useful approach. We address the comments related to evaluation and make some clarifications below.\n\n## We have now computed more metrics compatible with protostructures\nIt should first be noted that since we are not generating full geometrical information in our method, metrics that require this are generally not accessible to us which applies to several metrics used in previous works. Hence, we have relied on FWD and others we can compute (Table 1), and statistics on prototypes (Table 2). \n\nThis also applies to the SUN metric proposed by the reviewer. We therefore adapted a protostructure version of SUN, SUN-Wren, where stability is computed by using the Wren model to estimate formation energies directly from the _protostructures_, which is then compared to the Materials Project (MP) 2023 convex hull, as for stability in FlowMM. For novelty, we compared relative to the _protostructures_ present in the training set WBM. Uniqueness can similarly be compared to other _protostructures_ in the generated set. We can then also calculate the stability ratio of our training set WBM relative the MP 2023 hull to compute a reference stability score that arguably is what all generative models trained on this data set should strive to match. We only apply this metric on structures containing elements with atomic number 1 to 83 as this is where Wren has been trained for. We find that the symmetry-based models obtain SUN-Wren ratio close to WBM while CDVAE produces protostructures that are to a much larger degree unstable. We think this emphasizes the importance of incorporating symmetry in the models, as this helps producing useful materials (see next section). Due to the short time frame in which we create these, they should be regarded as preliminary and we may need to update them in further responses (if the referees agree it is a relevant metric).\n\n## Low symmetry materials are not the interesting materials found in databases (related to line 31 and 34, col 2) \nOur notion of symmetry starts from classifying crystal structures into space groups, which are defined by a set of symmetry operators that map the crystal structure on itself. Space group 1 has the lowest symmetry, as there is no symmetry at all. See comment to pC7E for more on why higher symmetry is the interesting materials. In addition to that answer, FlowMM learns distributions that are invariant to SE(3) transformations, which DiffCSP++ (which is extension of DiffCSP) and SymmCD also do, but they additionally incorporate knowledge about space group symmetries, and that is why we benchmark against these.\n\n## We investigate additional dataset\nAs suggested, we ran experiments also on Carbon24, as we are mostly interested in the structural properties of materials and Perov5 only contains Perovskites which is a certain fixed structure. We find that all models in general struggle to generate novel protostructures: the novelty for DiffCSP++ and WyckoffDiff is a mere 1-2 \\%, while for SymmCD the number is 6.5 \\%. However, WyckoffDiff is again much faster, with 12-14 nov/min compared to second best SymmCD with 6 nov/min, and places in between DiffCSP++ and SymmCD in FWD(novel). CDVAE has novelty of $\\sim$ 7 \\%, but again has a much higher FWD than the other models. \n\n## We will clarify regarding low novelty\nWe think the low novelty arises due to \"memorization\" from the model, which is why we present the results of training for shorter times, showing that this increases the novelty. We will add a clarification about that in line 315 col 2. See also response to pC7E related to novelty.\n\n## Regarding scalability\nA bottleneck in our method is that we are operating on complete graphs, meaning that for space groups with many positions, the number of edges in the graph increases quickly. On the other hand the data dimensionality is fixed for a certain space group, and more atoms in the unit cell does not change that. E.g., in fig 1, the number of Cs atoms occupying the \"c\" position is represented by an integer, so increasing this from 0 to, say, 4, doesn't affect the dimensionality of the data. Increasing the size of the set of elements in the materials will make the unconstrained positions grow in size, but for WBM we without problem used 100 as the maximum atom number which is high.\n\n## Numbers in Table 1 exclude void and NaN materials\nMaterials are filtered before computing metrics, but we still make sure to generate enough materials to have 10k materials after filtering. We will change the text for the two stars ** in the caption of Table 1 to clarify this.\n\n## We will add references to WyCryst and FlowMM\nSee also discussion about WyCryst in response to Co5n.", "claims_and_evidence": "Yes, the claims in the paper are clear.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria are correctly done. Missing a few commonly used evaluation metrics for crystal generation: \n like validity, coverage, stability, cost, SUN rate, and SUN cost.  See references [2,5].\n\n2. SymmCD: Symmetry-Preserving Crystal Generation with Diffusion Models, Levy et al. [ICLR 25]\n\n5. FlowMM: Generating Materials with Riemannian Flow Matching, Miller et al. [ICML 24]", "theoretical_claims": "There are no theoretical claims made in the paper.", "experimental_designs_or_analyses": "Yes. \n\nThe proposed model in the paper generates protostructures and evaluates the novelty of the protostructures generated. Although given that other methods do have a protostructure-based model, it would be useful to have the usual evaluation metric for a few generation samples of each protostructure to have a fair comparison.", "relation_to_broader_scientific_literature": "The key contributions of the paper on using Wyckoff positions are not entirely novel (see reference [1]). Previously, space groups of crystals have been used [2] in addition to modeling atom positions, while [3,4,5] use E(3) symmetries for modeling crystals. Using the symmetries commonly found in crystals through Wyckoff position seems to be a reasonable choice and combining it with discrete diffusion is also useful. \n\n1.  WyCryst: Wyckoff inorganic crystal generator framework, Zhu et al. [Matter 24]\n\n2. SymmCD, Levy et al. [ICLR 25]\n\n3., 4.  DiffCSP and DiffCSP++, Jiao et al.[NeurIPS 23]\n\n5. FlowMM: Generating Materials with Riemannian Flow Matching, Miller et al. [ICML 24]", "essential_references_not_discussed": "Yes, most of the related works are covered in the paper. \nMissed reference in related works. \n\n1. WyCryst: Wyckoff inorganic crystal generator framework, Zhu et al. [Matter 24]\n2. FlowMM: Generating Materials with Riemannian Flow Matching, Miller et al. [ICML 24]\n\nA few sentences in the paper seem to require citations that are missing. Listing them below:\n\nln 31 [column 2]: 'Additionally, the infinite space of continuous coordinates also\nopens the risk of generating degenerate materials or structures outside of the symmetry proximity. '\n\nThis does not seem to be the case for FlowMM or DiffCSP, which uses E(3) symmetries using continuous coordinates on a torus or Euclidiean plane. \n\nln 34 [column 2]: 'Since materials of\nhigh symmetry are generally the interesting materials to explore, generation of large sets of low symmetry materials is\ninefficient.'\n\nln 85 [column 2] typo in AFLOW prototype label", "other_strengths_and_weaknesses": "## Strengths\n- The related work section is well written, and relatively new ideas like Wyckoff positions and Wren energies are explained well. \n- The proposed method of combining wchkoff positions modeling with discrete diffusion seems to be a reasonable choice for generating crystals. \n\n## Weaknesses\n\n- Missing a few commonly used evaluation metrics for crystal generation: like validity, coverage, stability, cost, SUN rate, SUN cost, see references [2,5]. \n- Additional dataset for experiments would have been useful.", "other_comments_or_suggestions": "It would be interesting to see additional material datasets in the experiment section, like Carbon24 or Perov5.", "questions_for_authors": "1. ln 34 [column 2]: 'Since materials of\nhigh symmetry are generally the interesting materials to explore, generation of large sets of low symmetry materials is\ninefficient.' What is low symmetry?\n\n2. What is the scalability of the proposed method? \n\n3. ln 322[ column 2] 'compared to baselines..' The explanation provided for lower novelty for Wyckoffdiff is unclear. \n\n4. Do the numbers reported in Table 1 exclude the void materials with 0 atoms?, Is it the same for SymmCD, where the NaN values are removed?\n\n\nand see Weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JvEjPM5htU", "reviewer": "Reviewer_Co5n", "summary": "The authors introduced WyckoffDiff, a discrete diffusion model for generating crystalline materials with explicit symmetry constraints. WyckoffDiff encodes crystals via protostructure, which includes a space group and Wyckoff positions. The authors evaluated their approach on materials benchmark (WBM dataset). They also proposed a new evaluation metric, the Fréchet Wrenformer Distance (FWD), which quantifies how closely the generated protostructures match the symmetry characteristics of the training data (analogous to FID).", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are happy to see that the reviewer thinks our approach is novel and that generating protostructures is an approach that can enable efficient material generation. We address the concern that we do not quantify how structures can be realized from protostructures, and elaborate on differences between related work, below.\n\n## We do address how protostructrues can be realized into full atomic structures, run a proof-of-concept study, and discover new materials\nIn section 5 (with further details in the appendix), we presented a study on how the generated protostructures can be realized into full structures, and following this simple approach, we also find materials outside of the WBM dataset that are below the convex hull (see fig. 3 for illustrations of the crystal structures and the corresponding phase diagrams), showcasing the effectiveness of WyckoffDiff in generating protostructures which subsequently can also be realized into stable materials. The aim of this study was to highlight that while protostructures alone are not enough, it is relatively straightforward to obtain full structures and doing this, we do actually find new materials, showcasing the usefulness of WyckoffDiff for materials discovery. We will add an additional mention of this to the abstract. \n\n## We will elaborate on differences to WyCryst and SymmCD\nWe agree with the reviewer that a further elaboration on the differences between our method can make the paper clearer. We will add this to the related work section, but to summarize:\n\n* **WyCryst:** This work is similar in the sense that it generates a Wyckoff-based description of the material (and then assign exact coordinates in a later step). However, it is based on a different Wyckoff-representation, and they do not discuss how the problem of positions with 0 degrees of freedom (in our work called constrained positions) are handled. This last point is something that we are very careful to respect, and the reason for why we separate the positions into \"constrained\" and \"unconstrained\". We can also not find that the describe how they handle the varying number of Wyckoff positions for different space groups (which we do by using our graph representation). Their work also focus on generating strictly ternary materials while we put no such restrictions on the materials.  \n\n* **SymmCD:** Apart from also generating positions, SymmCD takes on a different approach when assigning Wyckoff positions: essentially it starts by sampling a number M of \"representative\" atoms which is kept fixed, and then the element, Wyckoff position, and multiplicity of these respective atoms are generated (these atoms are representatives of the corresponding orbit). We, on the other hand \"start\" from all Wyckoff positions, and then generate which element(s) (if any) occupy each position, and their respective multiplicity, which is conceptually different. Given the results in sec 4, it also seems very effective.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "No.", "experimental_designs_or_analyses": "Yes.", "relation_to_broader_scientific_literature": "The idea is built on discrete diffusion models (D3PM) to generate the protostructures.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "**Strengths**:\n\n- Integration of crystal symmetry:\nThe idea of explicitly incorporating crystallographic symmetry into the generative modeling process via discrete diffusion models is novel. The authors introduce a \"protostructure\" representation that simplifies the crystal generation problem by explicitly encoding symmetry constraints, potentially enabling efficient material generation.\n- Evaluation and metrics:\nThe authors introduced a new evaluation metric, Fréchet Wrenformer Distance (FWD), that is designed to evaluate the quality of the generated materials by measring the similarity with the training set.\n\n**Weaknesses**:\n\nThe paper still has several weaknesses. \n\n- Unclear differences and advantages from existing methods: While the paper integrates crystal symmetry, the differences from existing symmetry-aware methods, such as SymmCD and WyCryst is not clear. For instance, Wyckoff also incorporates crystal symmetry through Wyckoff positions. The paper did not provide a sufficiently detailed comparison of the differences and advantages of WyckoffDiff over these methods. \n\n- Converting protostructures to atomic structures:\nWhile generating protostructures simplifies the generation step, the practical utility depends heavily on the subsequent realization into full atomic structures. The realization step might involve additional complexity (e.g., choosing internal degrees of freedom) that can affect the final material properties. The paper did not address or quantify how protostructures can be converted into a full crystal structure.", "other_comments_or_suggestions": "No.", "questions_for_authors": "Can you explain if is possible to convert the generated protostructures into complete atomic geometries?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This papers presents a discrete diffusion model to generate crystal structures based on the symmetry properties of the crystal. The model employs a representation that includes the elements, the space group and the Wyckoff positions occupied by the atoms. The generated samples do not fully describe a crystal but the authors offer a post-processing procedure to obtain atomic positions, by obtaining initial positions with PyXtal and then relaxing the structure with MACE. The main criticism by reviewers is that the model generates proto-structures instead of fully described crystals. However, in my opinion, the authors provide convincing arguments both in the rebuttal and in the paper about why this is not necessarily a limitation. While the results are not spectacular and the reviewers are not overly enthusiastic, the reviews do not reflect any fundamental flaws and I also did not find any in my reading of the paper. Furthermore, all reviewers agree that the paper is well written. In this regard, I think the paper has sufficient quality and would be appreciated by at least some members of the community and serve as inspiration for future work. For these reasons, I recommend the acceptance of the paper.
Decision: Accept (poster)