SymmCD: Symmetry-Preserving Crystal Generation with Diffusion Models
Authors: Daniel Levy
Siba Smarak Panigrahi
Sékou-Oumar Kaba
Qiang Zhu
Kin Long Kelvin Lee
Mikhail Galkin
Santiago Miret
Siamak Ravanbakhsh
Venue: ICLR 2025 Poster (2025)
Keywords: Crystals
Symmetry
Materials
Diffusion
Generative Models
Equivariance
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We generate symmetric crystal structures by generating an asymmetric unit along with symmetry transformations

Abstract: Generating novel crystalline materials has potential to lead to advancements in fields such as electronics, energy storage, and catalysis. The defining characteristic of crystals is their symmetry, which plays a central role in determining their physical properties. However, existing crystal generation methods either fail to generate materials that display the symmetries of real-world crystals, or simply replicate the symmetry information from examples in a database.  To address this limitation, we propose SymmCD, a novel diffusion-based generative model that explicitly incorporates crystallographic symmetry into the generative process. We decompose crystals into two components and learn their joint distribution through diffusion: 1) the asymmetric unit, the smallest subset of the crystal  which can generate the whole crystal through symmetry transformations, and; 2) the symmetry transformations needed to be applied to each atom in the asymmetric unit. We also use a novel and interpretable representation for these transformations, enabling generalization across different crystallographic symmetry groups. We showcase the competitive performance of SymmCD on a subset of the Materials Project, obtaining diverse and valid crystals with realistic symmetries and predicted properties.

Published as a conference paper at ICLR 2025
SYMMCD: SYMMETRY-PRESERVING CRYSTAL GENERATION WITH DIFFUSION MODELS
Daniel 2 Siba Smarak Panigrahi 1,2.3 Sékou-Oumar Kaba" 1.2 Qiang Zhu', Kin Long Kelvin Lee", Mikhail Galkin', Santiago Miret', Siamak Ravanbakhsh 'MeGill University, Mila, École Polytechnique Fédérale de Lausanne (EPFL), *University of North Carolina at Charlotte, Intel Labs
ABSTRACT
Generating novel crystalline materials has the potential lead advancements in fields such as electronics, energy storage, and catalysis. The defining character- istic of crystals their symmetry, which plays a central role determining their physical properties. However, existing crystal generation methods either fail to generate materials that display the symmetries real-world crystals, or simply replicate the symmetry information from examples in database. To address this limitation, we propose SymmCD" a novel diffusion-basee explicitly incorporates crystallographic symmetry into the generative process. We decompose crystals into two components and learn their joint distribution through diffusion: 1) the asymmetric unit, the smallest subset of the crystal which generate the whole crystal through symmetry transformations, and; 2) the sym- metry transformations needed to applied each atom in the asymmetric unit. We also use a novel and interpretable representation for these transformations, enabling generalization across different crystallographic symmetry groups. We showcase the competitive performance of SymmCD on subset of the Materials Project, obtaining diverse and valid crystals with realistic symmetries and pre- dicted properties.
generative model that
INTRODUCTION
Crystals serve the fundamental building blocks of many materials, and the discovery of new crystalline materials is expected to lead to diverse technological breakthroughs in fields ranging from energy storage to computing hardware (Miret et al. 2024). Generative models have the potential to greatly accelerate this process by proposing new candidates materials, and possibly conditioning on desired properties compositions.
The defining characteristic of crystals is their symmetry. These symmetries are Euclidean transfor- mations that map the crystal structure back itself. They can in general be some specific transla- tions, rotations, reflections and combinations of these. The set of these operations is called the space group of the crystal. known that space groups three dimensions fall into 230 distinct classes (Hahn et al.) 1983). The symmetry of crystal plays crucial role in determining its stability along with its thermodynamic, electronic and mechanical properties (Nye) 1985). classic example given by piezoelectricity. the ability of a material to generate an electric dipole under mechanical stress, which can only be manifested in materials lacking inversion symmetry.
Importantly, many of the recently proposed generative models for crystals do not generate samples with non-trivial symmetry: for example, the most frequently generated crystals by DiffCSP (Jiao et al. 2023) and CDVAE (Xie et al. 2022) are in the low-symmetry P1 space group, which is very rare in nature. MatterGen (Zeni et al. 2023) can generate crystals conditioned on a desired space group for space groups that are highly represented in the dataset, but they only recover the target space group roughly 20% of the time, dropping to about 10% for more symmetric space groups.
"Equal Contribution. Correspondence to: daniel. d meqi 11 11
Our code is publicly available at
baama /

---

Published as a conference paper at ICLR 2025
Thiffision
G pim
mondaae
X
Dencösing
Figure I: Illustration of the SymmCD method. Left. Representation of the unit cell of a 2D crystal with p4m symmetry where the asymmetric unit and the site symmetries of the atoms highlighted. Leveraging symmetry results in much more compact, yet complete representation. Right. Diffu- sion and denoising on the different components of the representation. For site symmetries and atom types, discrete diffusion is used. For the coordinate and asymmetric unit continuous diffusion is used. The diffusion and denoising processes preserve the space group symmetry.
Cheetham & Seshadri (2024) analyse the space groups of the stable crystal structures proposed by the GNoME model of Merchant et al. (2023), finding that the top 4 most commonly generated space groups account for 34% of all generated crystals, in less than 1% of crystals in the Inorganic Crystal Structure Database (Hellenbrand) 2004).
though each of those 4 space groups appears
In this work, we propose novel approach for generative modeling of inorganic crystals that ensures any desired distribution of space groups. The idea is similar to that of creating a paper snowflake, where we fold the paper create an unconstrained space, and after an unconstrained cutting of the paper in this space, its unfolding creates an object with desired symmetries. In the context of crystals, the unconstrained space called the asymmetric unil, which unit cell with no redundancy. To unfold the asymmetric unit, we need to generate the site symmetry of each atom inside the unit, i.e. the symmetry transformations that fix the atoms in place. In generative process, the atomic positions made consistent with generated site symmetries, enabling the unfolding of an asymmetric unit into a symmetric crystal; see Figure I
maximal subset of the
Crystals and their individual atoms have many different types of symmetries, so we need to address the issue fragmerrammntiiinn. By representing symmetry information using standard crystallo- graphic notations, such as Hermann Hermann-Mauguin notation or Wyckoff position labels (Hahn et al.) 1983). we are faced with many crystals and site symmetries that have a low frequency in the training data. To address this problem, introduce novel representation of symmetries binary matrices, which enables information-sharing and generalization across both crystal and site symmetries.
The main contributions of this work ating crystals through the unconstrained generation of asymmetric units, along with their symmetry information. II) We introduce physically-motivated representation for crystallographic site sym- metries that generalizes across space groups. III) We experimentally evaluate our method, finding that it performs on par with previous methods at generating stable structures, while offering signif- icantly improved computational efficiency due to our representation. IV) We analyse the symmetry and diversity of crystal structures generated by existing generative models.
follows: I) We demonstrate novel approach to gener-
RELATED WORK
There has been a growing body of work in developing machine-learning methods for crystal structure
modeling, including the development of datasets and benchmarks (Jain et al. [0013] Saal al. 2013
Chanussot et al. 2021 Miret et al. 2023 Lee et al. 2023 Choudhary et al. 2024). Recent work
has also focused on developing architectures that equivariant various symmetries Duval et al.
(202)) or are specifically designed to include inductive biases useful for crystal structures (Xie &
Grossman 2018 Kaba & Ravanbakhshi 2022 Goodall al. 2022
Yan al. 2022
2024).
2

---

Published as a conference paper at ICLR 2025
In addition to structure-based modeling, prior work has also generated full-atom crystal structures, structure are generated. A range of generation methods
in which all atoms of the
including VAEs (Noh et al., 2019 Xie et al.
2022), , GANs (Nouira et al. 2018
Kim et al.] 2020),
reinforcement learning (Govindarajan jan et al.
2023). diffusion models (Zeni et al.
2023 Yang et al,, 2024). and active
2023
Tiao et al. 2023 Kliifel et al. 2024).
flow-macching models (Miller et al.
learning based discovery (Merchant et al.
2023) have been used. These follow similar works in
3D molecule generation (Hoogeboom et al. 2022 Garcia Satorras et al. 2021 Xu et al. 2022), but extend them by incorporating crystal periodicity. In addition to full-atom crystal generation, prior work has also applied text-based methods to understand and generate crystals using language
models (Flam-Shepherd & Aspum--Gzzk] 2023 Gruver et al. 2024 Alampara et al. 2024).
Other works have pointed out the importance of symmetry of the generated structures. D Jiao et al. 2024), does so by using predefined structural templates from the training data and learn- ing atomic types and coordinates compatible with the templates. While this an interesting solution, we show that predefining the templates in this way severely limits the diversity and novelty of the generated samples. CrystalGFN (Al4Science et al. 2023) incorporates constraints on the lattice pa- rameters and composition based on space groups but does not guarantee that the atomic positions respect the desired symmetry. Finally, the concurrent works CrystalFormer (Cao et al.) 2024) and WyCryst (Zhu et al.) 2024) generate symmetric crystals by predicting atom symmetries. However, they directly use the labels of Wyckoff positions to encode symmetries, which does not enable gen- eralization across groups. The methods are therefore limited to generating from space groups that are common in the dataset. By contrast, our method generalizes across groups and can generate valid crystals even from groups that are rare in the dataset.
BACKGROUND
Lattices and unit cells Crystals are macroscopic atomic systems characterized by a periodic struc- ture. A crystal can be described as an infinite 3-dimensional lattice of identical unit cells, each containing atoms in set positions. We can represent a crystal with the tuple C = (L, [,, where L = (1,, 12, 1a) E R3x3 is matrix of lattice vectors, X E [0, 1)3xN represents the fractional co- ordinates of N atoms within a unit cell, and A E (0, 1 possible elements for each atom. The lattice describes the tiling of unit cells: the cartesian coordi- nates of atoms can be given by X LX, and if x; is the cartesian coordinate of an atom in a unit cell, then the crystal will also contain an identical atom xf Lj, Vj Z3.
matrix of one-hot vectors of Z
Crystal symmetries In addition to the translational symmetry of the lattice, crystals typically have many other symmetries. Understanding these symmetries is fundamental in characterizing crystals and directly relates to many of the properties of these materials. The space group G of a crystal is the group of all Euclidean transformations that leave the crystal invariant, i.e., that simply permutes atoms of the same type. As space groups are subgroups of the Euclidean group, their elements can be represented as (O, t), where E O (n) and t E RR3 with action on x E IR' defined as (O,t) Ox + t. The operations that are part of a space group can be generally understood as belonging to different types: translations, rotations, inversions, reflections, screw axes (combinations of rotations and translations), and glide planes (combinations of mirroring and translation).
Two space groups belong to the same type if all their operations can be mapped to each other by an orientation- preservinn Euclidean transformation (coordinate change). We denote the set of all space group types as G. In 3 dimensions, there are only 230 unique space group types. By choosing a canonical coordinate system, P of a space group G is the image of the homomorphism (O, t) + O, i.e the group obtained by keeping only the orthogonal parts of G. By contrast with space groups, any point group must least preserve a single point, that is the origin. By a similar procedure to space groups, we can classify point groups and find that there are 32 crystallographic point group types, consisting of inversions, rotations, and reflections. We denote the set of all point group types as P.
in general work only with space group types. The point group
Wyckoff positions Having classified symmetry groups, we can now also classify points of space using symmetry considerations. This will be important to our method, as we will seek to use these semantically meaningful classes to guide the generation process. Given a space group G. we say that two points x, x' E R are part of the same crystallographic orbit if there is (O,t) G such that
3

---

Published as a conference paper at ICLR 2025
(0,t)x=x' The orbits form partition of R they can be understood as the finest level of classifi- cation under G. We define the site symmetry group of a point x, Sx = ((O,t) E G (O,t)x x= x} as the subgroup of G that leaves invariant. It is clear that the site symmetry must be a point group (since translations do not preserve any point), and is a subgroup of P. From the orbit-stabilizer the- orem (see e.g. [Dummit & Foote (2004)), we find that the number of points in the orbit and in the unit cell is given by [PT/|xx Points in highly symmetric positions, therefore, result in smaller orbits. A point is in general position if its site symmetry group is trivial. In this case, there is one-to-one correspondence between points in the orbit and group members. If the site symmetry is non-trivial, a point is said be in a special position.
Points in the same orbit have conjugate site symmetry groups. Therefore, site symmetry groups related by conjugation can be understood as equivalent. This motivates coarser level of classifica- tion that will be very useful. Two points x, x' E R are part of the same Wyckoff position if their site symmetry group is conjugate. Wyckoff positions have a clear meaning: they classify regions of space in terms of their type of symmetry. The multiplicity of Wyckoff position is the number of equivalent atoms that must occupy that position and is equal to the [P|/|5| ratio introduced earlier.
Asymmetric Units is a small part of the unit cell that contains no symmetry but can be used to generate the whole unit cell by applying the symmetry transformations of the space group. An asymmetric unit will only contain single atom from each orbit.
The unit cell of crystal can further be reduced into an asymmetric unit, which
4 METHOD: SYMMETRIC CRYSTAL DIFFUSION (SYMMCD)
4.1 REPRESENTATION OF CRYSTALS WITH WYCKOFF POSITIONS
As explained in the previous section, crystal structure can in general be represented by the tuple (L, X. A). This representation has been used in previous generative models for crystals (Xic et al. 2022 Jiao al.) 2023 Luo et al., 2023 Zemi et al. 2023). However, fundamental limitation of a model based on this representation is that it does not leverage the inductive bias of crystal symmetry and offers guarantees for the crystal to satisfy anything but trivial space group.
We introduce an alternative representation that respects symmetry in addition to having many de- sirable properties. First, we explicitly specify the space group type of the crystal G E in the representation. Given the space group, instead of representing each of the N atoms individually with X E R3xN and A Z we represent the M/ crystallographic orbits; replicating the atoms within the orbit then creates the crystal. As explained in Section 3 the Wyckoff position identi- fies a set of orbits by site symmetries. Therefore, specifying the site symmetry and an arbitrary orbit representative is sufficient to identify crystallographic orbit. This corresponds to a repre- sentation of an asymmetric unit within the unit cell. We thus define the set of orbit representatives with their Wyckoff positions as the tuple C" = (k, X', S. A'), where k is a parametrization of the lattice (to be explained later), X' = [x],..., XMM E 333M are the representative's fractional co- ordinates the asymmetric unit, S = [ Sx E PM are the site symmetry groups and A' = ahyll E 7 are the atomic types.
From the set of representatives, we can go back to the representation X and A in a unique way. This is done by generating the orbits using the replication operation that depends on the group G and the site symmetry S. The replication operation essentially consists of applying all of the symmetry operations of the space group except for the ones included in the site symmetry group- The details of this operation are included in Appendix A Finally, the lattice L can be constrained to be compatible with the space group in a convenient way using the vector k E IR5 (Jiao et al. 2024):
(L) E k, where the B, E R333 is standard basis over symmetric matrices. This basis and the constraints on for each space group are described in Appendix
Our representation of crystals that explicitly takes into account symmetry is therefore given by the tuple C' = (G, k, X', S, A'). We convert crystal structures to this representation using the SPGLIB symmetry finding algorithm (Togo & Tanaka 2018b) provided by PYMATGEN (Ong et al.) 2013).
In addition to accounting for the symmetry, this representation of a crystal provides two important advantages compared existing methods. First, it provides the generative model with a powerful
4

---

Published as a conference paper at ICLR 2025
MX4
[Bu
[20]
pTN
(EA)
(122]
[31]
Figure 2: Crystal symmetry axes. The different axes describe the directions along which symmetry operations can occur. For each of the 15 axes, there are 13 possible symmetry operations.
physically- motiivated inductive bias. is known from crystallography that atoms are typically not lo- cated in arbitrary positions in the unit cell (royo|2013) Rather, it is energetically more favourable for atoms to occupy positions of high symmetry, e.g. special Wyckoff positions. The representation in terms of positions X does not make this explicit. The representation using Wyckoff positions (X',S) provides explicit supervision to the model and guides the generation process: the model decides in which type of high-symmetry position an atom should be located and generates position compatible with that type. Second, the representation in terms of Wyckoff positions is much more compact than the representation that operates on individual atoms. M is often significantly smaller than N. In the MP-20 dataset (a subset of the Materials Project dataset (Jain et al. 2013) for exam- ple, the average number of orbits is M = 4.7 whereas the average number of atoms per unit cells is N = 9.9, representing a fourfold difference F
We therefore eliminate the redundant information from the representation and increase the computational efficiency of our method.
4.2 SYMMETRY REPRESENTATION
A key component of our representation of crystals with Wyckoff positions is the encoding of the space group G and site symmetry groups S'. While there are many existing methods to encode these symmetries, they generally do not make explicit the commonalities between the site symmetries of Wyckoff positions in the same space group, and the commonalities between different space groups across crystal systems. This is an important limitation: because there are 230 space groups, not having representation that is common across space groups results in dividing the effective amount of data the model trained on by large amount. For example, in the MP-20 dataset (Jain et al.) 2013 Xie & Grossman 2018) 113 space groups out of 169 in the training set have fewer than a hundred samples associated with them, and specific Wyckoff positions in each group have even fewer samples. We propose method to represent the site symmetries of different Wyckoff positions and to encode the symmetries of space groups to address this shortcoming.
We represent atom site symmetries using binary representation based the oriented site symmetry symbol used by the International Tables for Crystallography to describe Wyckoff positions (Hahn al. 1983 Donnay & Turrell, 1974). The oriented site symmetry symbols denote generators of the site symmetry group along different possible axes, illustrated in Figure 2 In total, there are 15 possible axes of symmetry in a crystal, corresponding to each of the Cartesian axes, along with body and face diagonals.
Examples of possible symmetry operations along each axis include rotations and roto-inversions, as well as mirror symmetry along a plane perpendicular to the axis. There are 13 possible symmetries along each axis. Listing out the site symmetry operation along each axis yields 15 x 13 binary matrix, or equivalently 15 different one-hot vectors. There is an injective mapping between site symmetries and site symmetry matrix representations, so representative atom can be replicated to produce full orbit using this representation.
The space group G can also be encoded into a binary representation using similar scheme, by listing out the 15 possible axes of symmetry and listing out the possible symmetry operations along each axis. Unlike the point group symmetries of atoms, these space group symmetry operations may involve translations and include serew and glide transformations, leading to 26 possible symmetry operations. Further details are included in Appendix 4.1 As a part of this project, both
This is using the conventional unit cell, not the primitive unit cell. A conventional cell may be twice or four times big as primitive cell.
5

---

Published as a conference paper at ICLR 2025
Denoising model
Pre processing
Sample
Noty
1, Fg
Tralning
Symmetzy
Graph neural network
Pust-procee prec-ppocsssing
Noise prediction
Sampled structure
Wyrket
Replootion
projectoo
Denuiss to.
Compute Iees with
Figure 3: SymmCD training and sampling pipeline. For training (green), the crystal structures pre-processed to find the space group G along with the site symmetries S and set of orbit representatives inside the asymmetric unit. The denoising model a GNN with fully connected graphs, followed by decoder. For sampling (blue), the positions projected to the closest one compatible with their site symmetry. Then, the asymmetric unit is replicated to obtain the unit cell.
the site symmetry representations and the space group representations have been incorporated as functions in the XtXl software library (Fredericks et al. 2021).
4.3 DIFFUSION MODEL
We can now describe the generative model and training process. In SymmCD, the space group and the number of orbit representatives are first sampled from separate distributions obtained from data, such that the distribution over crystal structures is (C) (k, X', S, A' M, G) (M G) (G). We will seek model the conditional distribution p (k, X', A' M,G) with denoising diffusion
model (Sohl-Dickstein et al. 2015 Ho al. 2020).
We leverage our binary representation for incorporating crystal symmetry information (described in Section 4.2) and perform joint diffusion over lattice representation (k), fractional coordinates of atoms (X ), their types (A),, and the associated binary representation of site symmetry (S).
Diffusion process We consider separate diffusion process the different components of the crystal representation. We apply discrete diffusion from Austin et al.) (2021) for site symmetries and atom types. Rather than adding Gaussian noise in conventional diffusion, categorical features by multiplying probability vectors by a transition matrix and sampling from the new probabilities. Inspired by Vignac al. (2023). the transition matrices are parameterized that the process converges to the marginals from the data distribution for atom types and site symmetries. The loss function used for discrete diffusion on atomic types is
add noise to
LA E Ear-Catia, Dean-CCata,T.,.tuUUll Q().)
Σ CrossEntropyya, á.);
where ag is the initial one-hot encoding of the atom types for single representative and Q Th Q x is the cumulative product of transition matrices between timesteps, and å, the predicted denoised probabilities. The same loss function is used for site symmetries.
Continuous diffusion is used for fractional coordinates and lattice parameters, similar to Jiao al. (202)) The loss function for the continuous diffusion on lattice parameters is
L (0
[l]m @ ck (C, +)]]
where m is space group-dependent mask, and é the predicted denoising vector. The same loss function is used for the fractional coordinates, except that to capture their periodic nature, we use wrapped normal distribution WN((,, 1)3xM, We provide more details in Appendix D
Denoising network The architecture of the denoiser is message- message-passing neural network that op- erates on a fully connected graph of representatives, based on Jiao et al. (2023). Features for each representative h, are initialized using embedding of their atom types a; and their site symmetries

---

Published as a conference paper at ICLR 2025
Si, along with the graph-level features of the diffusion timestep t, the lattice features k, and an bedding of the space group G. At each layer, messages mij are computed between representatives i and j by applying an MLP to h, hj, and Fourier embedding of the vector X; x, to respect periodic invariance. These messages are then used to update h;. More details on the architecture included in Appendix [ Note that the denoising network is equivariant. is not necessary since the unit cell axes provide canonical reference system (Kaba et al. (023). We also found that using an equivariant denoising network like EGNN Satorras al. (2021) did not work well in part due to the fact that since we use periodic encodings, the crystal structure input has translational
symmetry. equivariant model may not be able to break that symmetry (Kaba & Ravanbakhshj 2023) resulting in a inability to output correct positions in the asymmetric unit (or unit cell).
Putting all together The algorithm for training our diffusion model is outlined in Algorithm | We use different loss coefficients mk. Ax', A and Ag to weigh the importance of the different com- ponents the model. The algorithm for sampling from the diffusion model is shown in Algorithm The full pipeline is summarized in Figure Since both the diffusion and the denoising process both operate only on the asymmetric unit, they fully preserve the symmetry of the crystal.
Algorithm Training SymmCD
1: Input: Dataset crystals
2: while not converged do
3: Sample a crystal C = (I, X, A) from dataset DD and a timestep t ~ Uniform(1, T)
4: Derive the asymmetric representation C' (G, X', A', from C
5: Add noise to k, X', A', and S':
6:
k, Vārko + v1 0g
Ck ~ ~0,1
7:
X Va,Xi VI O 6x ~ WV
8:
A, ~ Cal(AQ Qa.t)
9:
S., Cat(SQQ....)
10:
Use denoising network @ to predict &k. εx,, Á from noisy C, = X,, Compute losses Lx, Lx,, LAi, Ls'
11:
12:
Update the denoising network &i using total loss:
13:
L. XxLl + Lx AA + AsLss
14:
end while
Algorithm 2 Sampling from SymmCD
1: Input: Target space group G, Number of representatives M
2: Initialize:
3: Sample kt No0, I)
4: Sample X/- ~ (0, 1) 1)3xM
5: Sample A Pmarginaa AA)
6: Sample St ~ Pmuugginal ()) (site symmetries)
7: for t = T to 1 do
A' 8: Compute & êx,, A using denoising network ((-)
9: Sample 1,- X/ A S using ék. éx,, A',
end for
11: Project Sy onto nearest valid point group
12: Project X/, onto nearest Wyckoff position with that site symmetry
13: Replicate representative atoms X using site symmetries Sy to generate full crystal Xo
14: Output: Crystal structure Xo, Atom types Ag, lattice Lo
EXPERIMENTS
We test our model on de novo crystal generation using the MP-20 dataset (Xie et al. 2022), a subset of the Materials Project (Jain et al. 2013) consisting of 40,476 crystals, each with up to 20 atoms per primitive unit cell. The data preprocessed to use the conventional unit cell rather than the primitive unit cell, as the former has more conveniently expressed symmetries and constraints. A conventional unit cell may be larger than a primitive unit cell, which results in up to 80 atoms in the unit cell. We withhold 20% of the dataset as validation set, and 20% test set.

---

Published as a conference paper at ICLR 2025
MP20 DiT
COVAE
DITCCP
FlowMM
SymmCD (All SGs)
SymmCD (10 5Gs)
Figure 4: Proportion of space group symmetries of the dataset, and each method. The width of each color segment represents the proportion of crystals with that symmetry. From left to right, the first few space groups are: Pl, Fm3m, Cm, Pi, C2/m, 14/mmm, Pm3m, P6y/mmc, and Pm.
We empirically demonstrate our contributions, particularly in ensuring we generate crystals with de- sired symmetries while being competitive with existing baselines. We compare proposed method with four recent strong baselines: CDVAE (Xie et al. 2022). DiffCSP (Jiao et al. 2023). D Jiao et al. 2024) and FlowMM (Miller et al.] 2024). We retrained each method according their given hyperparameters, and generated 10,000 crystals each. We also consider a variant of our model (10 SGs) where only sample from the 10 most common space groups in the MP-20 dataset, similar to (Cao et al. 2024) f This is to provide a more nuanced comparison with other methods, which are not constraied in matching the space group distribution. This choice still captures large portion of the data distribution, since these are the most prevalent space groups.
5.1 SYMMETRY AND STRUCTURAL DIVERSITY
First, we evaluate the different methods on their ability to generate crystals with diverse structures and space groups. This aspect has not been investigated yet for the considered baselines, yet is significant in understanding if they generate realistic structures.
Space groups To detect the space group of the
Table 1: Template statistics for each model.
generated structures, we use spglib's 's symmetry find- ing method (Togo & Tanakal 2018a; Ong et al. 2013) with tolerance O.IA. The distribution of space groups of the generated structures is shown in Figure 4 It can be observed that while SymmCD matches the highly diverse data distribution, CDVAE mostly generates crystals with trivial P1 symmetry. and DiffCSP and FlowMM generate many crystals with low symmetry and generally have lower diver-
Method
# Unique
% in Train
New
Training Set
3318
100
CDVAE
797
28.7
568
DifCCSP
1347
43.2
764
DiffCSP++
1905
94.2
110
FlowMM
1291
41.7
753
SymmCD
2794
40.8
1654
sity of space groups. We also consider new quantitative metric characterize the space group distribution, deg. which is calculated as the Jensen-Shannon distance between the distribution of space groups of the generated structures and the test set. We report for the different methods in the rightmost column of Table 3 The results confirm that SymmCD and DifiCSP++ are the only methods that accurately match the distribution of space groups in the dataset.
Unique Templates We also evaluate the ability of the different methods to generate diverse crystal structures. We define a structural template to be a combination of space group and multiset of occupied Wyckoff positions, regardless of the atomic types in the Wyckoff position. Templates, also known as Wyckoff sequences, are used practice to classify crystals by their symmetry. They have the advantage of providing a notion of structure that is highly flexible, while being robust to pertur- bations of coordinates that do not change the position of atoms with respect to symmetry elements. Most potential templates have not yet been experimentally observed, motivating the development of methods that can discover materials with new templates (Hornfeck) (022).
The training dataset contains 3318 such unique templates. We examine the templates for the 10,000 crystals generated by each method, and report results in Table [ We find that SymmCD performs best out of all models, proposing unique and novel templates. This highlights important limitation of DiffCSP++. While it can produce diverse space groups and to certain extent diverse templates, since it uses pre-defined templates it fails to generate structures with novel templates. Our method does not suffer from this problem since it learns to generate templates.
Thhese space groups are numhered: 2, 12, 14, 62, 63, 139, 166, 194, 221, 225
8

---

Published as a conference paper at ICLR 2025
5.2 STABLE, UNIQUE AND NOVEL (S.U.N.) STRUCTURES
Regardless of their target application. gener- ative models for crystals should produce sets of crystals that are thermodynamically stable, unique (not duplicated within the predicted set), and novel (not already in the training data), or
Table 2: Percent of stable and S.U.N. samples pro-
duced from an initial set of 10,000 generated crys- tals for each method.
Initial Stable
Relaxed Stable
Relaxed S.U.N.
S.U.N. To this end, we adapt the evaluation pro-
cedure of MMlller et al.] (2024) to assess the ca- pability of our model to generate S.U.N. ma- terials. Thermodynamin stability is determined by estimating the energy of a material with re- spect to conver hull. The convex hull gives linear combinations known phases that rep- resent the lowest-energy mixtures of materials;
CDVAE
0.24%
4.42%
4.26%
DifICSP
7.82%
11.32%
8.92%
D
6.99%
11.36%
8.62%
FlowMM
4.26%
9.05%
6.49%
SymmCD (All SGs)
4.96%
9.34%
6.86%
SymmCD (10 SGs)
6.85%
10.92%
7.59%
a material has an energy above the hull, it is energetically favorable for it to decompose into a combination of stable phases and is therefore thermodynamically unstable. We assess the stability of generated crystals by estimating their energies using a pretrained CHGNet model (Deng et al. 2023). and comparing that convex hull computed for Materials Project (Riebesell al. (024))
We predict the stability of the 10,000 samples generated by each method, and then use CHGNet to compute relaxed structures for each crystal, which results in higher stability. Finally, we check whether the stable relaxed crystals are also unique and novel. Details of this procedure are included in Miller et al. (2024). The results are shown in T We see that SymmCD performs on par with the baselines. Sampling over the most common space groups results in more stable structures.
5.3 VALIDATION WITH DENSITY FUNCTIONAL THEORY RELAXATIONS
We carried out electronic structure calculations as a more accurate way to evaluate the crystal struc- tures generated by the different methods. Since these computations are significantly more expensive than the ones with CHGNet, we performed them on 100 structures sampled from each model. Con- cretely, these involved using the CP2K (Kühhe et al., 2020) suite of programs perform cell and geometry relaxations to assess the quality of generated structures based on their distances from "true" local minima. We expect a better method to produce structures with less distance traveled, smaller per-atom forces (i.e. energy gradient with respect to positions), and with fewer iterations.
In these calculations, we find that the convergence rates of methods are not significantly different (see Figure TT). Comparing expectation values of the maximum force after optimization, however, we see that SymmCD is significantly more successful in generating structures that are readily op- timized. The full results of this analysis, well the parameters used for CP2K can be found in Appendix F.6
5.4 PROXY METRICS
Although ultimately we care about the stability, novelty, and properties of crystals after structural relaxation, we also compare the different methods using the proxy metrics established by Xie et al. (2022). as they are cheap enough to compute over large set of generated samples and demon- strate the ability of different methods to capture the properties of target distribution. These metrics include heuristies of validity, coverage of the test set, and the Wasserstein distances between the dis- tributions of three properties of the generated samples and the test set: atomic densities dp. number of unique elements delem- and predicted formation energy dg. We also include dag. the Jensen- Shannon distance between space group distributions. Further details are included in Appendix F.5)
For each method, we retrained a model with 5 different seeds, generating 10,000 samples for eval- uation per seed. The results are shown in Table 3 Although SymmCD performs similarly to other methods for most metrics, it is more likely to generate structurally invalid crystals, possibly because it cannot easily see distances between atoms. We also note the large variance in the results across different seeds for each method.

---

Published as a conference paper at ICLR 2025
Table 3: Results for comparing the validity, coverage, and property distribution metrics.
Validity (%) (f)
Coverage (%) (1)
Property Distribution (
MP-20
Struct.
Comp.
Recall
Precision
dp
dg
degcm
deg
CDVAE
85.61 10.9
99.31 1.
99.47.0.11
0.70
0.24 .6
1.28
0.69
DifICSP
97.4433..10
82.50 1.34
99.55 1.5...
98.7311.34
0.18
0. 0.14_.oo
0.561.07
0.441.000
DifICSP++
99.4440.08
26.50
99. 29.72+ 00
99.61
0.12+.01
0.05_.o1
0.33
0.16g.009
FlowMM
90.67
83.2520.18
99.49 os
99.58-20.10
0.23
0.09_.02
0.08
0.50
SymmCD All sGs
90.341...9
85.81 10.87
99.58
97.7610.85
0.231..0
0.21
0.401.0s
0.1641..oc
SymmCD
92.30
87.13
97.33+
98.7840.60
0.53
0.21=.09
0.16
0.4692.002
5.5
COMPUTATIONAL EFFICIENCY
We demonstrate significant compu- tational efficiency gains and reduced memory footprint due to using a more compact representation based on crystallographic orbits. We com- pare our model to an equivalent model that looks at a full unit cell, rather than just the asymmetric unit. It also uses a fully connected graph to represent the atoms in the unit cell, but does not predict site symmetry representations. This makes the model essentially equivalent to DifICSP (Jiao et al. 2023 but with the same architecture and hyperparameters as SymmCD for consistent comparison. We compare the two representations for one epoch of training using 40GB of RAM and single NVIDIA MIG A100 and report the results in Table These results highlight SymmCD's memory efficiency and faster training capabilities.
Table 4: Computational efficiency of our representation
Asymmetric Unit (ours)
Conventional Unit Cell
Maximum batch size (t)
8192
512
Memory for 512 hatch size (4)
3.6 GB
31 GB
Time for 1 training epoch (1)
27 sec.
52 sec.
5.6 SCALING UP TO MPTS-52
To validate that SymmCD can scale up to datasets of larger crystals thanks to its compact represen- tation, we also validate our model by training it the more challenging MPTS-52 dataset (Baird al.) 2024). which contains crystals with up 52 atoms per primitive unit cell, and includes more chemical elements. elements. Further details of this dataset along with the results are included in Appendix F.I
We note that none of the baselines we compare against (CDVAF, DiftCSP, FlowMM, and Dif- FCSP++) present results for de-novo generation on this dataset. We can see from these results that SymmCD is able to scale to much larger crystals than those in the MP-20 dataset and is still capable of producing valid, stable, and novel crystals that match the training distribution of the dataset. The metrics reported are worse than those for MP-20, as this a more difficult dataset.
CONCLUSION
In this paper, we introduced a novel approach for generating crystals with precise symmetry proper- ties. We proposed to leverage asymmetric units and site symmetry representations within diffusion model framework. This approach ensures that the generated crystals inherently preserve desired symmetries while allowing greater diversity, computational efficiency, and flexibility in the genera- tion process. To encode crystal and site symmetries we introduced a new representation of crystal symmetries that enables information sharing across space groups, improving generalization when learning with a diverse set of crystal symmetries, Our results indicate that this method produces stable, novel, and structurally diverse crystals, with improved computational efficiency, showing promise for discovery in materials science. In this work, we focused on inorganic crystals, but SymmCD could potentially be promising for applications on molecular crystals and co-crystals, which also have non-trivial symmetries. Beyond materials, other data types such molecules and graphs often exhibit complex symmetries, and future work could investigate if symmetry constraints could also be useful in generative models for these modalities.
A limitation of our framework is that it makes more challenging to perform crystal structure prediction given composition since it relies sampling a space group first, and then a composition conditioned on the space group. Finally, an important area of future work in generative models for crystals is also to go beyond single crystals, and consider the generation of polycrystalline materials. These types of materials are common in applications, yet not suited to generation using single unit cells asymmetric units.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This project supported by Intel Labs, CIFAR and the NSERC Discovery grant. S.-O. K.'s re- search is also supported by IVADO and the DeepMind Scholarships, and D.L.'s research is partly supported by the FRQNT. Computational resources were provided by Mila and Intel Labs. We are thankful to Alexandra Volokhova, Victor Schmidt, Alex Hernandez-Garcia, Alexandre Duval, and Félix Therrien for helpful discussions.
REFERENCES
Mila A14Science, Alex Hernande7-GGacia. Alexandre Duval, Alexandra Volokhova, Yoshua Bengio, Divya Sharma, Pierre Luc Carrier, Michal Koziarski, and Victor Schmidt. Crystal-gfn: sampling crystals with desirable properties and constraints. arXiv preprint arXiv:2310.0490 2023.
Nawaf Alampara, Santiago Miret, and Kevin Maik Jablonka. Mattext: Do language models need more than text & scale for materials modeling? In Al for Accelerated Materials Design-Vienna Vienna 2024, 2024.
Mois Ilia Aroyo. International Tables for Crystallography. John Wiley and Sons Limited, 2013.
Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems, 19:11981--7999 2021.
Sterling G Baird, Hasan M Sayeed, Joseph Montoya, and Taylor Sparks. matbench matbench-genmetriees A python library for benchmarking crystal structure generative models using maeed splits of materials project structures. Journal of Open Source Software, 9(9)):6618, 2024.
Zhendong Cao, Xiaoshan Luo, Jian Lv, and Lei Wang. Space group informed transformer for crystalline materials generation. arXiv preprint arX1v:2403.15734 2024.
Lowik Chanussot, Abhishek Das, Siddharth Goyal, Thibaut Lavril, Muhammed Shuaibi, Morgane Riviere, Kevin Tran, Javier Heras-Doming Caleb Ho, Weihua Hu, et al. Open catalyst 2020 (oc20) dataset and community challenges. Acs Catalysis, 11(10):605-6677 2021.
Anthony K Cheetham and Ram Seshadri. Artificial intelligence driving materials discovery? per- spective on the article: Scaling deep learning for materials discovery. Chemistry of Materials, 36 (8):3490-3495, 2024.
Kamal Choudhary, Daniel Wines, Kangming Li, Kevin F Garrity, Vishu Gupta, Aldo H Romero, Jaron T Krogel, Kayahan Saritas, Addis Fuhr, Panchapakesan Ganesh, et al. Jarvis-lcaderben a large scale benchmark of materials design methods. npj Computational Materials, 10(1):93, 2024.
Daniel W Davies, Keith Butler, Adam Jackson, Jonathan M Skelton, Kazuki Morita, and Aron Walsh. Smact: Semiconducting materials by analogy and chemical theory. Journal of Open Source Software, 4(38):1361, 2019.
Bowen Deng, Peichen Zhong, KyuJung Jun, Janosh Riebesell, Kevin Han, Christopher J Bartel, and Gerbrand Ceder. Chgnet as pretrained universal neural network potential for charge-informed atomistic modelling. Nature Machine Intelligence, 109):1031 2023.
JDH Donnay and G Turrell. Tables of oriented site symmetries in space groups. Chemical Physics, 6(1):1-18, 1974.
David Steven Dummit and Richard M Foote. Abstruct algebra, volume 3. Wiley Hoboken, 2004.
Alexandre Duval, Simon Mathis, Chaitanya K Joshi, Victor Schmidt, Santiago Miret, Fragkiskos D Malliaros, Taco Cohen, Pietro Liò, Yoshua Bengio. and Michael Bronstein. A hitchhiker's guide to geometric gnns for 3d atomic systems. arXiv preprint arXiv:2312.07511, 2023.
11

---

Published as a conference paper at ICLR 2025
Daniel Flam- Shepherd and Alán Aspuru als, and protein binding sites directly in three dimensions as xyz, cif, and pdb files. arXiv preprint 23X.::2300.00708. 2023.
Aspuru-Guzik. Guzik. Language models generate molecules, materi-
Scott Fredericks, Kevin Parrish, Dean Sayre, and Qiang Zhu. Pyxtal: A python library for crystal structure generation and symmetry analysis. Computer Physics Communications, 261:107810, 2021. ISSN 0010-4655. doi: http://www.......aaaaaaaaaaaaaaa
10700.0000... URL http://ww/
sciencedirect com/
Victor Garcia Satorras, Emiel Hoogeboom, Fabian Fuchs, Ingmar Posner, and Max Welling, E (n) equivariant normalizing flows. Advances in Neural Information Processing Systems, 34:4181-
2021.
Johannes Gasteiger, Shankari Giri, Johannes T Margraf, and Stephan Günnemann. Fast and uncertainty- directional message passing for non-equilibrium molecules. arXiv preprint arXiv:2011.14115. 2020a.
Johannes Gasteiger, Janek GroB, and Stephan Günnemann. Directional message passing for molec- ular graphs. arXiv preprint 2rXiv:2003.03123, 2020b.
Rbys E. A. Goodall, Abhijith S. Parackal, Felix A. Faber, Rickard Armiento, and Alpha A. Lee. Rapid discovery of stable materials by coordinate-free coarse graining. Science Advances, 8(30): eabn41 17, 2022.
Prashant Govindarajan, Santiago Miret, Jarrid Rector-Brooks, Mariano Phielipp. Janarthanan Ra- jendran, and Sarath Chandar. Learning conditional policies for crystal design using offline inforcement learning. In Al for Accelerated Materials Design NewrIPS 2023 Workshop, 2023.
URL httpttt::///pereeeeeeee net. Eorum? wow.oorrni-----a-eee-------
Nate Gruver, Anuroop Sriram, Andrea Madotto, Andrew Gordon Wilson, Lawrence Zitnick, and Zachary Ward Ulissi. Fine-tuned language models generate stable inorganic materials as text. In The Twelfth International Conference on Learning Representations, 2024.
Theo Hahn, Uri Shmueli, and JC Wilson Arthur. International tables for crystallography, volume 1. Reidel Dordrecht, 1983.
Mariette Hellenbrandt. The inorganic crystal structure database (icsd)-present and future. Crystal- lography Reviews, 10(1):17-22, 2004.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in
neural information processing systems, 13:6840-6855 2020.
Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffu- sion for molecule generation in 3d. In International conference on machine learning, pp. 8867- 8887. PMLR, 2022.
Wolfgang Hornfeck. On the combinatorics of crystal structures: number of wyckoff sequences of given length. Acta Crystallograhhica Section A: Foundations and Advances, 78(2):149-154, 2022.
Anubhav Jain, Shyue Ping Ong, Geoffroy Hautier, Wei Chen, William Davidson Richards, Stephen Dacek, Shreyas Cholia, Dan Gunter, David Skinner, Gerbrand Ceder, et al. Commentary: The ma- terials project: A materials genome approach to accelerating materials innovation. APL materials 1(1):011002, 2013.
Rui Jiao, Wenbing Huang. Peijia Lin, Jiaqi Han, Pin Chen, Yutong Lu, and Yang Liu. Crystal struc- ture prediction by joint equivariant diffuusion. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https: /ttp://::////eeeeee...... net wtte/forrmmm id --DdNN2 6m2Jk
Rui Jiao, Wenbing Huang, Yu Liu, Deli Zhao, and Yang Liu. Space group constrained crystal generation. arXiv preprint 0et:2402.03992,, 2024.
Oumar Kaba and Siamak Ravanbakhsh. Equivariant networks for crystal structures. Advances in Neural Information Processing Systems, 35:4150-4164 2022.
12

---

Published as a conference paper at ICLR 2025
Oakaar Kaba and Siamak Ravanbakhsh. Symmetry breaking and equivariant neural networks. arXiv preprint arXiv:2312.09016, 2023.
Sékou-Oumar Kaba, Arnab Kumar Mondal, Yan Zhang, Yoshua Bengio, and Siamak Ravanbakhsh. Equivariance with learned canonicalization functions. In International Conference on Machine Learning. pp. 35546-15566 PMLR, 2023.
Sungwon Kim, Juhwan Noh, Geun Ho Gu, Alan Aspuru-Guzik, and Yousung Jung. Generative adversarial networks for crystal structure prediction. ACS central science, 6(8):1412-1420, 2020.
Astrid Klipfel, Yaël Fregier, Adlane Sayede, and Zied Bouraoui. Vector field oriented diffusion model for crystal material generation. In Proceedings of the AAAI Conference Artificial Intel- ligence, volume 38, pp. 22193-22201, 2024.
Thomas D. Kühne, Marcella lamnuzzi, Mauro Del Ben, Vladimir V. Rybkin, Patrick Seewald, Fred- crick Stein, Teodoro Laino, Rustam Z. Khaliullin, Ole Schütt, Florian Schiffmann, Dorothea Golze, Jan Wilhelm, Sergey Chulkov, Mohammad Hossein Bani-Hashemian, Valéry Weber, Ur- ban Borstnik, Mathieu Taillefumier, Alice Shoshana Jakobovits, Alfio Lazzaro, Hans Pabst, Tiziano Müller, Robert Schade, Manuel Guidon, Samuel Andermatt, Nico Holmberg, Gregory K. Schenter, Anna Hehn, Augustin Bussy, Fabian Belleflamme, Gloria Tabacchi, Andreas GlöB, Michael Lass, lain Bethune, Christopher J. Mundy, Christian Plessl, Matt Watkins, Joost Vande- Vondele, Matthias Krack, and Jürg Hutter. CP2K: An electronic structure and molecular dynam- ics software package Quickstep: Efficient and accurate electronic structure calculations. The Journal of Chemical Physics, 152(11): May 2020. ISSN 0021-9606, 1089-7690. doi:
Kin Long Kelvin Lee, Carmelo Gonzales, Marcel Nassar, Matthew Spellings, Mikhail Galkin, and Santiago Miret. Matsciml: A broad, multi-task benchmark for solid-state materials modeling. arXiv preprint arXiv:2309.059:. 2023.
Youzhi Luo, Chengkai Liu, and Shuiwang Ji. Towards symmetry-aware generation of periodic materials. In seirth- Conference on Neural Information Processing Systems, 2023. URL htttps://openeeview net/ wtt.:eewweeeettooddddeeeeeee. 4vn1a2
Amil Merchant, Simon Batzner, Samuel S Schoenholz, Muratahan Aykol, Gowoon Cheon, and Ekin Dogus Cubuk. Scaling deep learning for materials discovery. Nature, pp. 1-6, 2023.
Benjamin Kurt Miller, Ricky TQ Chen, Anuroop Sriram, and Brandon M Wood. Flowmm: Generat- ing materials with riemannian flow matching. In Forty-first International Conference on Machine Learning, 2024.
Santiago Miret, Kin Long Kelvin Lee, Carmelo Gonzales, Marcel Nassar, and Matthew Spellings. The open matsci ML toolkit: A flexible framework for machine learning in materials sci- ence. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https:
//openzeview. fenet//foum?e.m d=ddd--2MMyyMPMd
Santiago Miret, NM Anoop Krishnan, Benjamin Sanchez-Lengeling. Marta Skreta, Vineeth Venu- gopal, and Jennifer N Wei. Perspective on ai for accelerated materials design at the ai4mat-2023 workshop at neurips 2023. Digital Discovery, 2024.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffuuson probabilistic models. In International Conference on Machine Learning, pp.
PMLR, 2021.
Juhwan Noh, Jaehoon Kim, Helge S Stein, Benjamin Sanchez-Leegellngg John M Gregoire, Alan Aspuru-Guzik, and Yousung Jung. Inverse design of solid-state materials via a continuous repre- sentation. Matter, 165):1370-1384, 2019.
Asma Nouira, Nataliya Sokolovska, and Jean-Claude Crivello. Crystalgan: learning to discover crystallographic structures with generative adversarial networks. arXiv preprint arXiv:1810.11203, 2018
John Frederick Nye. Physical properties of crystals: their representation by tensors and matrices. Oxford university press, 1985.
13

---

Published as a conference paper at ICLR 2025
Shyue Ping Ong, William Davidson Richards, Anubhav Jain, Geoffroy Hautier, Michael Kocher, Shreyas Cholia, Dan Gunter, Vincent L. Chevrier, Kristin A. Persson, and Gerbrand Ceder. Python Materials Genomics (pymatgen); A robust, open-source python library for materials analysis, June 2013. https:::t: / thu.. wwtp:/ccocmmaaereoo wtt.maaeeerrrrrrrrreooeooooooooe fttp:///atttyytttt../..
John P. Perdew, Kieron Burke, and Matthias Emzerhof. Generalized Gradient Approximation Made Simple. Physical Review Letters, 77.3865.
October 1996. doi: 10. 10.1103/PhysRev..et
Janosh Ricbesell, Rhys E. A. Goodall, Philipp Benner, Yuan Chiang, Bowen Deng, Alpha A. Lee, Anubhav Jain, and Kristin A. Persson. Matbench discovery a framework to evaluate ma- chine learning crystal stability predictions, 2024. URL
14920
James Saal, Scott Kirklin, Muratahan Aykol, Bryce Meredig, and Christopher Wolverton. Ma- terials design and discovery with tigh-throughput density functional theory: the open quantum materials database (oqmd). Jom, 65:1501-1509, 2013.
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural net- works. In International conference on machine learning, pp. 9323-9332. PMLR, 2021.
Jascha Sohl-Dickstee Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learn- ing. pp. 2256-2265. PMLR, 2015.
Atsushi Togo and Isao Tanaka. Spglib: software library for crystal symmetry search. https: 7/github. com/ https/ppphgbpppipiie 2018a.
Atsushi Togo and Isao Tanaka. Spglib: software library for crystal symmetry search. arXiv preprint 10Xiv:1808.015900 5, 2018b.
Clément Vignac, Igor Krawezuk, Antoine Siraudin, Bohan Wang, Volkan Cevher, and Pascal Frossard. Digress: Discrete denoising diffusion for graph generation. In Proceedings of the 11th International Conference on Learning Representations, 2023.
Logan Ward, Ankit Agrawal, Alok Choudhary, and Christopher Wolverton. A general-purpose machine learning framework for predicting properties of inorganic materials. npj Computational Materials, 2(1):1-7, 2016.
Tian Xie and Jeffrey C Grossman. Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties. Physical review letters, 120(14): 1453001, 2018
Tian Xie, Xiang Fu. Octavian-Euger Ganea, Regina Barzilay, and Tommi S. Jaakkola. Crys- tal diffusion variational autoencoder for periodic material generation. In International Confer- Learning Representations, 2022. URL ptetps://openset ew. net. forum?id- 0 03RLpj -ttc
Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. Geodiff: geo- metric diffusion model for molecular conformation generation. arXiv preprint 22X 2022.
Keqiang Yan, Yi Liu, Yuchao Lin, and Shuiwang Ji. Periodic graph transformers for crystal material property prediction. Advances in Neural Information Processing Systems, 35:15066-50080, 2022.
Keqiang Yan, Alexandra Saxton, Xiaofeng Qian, Xiaoning Qian, and Shuiwang Ji. space group symmetry informed network for o (3) equivariant crystal tensor prediction. arXiv preprint arXiv:2406. /2888, 2024.
Mengjiao Yang, KwangHwan Cho, Amil Merchant, Pieter Abbeel, Dale Schuurmans, Igor Mor- datch, and Ekin Dogus Cubuk. Scalable diffusion for materials generation. arXiv preprint 1oXiv:2311.09221 2023.
14

---

Published as a conference paper at ICLR 2025
Claudio Zeni, Robert Pinsler, Daniel Zügner, Andrew Fowler, Matthew Horton, Xiang Fu, Sasha Shysheya, Jonathan Crabbé, Lixin Sun, Jake Smith, et al. Mattergen: a generative model for inorganic materials design. arXiv preprint arXhv:2312.0366// 2023.
Ruiming Zhu, Wei Nong. Shuya Yamazaki, and Kedar Hippalgaonkar. Wyeryst: Wyckoff inorganic crystal generator framework. Matter, 7(10):3499-3488, 2024.
Nils ER Zimmermann and Anubhav Jain. Local structure order parameters and site fingerprints for quantification of coordination environment and crystal structure similarity. RSC advances, 10 (10):6063-6081, 2020.
15

---

Published as a conference paper at ICLR 2025
A REPLICATION
We define the replication operator R GxPxR3 - 2 This operation is defined by considering the group Sx x I'. with Ts being the group translations defined by the lattice. Sx Ts is the set of operations that preserve the position of x within the unit cell as opposed to within the crystal. We can then consider Then, denote by [G/ (Sx Ts)lo system of coset representatives where the translation parts are chosen to move only within the unit cell. This defines the set of operations that move position within its orbit and the unit cell. The replication operation then simply consists of applying all these operations:
coset decomposition of the space group with respect to that group G/ (Sx x Tg).
R (G, Sx.x) x) = ((O,t) x x
(O,t) (G x Ts)lo)
The representation in terms of individual atoms is then:
X
O R (R,
(2)
M
A repeat (a:, [G (Sx x Ts)])
(3)
where repeat (a, n.) repeats the vector times and [G (Sx Ts)] is the multiplicity of the orbit.
In our diffusion model, our predicted site symmetries S do not always necessarily correspond valid crystallographic point group. To get around this, we project $ the nearest point group that is subgroup of the given space group, as measured by the Frobenius Norm of their difference. Once point group is chosen, the PyXtal search closest wp function is used to get the nearest coordinates to X' that correspond to Wyckoff position with the given site symmetry, and X' is updated be placed on those coordinates (Fredericks et al. (02T). Finally, the representative atoms at the Wyckoff position are replicated, using operations implemented in PyXtal.
B LATTICE REPRESENTATION
We use the lattice representations derived by Jiao et al. (2024). as they are useful for constraining lattices to respect the symmetries of given space group. The authors found that any lattice matrix can be written as = Q exp(S) for some orthogonal Q (which we can ignore, as orthogonal transformations do change the lattice), and symmetric S. The matrix S can then be decomposed into a sum of the following basis lattices:
0
1
0
0
0
I
0
B1
1
0
0
B2
0
0
0
0
0
0
0
0
0
0/
0
1
0
0
0
0
0
(1
0 0
B,
0
B;
0
1
0
Bg
0
0
0
-2
0
0
with S >9 i=
B; They derive constraints on k; depending the space groups that crystal
belongs to:
Trielinic: k = (k:,, k2 Ka, Ka, Kis ki))
Monoclinic: k = (0, 12, 0, Ais, kis Ki)
Orthorhombic: k = (0, 0, O, Ka, kis, kis)
Tetragonal: k = (0, 0, 0, 0, k5. ka)
Hexagonal: k (- log(l))/44)))
ko)
Cubie: k 00,0,0,0,0,ks
16

---

Published as a conference paper at ICLR 2025
C SITE SYMMETRY REPRESENTATION
The 15 possible symmetry axes of crystals are: [001], [010], [100], (II), (1I1), [TII], (110). [110], [101]. [101]. [011]. [011], [210], [120], [110]. These axes are written in short form: for example, [110] denotes the direction of the vector (1, -). They are shown in Figure 2 The axes depend on the symmetries of the crystal system: for example, in an orthorhombic crystal (a rectangular prism whose side lengths are not necessarily equal), a crystal may have different site symmetries oriented around the x, y, Z-axes. Conversely, in a tetragonal crystal (a rectangular prism with a square base), any site symmetry oriented along the x-axis must also be along the y- axis, there may be additional symmetries along the diagonal of the x-y plane.
The possible set of symmetry elements along each axis for a site symmetry group correspond to the identity 1; an inversion I; rotations of different orders 2, 3, 4, and 6; rotoinversions 2 (equivalent to a mirror symmetry m across plane perpendicular to the axis). 3, 4, and and combinations ro- tations and mirror reflections 2/ m, 4/ 4/m, and 6/m. This enumeration yields 13 possible symmetries along each axis.
The possible symmetry elements along each axis for space group correspond the identity 1; an inversion I; rotations of different orders 2, 3, 4, and 6; rotoinversions 2 (equivalent to mirror symmetry across plane perpendicular the axis), 4, and 6; screws 21, 31, 32, 41, 42, 43, 61. 62. 63, 64, 65, and glides a, b, c, d,
To encode a space group, an additional 7-dimensional one-hot encoding used to denote the Bravais lattice to which the space group belongs. This yields a (26 x 15) + 7 = 397 dimensional binary representation of space group.
The representations can now be accessed using the symmetry module of PyXtal (Fredericks et al. 2021
D DIFFUSION AND DENOISING PROCESS DETAILS
Diffusion on lattice parameters k Inspired by Jiao et al. 2024). we perform diffusion over k, the -nnaannari lattice representation. The forward noising process given by g(k; kk) ~ II) where k, is the noised version of ko timestep t. Here, similar to Nichol & Dhariwal 2021 a; = 1 (1 3,), where 3, E (0, 1) determines variance each step con- trolled by the cosine scheduler. During the generation process, we start with ky ~ N I) and use learned denoising network generate k from kg
((kklyā,ko
Pa pa(ke-1/C) = / kr- (k-il (())
[t)(t) =
1 (ki Var
Br
(x(C,, (x(C(tt)),
1- āx-1 1-1011 =
VI a
āi
Here, C₁ is the noised crystal and (C), t) is the predicted denoising term predicted from denoising network OCC), t). We also use mask m only implement diffusion unconstrained dimensions of k,, since depending upon space groups, certain dimensions have fixed values (Appendix B). The mask can be represented m {0, 1} and m, 1 indicates that 7th index of k is unconstrained. The corresponding loss used to train the denoising network is:
L* Fr [e.--Wo..,,.. [l]m 6 -C, 1)13
where the elementwise product and (1,T)) a uniform distribution over timesteps.
Diffusion over representative fractional coordinates X' We perform diffusion over the frac- tional coordinates using the same method as Jiao et al. (2023). Due to the periodicity of fractional coordinates, the noising process qXx, (Xo) is determined by Wrapped Normal distribution rather than a Gaussian distribution, and we initialize the fractional coordinates X,, with the uniform dis- tribution (0,)) when sampling.
17

---

Published as a conference paper at ICLR 2025
Diffusion on atom types A' We discrete diffusion from Austin al. 2021 to sample the atom types of each representative. If au E {0, 1] the one-hot encoding of atom types for a single representative, then we can noise it as: (( ao) = Ca;(a;; = a Q:), where Q2 = - Q2 E R2x2 is the cumulative product of transition matrices between timesteps. Inspired by Vignac et al. be parametrized as Q: 0t + -3 where ma the marginals
2023). the transition matrix over the atom types in the data, and and B are scheduling parameters. The effect of this noising scheme is that regardless of ag, the fully noised ar agj Qr ma- so we can sample from the prior distribution ma, which is close to the data distribution. The discrete diffusion model is trained using a cross-entropy loss:
M
CA. Ea Q.),~)((ITT)
Σ
CrossEntropy(a/ a(),
where å; the probabilities predicted by the model (C,,, t). To sample from the discrete diffusion model, we sample from the marginal distribution over atom types ma. then progressively denoise using:
q( a ar, an) Cat aa-lip a, a
(5)
ag
Quac
More details of this implementation can be seen
Vignae al. (0033
Diffusion for site symmetries S The site symmetry representation matrices described in Section can be thought of as 15 separate 13-dimensional categorical variables: one site symmetry oper- ation per axis. Our diffusion model over site symmetries is almost identical to the method for atom types, applying discrete diffusion separately over each the axes. Because the site symmetries de- the space group, use transition matrices that are different for each space group: Q = a + 8,ms..G. G. where ms..G denotes the marginals over site symmetry operations for axis Su given space group G. For each representative node, average the cross-entropy loss over each the axes.
pend strongly
E ARCHITECTURE DETAILS
E.1 DENOISING MODEL
We use a graph neural network based on the architecture of Jiao et al. (2023) We embed the timestep t using sinusoidal embeddings, (t). We embed our space group representation from Section 4.2 using MLP, @G(G). We embed our site symmetries by separately embedding each axis using the
same network, and feeding the resulting embeddings into secondary MIP: ds (as(Oz) u=1 do (Sw))- These are all used to initialize the node embeddings h;-
15 ①
(Sw) (G) (1)).
h; f on (at, Xzs dss
As noted earlier, directly coordinates because lattice, and so Euclidean symmetries are not necessarily useful here.
working a conventional or canonical
At each layer compute messages and use them to update node embeddings:
mij f- om (h, hy, k, s)(x, - x,))
M
h + h, + on (h, > m:y)
Here, v is a Fourier embedding, om and Φn are MIPs acting on edges and nodes respectively. We use SiLU activation function for each MLP. Finally, we output predicted Å' and S using the node embeddings h; and using E. h
18

---

Published as a conference paper at ICLR 2025
Train
Validation
Test
Figure 5: Proportion of space group symmetries of the MPTS-52 dataset. From left to right, the first few space groups are: Pnma, P2,/c, Fm3m, 14/mmm, P63/mme, Pm3m, C2/m, C2/c, 1, and Cmcm.
Table 6: Percent of stable and S.U.N. samples pro- duced from an initial set of 10,000 generated crys- tals for SymmCD trained on the MPTS-52 dataset.
Table 5: Template statistics for MPTS-52
Method
# Unique
% in Train
# New
Initial
Relaxed
Relaxed S.U.N.
Training Set
4452
100 48.3
Stable
Stable
SymmCD
2772
1432
1.72%
5.97%
4.62%
Table 7: The validity, coverage, and property distribution metrics for SymmCD trained on the MPTS-52 dataset.
Validity (%) (t)
Coverage (%) (1)
Property Distribution (4)
Struct. Comp.
Recall Precision
dp
dg
delem
dag
90.1
79.2
99.6
96.3
0.844
0.549
0.317
0.274
E.2 MODEL HYPERPARAMETERS
The graph neural network has 8 layers, and we use a representation dimension of 1024 for h; We encode distances between nodes using sinusoidal embedding, with 128 different frequencies. We encode the timestep t into a 10 dimensional vector. We apply layer normalization at each layer of the GNN. The loss coefficients selected were Ao- 5, Ax 1, AA 0.1 and Äs 10.
We performed two hyperparameter sweeps: we first tested each combination of Ak. A and Às set values in {0.1, 0.5, 1, 10), while keeping Ax fixed I, and then selected the loss coefficients that lead to the highest structural validity. Next, we performed random sweep of other architecture parameters, running 150 different hyperparameter combinations and choosing model that had high performance on structural validity, compositional validity, and dg. We varied the number of GNN layers in (6. 12, 16), representation dimension (256, 512, 1024), time embedding dimension in {10, 64, 256), and varied whether layer normalization was used.
F EXTENDED RESULTS
F.1 MPTS 52 DATASET
In addition to MP-20, we also trained SymmCD on the MPTS-52 dataset (Baird et al. 2024), a more challenging subset of the Materials Project that contains materials with up 52 atoms per primitive unit cell. Unlike MP-20, it does not filter out materials containing radioactive elements. The dataset contains 40,476 samples with a train/validation/test split of 27,380/5, 5000/ 096 crystals. The splits are in chronological order, with the materials in the test set having been discovered most recently, and the materials in the training set having been discovered earliest. The distribution of space groups found in the dataset are shown in Figure 5 None of the diffusion and flow-based methods we compared against (CDVAE, DiffCSP, DifiCSP++, or FlowMM) have reported de-novo generation results this dataset.
We trained SymmCD on MPTS-52 using all of the same hyperparameters as were used for the MP-20 dataset, but trained for 1500 epochs. We sampled 10,000 crystals, and checked the same proxy metrics. We also relaxed the generated crystals using CHGNet and checked for whether the generated crystals were S.U.N. The unique templates (as described in Section 5.1] of the dataset and of our generated crystals are shown in Table 5 The results for the proxy metrics are shown in Table and stability results shown in Table 6
19

---

Published as a conference paper at ICLR 2025
MP20 SymmCD
0.125
Proportion
0.075
0.025
Number of atoms
Figure 6: Histogram of the number of atoms in crystals from MP-20 and generated by SymmCD when trained MP-20.
0.08
MPTS52 SymmCD
0.06
Proportion
0.04
0.00
125
Number of atoms
Figure 7: Histogram of the number of atoms in crystals from MPTS-52 and generated by SymmCD when trained MPTS-52.
F.2 NUMBER OF ATOMS
To demonstrate that SymmCD able to correctly predict reasonable site symmetries, we show here that the distribution of number of atoms per crystal matches the dataset it is trained on. This is not trivial task, as the model needs to learn the multiplicity different possible site symmetries, which depends both the different symmetry elements of the site symmetry and the space group that it belongs to. The comparison for MP-20 is shown in Figure 6 and the comparison for MPTS-52 shown in Figure 7
F.3 PROPERTY PREDICTION TASK
We test the usefulness of our site symmetry representa- tion using regression experiment. We selected formation energy per atom as the target property to predict.
Table 8: Mean average when pre- dicting crystal formation energy. The input could be the asymmetric unit or multi-graph, and the site symmetry in- formation be encoded ignored. We observe that our encoding of site symmetry helps predict the target prop- erty.
We use DimeNet++ Gastteggrr et al. 2020b a) as base model to perform ablation over the type of input graph and encoding site symmetry information per node.
One input format is multi-graph (Xic et al. 2022), which describes the unit cell graph with nodes as atoms and edges between them according to a cutoff radius. These edges could potentially span to neighbouring unit cells. The other input format is the asymmetric unit that use in SymmCD. Under these two inputs, we test the effects of including site symmetry encoding for each node. We report the Mean Absolute Error (MAE) for the test set Table 8 We see that the effect of including site symmetry
Multigraph
Asymm. Unit
W/out S With S
0.0214 0.0212
0.0711
0.0490
20

---

Published as a conference paper at ICLR 2025
information is minimal when we have access to the full graph. However, we see that when we are restricted to only using the asymmetric unit, having access to the site symmetry info greatly helps, showing that recover some geometric information lost when using just an asymmetric unit by also including symmetry.
F.4 EXAMPLES
we include 6 randomly sampled crystals generated by SymmCD along with their re- spective space groups.
In Figure 8
(a) PaTi;
Pm3m
(b) CeSiGezOs 14mm
(c) Cs2 CuO,
14mmm
(d) SrYOs
I4mem
(c) TaaNba C.CoMs>C
Pmmm
(f) TbinAus
Fm3
Figure& 8: Example materials generated by SymmCD, along with their chemical formulate and space groups symmetries.
F,5 PROXY METRICS
We compare the different methods using the metrics established by Xie et al. 2022), measuring the validity, coverage, and property statistics the generated crystals. We measure the validity by checking structural validity, defined whether
atoms closer than 0.5 Á apart, and com- positional validity, defined as whether the charges are balanced as determined by SMACT (Davies) al. 2019). should be noted that the compositional validity of the MP-20 dataset is only 92%. To determine coverage, we examine the CrystalNN structural fingerprints (Zimmermann & Jain 2020) and Magpie compositional fingerprints (Ward al. (016)) of the valid generated crystals, and look at their distances to the fingerprints of the crystals in the test set. If the distance is under some cutoff, then the crystals are matched, giving both recall and precision metrics. We look the distances between the properties of the valid generated crystals and the crystals from the test set to compare the ability of each model to match the data distribution. We specifically compare the Wasserstein distances between the atomic densities dp. number of unique elements delem and the formation energy dg predicted by a pretrained DimeNet++ model (Gasteiger et al. 2020a).. We also look at the Ensen-Shannon distance between the space groups of the generated valid structures and the space groups of the crystals in the test set, denoted by dag- The coverage and property statistics computed only for random subset of 1000 valid crystals per method. For each method,
21

---

Published as a conference paper at ICLR 2025
Method
0.04
CDVAE
DiffCSP
0.03
DiffCSP++
Density
FlowMM
0.02
SymmCD-10 groups
SymmCD-All groups
0.01
0.00
0
10
20
30
40
50
Cumulative atom displacement (A)
Figure 9: Distribution of total atom displacements over the course of relaxation. Displacements are averaged over structures for given method.
train 5 different models with different random seeds that could the variance for each of the metrics.
F.6 DENSITY FUNCTIONAL THEORY
We performed cell and geometry relaxation calculations using the PBE functional, DZVP-MOLOPT- SR--SR-GG- GTH basis set, and GTH-PBE pseudopotential with the QUICKSTEP program from CP2K (Perdew et al.) 1996 Kühne et al. 2020). Table shows the settings- particularly convergence thresholds used for performing the relaxations. Values were generally tuned to balance between a feasible computational budget and a fair benchmark between methods: example, allow relaxations attempt to continue even with poorly convergent SCF in the hopes that optimization trajectories will still have a chance of finding local minimum. With that in mind, note that the convergence thresholds also generally set relatively "lax" values, and for end property values stricter convergence thresholds may be necessary.
FigureDshow shows distributions of the cumulative method: in other words, the total distance the average atom travels over the full course of the relax- ation. Naturally, the less atom displacement, the more likely a given generative method produces high fidelity crystal structure samples. We see that DiffCSP++ and both treatments of SymmCD pro- duce samples with displacements that peak close zero, while other methods peak closer to ~4.5A. The relatively long tails out to high cumulative displacements seen with CDVAE and FlowMM are attributed to trajectories that do not converge after the specified number relaxation steps. From this, we can conclude that with the exception of SymmCD and DifICSP++, the tested generative methods demonstrate similar degree fidelity requiring some degree of optimization.
displacement, averaged over structures for each
Figure 10 shows empirical distribution functions of the maximum gradient value at the end of the relaxation trajectory, regardless the state of convergence: the smaller the maximum gradient is, the closer the relaxation ended in local minimum. The first observation is that across all methods, a significant portion (~600) of trajectories fail to converge (the portion to the right of the dashed the majority of sampled structures fail to converge, based using our reasonable choice of method, basis set, and pseudopotential. Another observation is the long tail towards low values for the 10 space group treatment of SymmCD, which provides com- pelling evidence for extremely high fidelity samples being produced by SymmCD. The expected values of the maximum gradient for each method are shown Table TO] where we can observe that SymmCD tends to have lower maximum gradient after relaxation.
naive attempt relax them
22

---

Published as a conference paper at ICLR 2025
Table 9: Configuration settings for CP2K. Settings that are omitted from this table assume their default values.
Parameter
Value
Base SCF settings
EPS.SCF
10
MAX_SCE
300
MAX_ ITER
400
IGNORE_CONVERGENCEE
T
Orbital transformation
Orbital transformation method
IRAC
ENERGY_GAL
10-3
MINIMIZER
DIIS
LINESEARCH
2PNT
PRECONDITIONER
FULL.ALL
Outer SCF settings
MAX_SCF
20
EPS SCF
10-6
Cell optimization
TYPE
DIRECT.CELL.OPT
MAX_ITER
100
OPTIMIZER
BFGS
Geometry optimization
MAX_DR
3 X 10
MAX_FORCE
X 10
RMS
1.5 10 3
RMS _FRCE
X 10
MAX_ITER
100
OPTIMIZER
BFGS
BEGS TRUST RADIUS
0.25
1.0
Method
0.8
CDVAE
DiffCSP
Proportion
DiffCSP++
0.6
FlowMM
SymmCD-10 groups
0.4
SymmCD-All groups
0.0
10-7
10-6
6
10-5
10
10-3 -3
10 2
10-1
10"
Final maximum gradient value
Figure 10: Empirical cumulative distribution plots for the maximum, absolute value of the gradient across atoms after relaxation, regardless of convergence. The dashed line indicates the convergence threshold stated in Table 9
23

---

Published as a conference paper at ICLR 2025
Table 10: Expected values the maximum gradient for each method from integrating the curves in
Figure 10
Method
E V max (J)
DifICSP
0.000494
CDVAE
0.000235
FlowMM
0.000083
D
0.000072
SymmCD (All groups)
0.000057
SymmCD (10 SGs)
0.000038
24


Reviews:
[{"review_id": "bNaxLKvalw", "reviewer": "Reviewer_7azh", "summary": "The submission, \"Symmetry-Preserving Crystal Generation with Diffusion Models,\" proposes a method for generating single-crystal structures with precise symmetric properties. The authors use asymmetric units and site symmetry representation, followed by a diffusion model for generation. This method explicitly addresses the generation of crystals with respect to their symmetry group.\nThe method performs on par with existing approaches but has a lower computational footprint.", "questions": "Please consider the questions raised in the Weaknesses section.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "•\tThe manuscript's structure and clarity are excellent overall.\n\n•\tThe manuscript includes a well-written and comprehensive introduction, with a clear and well-developed motivation for the crystal generation problem as an application of diffusion models.\n\n•\tThe method is well-formalized and understandable even to non-experts in crystal generation.\n\n•\tExperimental tasks and evaluation: The authors assess their method and the baselines on relevant additional tasks, such as S.U.N. structure prediction and other proxy metrics, which highlight the proposed method's strengths.", "weaknesses": "•\tIntroduction: From my perspective, the problem of generating symmetric crystals is closely related to other structure generation tasks in general representation learning. For instance, in biological applications, such as neuron structure generation or vascular structure generation, it would be beneficial if the authors discussed the relation to other domains in structure generation and the types of methods that have been developed. For example, I see certain similarities to diffusion methods in molecule generation [https://ieeexplore.ieee.org/abstract/document/10419041] or graph generation [https://arxiv.org/abs/2209.14734], which are partly mentioned in the methods since they are used; however, a discussion of how these applications relate to the context of representation learning would be valuable.\n\n•\tReproducibility: I did not find a link to an anonymous repository or source code in OpenReview, hindering the evaluation of reproducibility for this submission.\n\n\n•\tExperimentation: There are only minor performance gains (if any) compared to the state of the art. What are the practical uses of crystal symmetry generation in academia or industry? Is the computational gain truly relevant, considering the regular applications and scenarios in which crystal symmetry generation methods are used?\n\n•\tExperimentation: \"We withhold 20% of the dataset as a validation set, and 20% as a test set\" (Line 377). The experimental setup suggests that the authors do not use a form of cross-validation or cross-testing. Is there a specific reason for this choice? Given that the authors describe their computational efficiency as a strength, extensive cross-validation across experiments would seem reasonable.\n\n•\tExperimentation: Hyperparameter Selection (Section E.2). The authors briefly describe their final hyperparameters: “These hyperparameters were chosen using a sweep” (Line 919). Without code availability and the validation issues mentioned earlier, this appears to be a limited experimental description. What was the hyperparameter search space/budget? How were the hyperparameters for the four baselines tuned exactly? The results show very small differences in performance, so a fair description of hyperparameter search is crucial for reproducibility.\n\n\nI am not an expert in crystal generation and potentially some of my questions are atypical in the field, I am curious to hear the authors and other reviewer comments and willing to change my rating accordingly.\n\n\n\n\n\n## Post rebuttal and discussion period comment: \n\nI want to thank the authors for replying and interacting in the discussion period. While the authors reply to all comments, I am not fully convinced by their replies. \n\nI am still convinced that the experimentation is not rigorous and sufficiently validated. Unfortunately, the authors do not reply to the very concrete questions I asked; see details in an extra comment.\n\nWhile I still like the idea and methodological contribution, I am worried about reproducibility, and I am slightly lowering my score and kindly ask the ACs to discuss the reproducibility aspect when considering the paper for acceptance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "r5snVYKJJ3", "reviewer": "Reviewer_Tsh1", "summary": "The authors present the generative model SymmCD, which allows for the generation of datasets of crystalline structures of non-molecular crystals while explicitly considering symmetry. The results obtained exhibit both high symmetry diversity and a significant percentage of thermodynamically stable structures, making SymmCD a solid choice for crystal structure prediction systems or virtual screening of crystalline materials.", "questions": "1. In Table 2, why is CDVAE bold instead of SymmCD (10 SGs) for Validity Comp.? I understand an argument could be made that comparing the 10 SGs version to the other methods might not be entirely appropriate, but then I would consider dropping this version from tables 1 and 2, and only discuss it in the context of table 3, where the S.U.N. shines. Please clarify the logic behind inclusion of SymmCD (10 SGs) for Table 1 and Table 2.\n2. The evaluation presented in Table 3 involves random subsampling of 10% of the generated crystals, followed by two predictive models to evaluate stability and S.U.N. properties. At the same time, the SymmCD shows only a marginal improvement compared to DiffCSP and DiffCSP++. Are these results statistically significant? Please provide details on the robustness of this evaluation.\n3. Which model appears in the Table 4 as Conventional Unit Cell? Please provide a citation and clarify how this model was included in other comparisons as well.\n4. Is it possible to provide a link to the anonymized repository reproducing experiments?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "- The developed method for vectorizing crystalline structures, which explicitly accounts for both the spatial symmetry of the crystal and the point symmetry of the orbits, is to my knowledge the first of its kind, therefore unique, and holds a great promise for application in crystal structure prediction (CSP) for both inorganic and organic crystals.\n- The article is well-structured and clearly conveys information, allowing individuals unfamiliar with this field to understand the crystallographic features of the problem with some investment of time.\n- I believe this work could be highlighted at the conference as a fine example of how rational design of vector representation can influence the overall effectiveness of the developed deep learning model.", "weaknesses": "The major weaknesses of the paper lie in the discussion of the obtained empirical results. Addressing these will significantly enhance the presentation of the work accomplished:\n\n1. Please indicate in the introduction that the initial focus is on non-molecular/inorganic crystals.\n\n2. Please mention in the conclusion remarks that your method of structural representation seems to be well-suited for molecular crystals as well. For the latter, the presence of intrinsic point symmetry and its interaction with the point symmetry of orbitals is one of the key factors determining the crystal structure.\n\n3. In your conclusions, when you state \"go beyond single crystals, and consider generating multi-component crystals and alloys,\" please clarify what you mean. \"Single crystal\" is a broad term contrasting with polycrystalline materials and does not directly relate to crystalline structure. A multi-component crystal refers to a crystal composed of multiple chemical substances; for instance, this includes pharmaceutical co-crystals. Clearly, your approach should be applicable to these systems.\n\n4. Please consider rewriting conclusions to emphasize advantages (applicability to molecular crystals, including co-crystals) rather than deficiencies (inapplicability to non-crystalline systems), but also, to provide a deeper discussion of the limitations of SymmCD along with practical implications for the actual industrial problems.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "p6sSSPiyAT", "reviewer": "Reviewer_B8yV", "summary": "This paper presents a novel diffusion-based generative model, SymmCD, for symmetry-preserving crystal generation. The proposed approach explicitly incorporates crystallographic symmetry into the generative process, using a unique representation that decomposes crystals into asymmetric units and symmetry transformations. This design enhances both computational efficiency and the diversity of generated crystal structures, addressing some limitations of existing models in terms of symmetry and structural validity.Overall, the paper presents a strong contribution to the field of crystal generation. By explicitly incorporating symmetry into a generative diffusion framework, SymmCD addresses critical limitations of prior methods and provides a promising tool for materials discovery.", "questions": "Since I am not a researcher in this field, I don’t know much about the specific background, so I am not very clear about the process shown in Figure 3. Figure 3 and the training pipeline section could benefit from additional annotations to improve readability for those unfamiliar with diffusion models in this context.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Innovation in Representation: The paper introduces a physically motivated representation based on crystallographic symmetry, using a binary matrix to encode symmetry, which addresses data fragmentation and enables generalization across symmetry groups.\n2. Computational Efficiency: By focusing on asymmetric units rather than full crystal structures, the model demonstrates significant improvements in memory usage and training speed, an aspect well-supported by experimental evidence.\n3. Diversity and Validity of Generated Structures: SymmCD shows impressive results in generating diverse, valid, and symmetry-conforming crystal structures across multiple symmetry groups, even those that are less common in training data.", "weaknesses": "1.Comprehensive Evaluation of Generated Crystal Properties: While the model’s ability to generate symmetric and diverse crystals is demonstrated, additional quantitative evaluations of properties such as thermodynamic and mechanical stability would further solidify the model’s applicability to real-world scenarios. Metrics that reflect physical applicability, such as structural stability under various conditions, could significantly strengthen the evaluation section.\n2.Efficiency on Larger Datasets: SymmCD’s efficient crystal representation is highlighted as a key advantage. However, a more comprehensive analysis of its computational efficiency on larger datasets, or under different hardware setups, could provide a more complete understanding of its scalability and practical utility in materials science applications.\n3.Clarification of the Binary Symmetry Encoding: The binary matrix representation for symmetry is an intriguing solution to data fragmentation, yet further explanation on why this approach outperforms traditional encodings in practical settings would be beneficial. Additional details in the architecture and experimental sections could clarify how the representation is effectively utilized in training.\n4.It may be helpful to provide a clearer explanation of the training algorithm, particularly in how the diffusion and denoising processes maintain symmetry.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)