Space Group Constrained Crystal Generation
Authors: Rui Jiao
Wenbing Huang
Yu Liu
Deli Zhao
Yang Liu
Venue: ICLR 2024 poster (2024)
Keywords: material generation
spacegroup
diffusion generative models
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Crystals are the foundation of numerous scientific and industrial applications. While various learning-based approaches have been proposed for crystal generation, existing methods neglect the spacegroup constraint which is crucial in describing the geometry of crystals and closely relevant to many desirable properties. However, considering spacegroup constraint is challenging owing to its diverse and nontrivial forms. In this paper, we reduce the spacegroup constraint into an equivalent formulation that is more tractable to be handcrafted into the generation process. In particular, we translate the spacegroup constraint into two cases: the basis constraint of the invariant exponential space of the lattice matrix and the Wyckoff position constraint of the fractional coordinates. Upon the derived constraints, we then propose DiffCSP++, a novel diffusion model that has enhanced a previous work DiffCSP by further taking spacegroup constraint into account. Experiments on several popular datasets verify the benefit of the involvement of the spacegroup constraint, and show that our DiffCSP++ achieves the best or comparable performance on crystal structure prediction and ab initio crystal generation.

Published as a conference paper at ICLR 2024
SPACE GROUP CONSTRAINED CRYSTAL GENERATION
Rui Jiao 1.2 Wenbing Huang Yu Liu", Deli Zhao', Yang Liu 1.2f
'Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University Institute for AIR, Tsinghua University
'Gaoling School of Artificial Intelligence, Renmin University of China
*Beijing Key Laboratory of Big Data Management and Analysis Methods, Beijing, China
5 Allibaba Group
ABSTRACT
Crystals are the foundation of numerous scientific and industrial applications. While various learning-basee approaches have been proposed for crystal gener- ation, existing methods seldom consider the space group constraint which is cru- cial in describing the geometry of crystals and closely relevant to many desirable properties. However, considering space group constraint is challenging owing its diverse and nontrivial forms. In this paper, we reduce the space group con- straint into equivalent formulation that is more tractable to be handerafted into the generation process. In particular, we translate the space group constraint into two parts: the basis constraint of the invariant logarithmic space of the lattice matrix and the Wyckoff position constraint of the fractional coordinates. Upon the derived constraints, we then propose DiffCSP++, novel diffusion model that has enhanced a previous work DifCCSP (Jiao et al, (203)) by further taking space group constraint into account. Experiments on several popular datasets verify the benefit of the involvement of the space group constraint, and show that our Dif- FCSP++ achieves promising performance on crystal structure prediction, ab initio crystal generation and controllable generation with customized space groups.
INTRODUCTION
Crystal generation represents critical task in the realm of scientific computation and industrial applications. The ability to accurately and efficiently generate crystal structures opens up avenues for new material discovery and design, thereby having profound implications for various fields, including physics, chemistry, and material science (Liu et al. 2017 Oganov et al. 2019
Recent advancements in machine learning have paved the way for the application of generative mod- els to this task (Nouira et al. 20188] Hoffmann et al. 2019 Hu et al.) |2200 Ren et al. 2021]. Among various strategies, diffusion models have been exhibited
be particularly effective in generating
realistic and diverse crystal structures (Xie et al., 2021 Jiao et al., (023). These methods leverage stochastic process to gradually transform a random initial state into stable distribution, effectively capturing the complex landscapes of crystal structures.
Despite the success of existing methods, one significant aspect that has been largely overlooked is the consideration of space group symmetry (Hiller 1986). Space groups play a pivotal role in crystallography, defining the geometry of crystal structures and being intrinsically tied to many properties such as the topological phases (Tang al. 2019 Chen et al. 2022). However, integrating space group symmetry into diffusion models is a non-trivial task due to the diverse and complex forms of space groups
In this paper, we supplement this piece by introducing a novel approach that effectively takes space group constraints into account. Our method, termed DifiCSP++, enhances the previous DiffCSP method (Jiao et al. 2023) by translating the space group constraint into a more manageable form, which can be seamlessly integrated into the diffusion process. Our contributions can be summarized as follows:
"Tii work is done when Rui Jiao works as an intern in Alibaba Group.
Wenbing Huang and Yang Liu corresponding authors.

---

Published as a conference paper at ICLR 2024
We propose to equivalently interpret the space group constraint into tractable parts: the basis constraint of the -ina-invari logarithmic space the lattice matrix in $ and the Wyckoff position constraint of the fractional coordinates in $ 4.2 which largely facilitates the incorporation of the space group constraint into the crystal generation process.
Our method DifiCSP++ separately and simultaneously generates the lattices, fractional coordinates and the atom composition under the reduced form of the space group constraint, through novel denoising model that is E(3)-invariant.
Extensive experiments demonstrate that our method not only respects the crucial space group constraints but also achieves promising performance in crystal structure prediction and ab initio crystal generation.
2 RELATED WORKS
Learning-basee Crystal Generation. Data-driven approaches have emerged as promising direc- tion in the field of crystal generation. These techniques, rather than employing graph-based models, depicted crystals through alternative representations such as voxels voxels (Court al. 2020 Hoff- mann et al. 2019 Noh et al. 2019)) distance matrices (Yang et al. 202T Hu al. 2020 2021) or 3D coordinates (Nouira al. 2018) Kim et al., 2020 Ren et al. 2021). Recently, CDVAE (Xie al., 2021] combines the VAE backbone with a diffusion based decoder, and generates the atom types and coordinates DiffCSP (Jiao et al. 2023) jointly optimizes the lattice matrices and atom coordinates via diffu- sion framework. Based on the joint diffusion paradigm, MatterGen (Zeni al., 2023) applies polar decomposition to represent lattices as O(3)-invariant symmetry matrices, and GemsDiff (Klipfel (et al. 2024) projects the lattice matrices onto decomposed linear vector space. Although these approaches share similar lattice representations with our method, they often overlook the constraints imposed by space groups. Addressing this gap. PGCGM (Zhao al.) (202)) incorporates the affine matrices of the space group as additional input into Generative Adversarial Network (GAN) model. However, the application PGCGM is constrained by ternary systems, thus limiting its universality and rendering it inapplicable to all datasets applied in this paper. Besides, PCVAE (Liu et al.) 2023) integrates space group constraints to predict lattice parameters using a conditional VAE. In contrast, impose constraints on the lattice within the logarithmic space, ensuring compatibility with the diffusion-basee framework. Moreover, we further specify the Wyckoff position constraints of all atoms, achieving the final goal of structure prediction.
multi-graph (Xie & Grossman 2018) built upon predicted lattice parameters.
Diffusion Generative Models. Diffusion models have been recognized as a powerful generative framework across various domains. Initially gaining traction in the field of computer vision (Ho) et al. demonstrated in their application to the generation of small molecules (Xu et al.) [002]] Hoogeboom et al.
2020 Rombach et al. 2021; Ramesh et al.) 2022), the versatility of diffusion models has been
202T, protein structures (Luo et al., 2022) and crystalline materials (Xie et al. [222]] Jiao al.
2023 Notably, Chroma (Ingraham al.) 2022) incorporates symmetry conditions into the
generation process for protein structures. Different from symmetric proteins, space group constraints require reliable designs for the generation of lattices and special Wyckoff positions, which mainly discussed in this paper.
3 PRELIMINARIES
Crystal Structures A crystal structure M describes the periodic arrangement of atoms in 3D space. The repeating unit called unit cell, which can be characterized by a triplet, denoted as (A, X, L). where A = [a1, a2 .... aNl E RA represents the one-hot representations of atom type, X [x1 32 .... N E N comprises the atoms' Cartesian coordinates, and L [4, 12, L3] E 3xx is the lattice matrix containing three basic vectors to periodically translate the unit cell to the entire
3D space, which can be extended as M == {(a,, 20/ & x,), where x _ æ, denotes that x is equivalent to a; if at can be obtained via integral translation of m; along the lattices L i.e.,
x; 3,, E 3x s.t. x = x + Lk,
(1)
Apart from the prevalent Cartesian coordinate system, fractional coordinates are also widely ap- plied in crystallography. Given lattice matrix L (4, 4z), the fractional coordinate f
2

---

Published as a conference paper at ICLR 2024
Spacegrong
Desskain
log(-)
Model
exp(-
Moi
Wockell Posike Cominin>
Orytal Famnly Condronta
Figure I: Overview of our proposed Di for the denoising from Me to M We decom- pose the space group constraints the crystal family constraints on the lattice matrix (the red dashed line) and the Wyckoff position constraints on each atom (the blue dashed line).
fa) E [0, 1)3 locates the atom at a EE= f,ly More generally, given a Cartesian coor- dinate matrix X [211 ....... Nx), the corresponding fractional matrix is derived as F L-
Space Group The concept of space group used to describe the inherent symmetry of a crystal structure. Given transformation g E(3), define the transformation of the coordinate matrix X as X which is implemented as X == OX + t1 for a orthogonal matrix O E O(3), translation vector
R and 3-dimensional all-ones vector If 9 lets M invariant, that is g-M = {(a.. = = MM (note that the symbol "=" here refers to the equivalence between sets), M is recognized to symmetric with respect to g. The space group symmetry g g-M M can also be depicted by checking how the atoms are transformed. Specifically, for each transformation E G(M), there exists a permutation matrix Pg E (0, 1) NNN that maps each atom its corresponding symmetric point:
A APg X 24 XPs
(2)
The set of all possible symmetric transformations of M constitutes a space group G(M) {g E R(3)]g (a0))lg-=-- M]. Owing to the periodic nature of crystals, the size of G(M) is finite, and the total count of different space groups is finite as well. It has been conclusively demonstrated that there are 230 kinds of space groups for all crystals.
Task Definition We focus generating space group-constrained crystals by learning condi- tional distribution G), where G is the given space group with size |G] = m. Most previous works (Xie al., 2021 Jiao et al. 2023) derive p(M) without G and they usually apply B(3)- equivariant generative models to implement p(M) to eliminate the influence by the choice of the coordinate systems. In this paper, the O(3) equivariance is no longer required as both L and X will be embedded to invariant quantities, which will be introduced & 4. The translation invariance and periodicity will be maintained under the Fourier representation of the fractional coordinates, which will be shown in $ 4.4
4 THE PROPOSED METHOD: DIFFCSP++
It is nontrivial to exactly involve the constraint of Eq. into existing generative models, due to the various types of the space group constraints. In this section, we will reduce the space group constraint from two aspects: the invariant representation of constrained lattice matrices in $ [.1 and the Wyckoff positions of fractional coordinates in $ 4.2 which will be tractably and inherently maintained during our proposed diffusion process in
4.1 INVARIANT REPRESENTATION OF LATTICE MATRICES
The lattice matrix L 3x3 determines the shape of the unit cell. If the determinant (namely the volume) of L is meaningful: det(L) > 0, then the lattice matrix is invertible and we have the following decomposition.
Proposition 1 (Polar Decomposition (Hall 2013)). An invertible matrix L 3x3 can be uniquely decomposed into L = Q exp(S)) where Q 3ax3 is an orthogonal matrix, S E 3axz is symmetric matrix and exp(S) = En n! defines the exponential mapping of S.
The above proposition indicates that L can be uniquely represented by symmetric matrix S. More- over, any 0(3) transformation of L leaves S unchanged, as the transformation will be reflected by
3

---

Published as a conference paper at ICLR 2024
Q. We are able to find 6 bases of the space of symmetric matrices, e.g.,
0
0
0
0
0
B
=
1
0
B2 =
0
BB
=
0
1
0
0
0/
1
0
0
0
0
1
0
0
B. =
0
-1
0
.Bs =
0
1
0
.B =
0
0
0
0
0
0
-2
0
Each symmetric matrix be expanded via the above symmetric bases as stated below.
Proposition 2. AS T S ST 3k (ki,... s..), k,
By joining the conclusions of Propositions D and 2 it is clear to find that L is determined by the values of kę Therefore, reflect the space group constraint acting on L. Actually, the total 230 space groups are classified into 6 crystal families, determining the shape of L. After careful derivations (provided in Appendix the correspondence between the crystal families and the values of k; is given in the following table. With such table, when want to generate the lattice restricted by a given space group, we first retrieve the crystal family and then enforce the corresponding constraint on k; during generation.
able to choose different combinations of the symmetric bases to
Table I: Relationship between the lattice shape and the constraint of the symmetric bases, where a. b, and az, B, y denote the lengths and angles of the lattice bases, respectively.
Crystal Family
Space Group No.
Lattice Shape
Constraint Symmetric Bases
Trielinic
1~2
No Constraint
No Constraint
Monoclinic
~ 15
a ?
90
kg 0
Orthorhombic
16 ~ 74
a= = 3
90"
kj
ln
= h3 0
a B 7 90"
k1
l/2 k3z
Tetragonal
75 142
0 ==
hig =
90", y 120
12 Kg 0, -1og(3)/4
Hcxagonal
143 ~ 194
a=b b
K4s 0
a 3 7= 90
ki h2 = hig 0
Cubic
195 230
a = b =
ks = ks 0
4.2 WYCKOFF POSITIONS OF FRACTIONAL COORDINATES
As shown in Eq. 2 each transformation 9 G is associated with permutation matrix P Considering atom i for example, it will be transformed to a symmetric and equivalent point j. if pgli.j] 1, * j. where [. returns the element of the i-th row and j-th column. Under some particular transformation 9. the atom s will be transformed to itself, implying that Pg[i, = Such transformations that leave i invariant comprise the site symmetry group, defined as G. = [g E Glg x; - in crystallography. For atom i, it shares the same Wyckoff position with atoms owning the conjugate site symmetry groups of G. There could be multiple types of Wyckoff positions in a unit cell.
) C G. Now, we introduce the notion of Wyckoff position that useful
Wyckoff positions can also be represented by the fractional coordinate system. Given a crystal structure with N atoms belonging to N' Wyckoff positions, we denote that each Wyckoff position contains n, atoms satisfying 1 I < N'. The symbol n. is named as the multiplicity of the s-th Wyckoff position and maintains Σ m, = N. In the following sections, we denote a',, f' as the atom type and basic coordinate of the s-th Wyckoff position, and a., fe, as the atom type and fractional coordinate of atom s; in the s-th Wyckoff position. A type of Wyckoff positions is formulated as a list of transformation pairs {(R,.. ) that project the basic coordinate f to all equivalent positions (R., ft. + ts, 1 Figure (auure2hinn iillustrates an example with N' 7
Since Wyckoff positions are inherently determined by the space group, can realize the space group constraint by restricting the coordinates of the atoms that are located in the same set of the Wyckoff positions during crystal generation, which will be presented in detail in the next subsection.
4

---

Published as a conference paper at ICLR 2024
Wyckoff Positions of P4mm (No.11)
la: (0,0) lb: (1/2,1/2)
(1/2,0), (0,1/2)
Special position
4d: (x.0), (xx.0,, (0,x), (0,xx
(x,1/2), (xx,122,, (1/2.0.
4f (x.0), (X,0), (x00), (xx-x)
General position
8g: (x39), (x-9) (x))))
(d 9)
(520), 09-0), (-00), (-9--)
Figure 2: Inspired by PyXtal (Fredericks et al., 202T), we utilize the 2D plain group P4mm as a toy example to demonstrate the concept of Wyckoff position. An asymmetric triangle is copied eight times to construct the square unit cell, and the general Wyckoff position (8g') has a multiplicity of eight. The other Wyckoff positions are restricted to certain subspaces. For instance, position 4e is constrained by the red dashed lines.
4.3 DIFFUSION UNDER SPACE GROUP CONSTRAINTS
To tackle the crystal generation problem, we utilize diffusion models to jointly generate the lattice matrix L, the fractional coordinates F and the atom types under the specific space group con- straint. We detail the forward diffusion process and the backward generation process of the three key components follows.
Diffusion on L. As mentioned in 4.1 the lattice matrix L can be uniquely represented by an 0(3)- invariant coefficient vector k. Hence, we directly design the diffusion process on kt, and the forward probability of time step / is given by
q(ke (ko) N -NKkkllaaro (1 (1- a.)))
(1-
(3)
where or II' (1) and E (0. determines the variance of each diffusion step, controlled by the cosine scheduler proposed in Nichol & Dhariwal (202))
Starting from the normal prior kT N0, I), the corresponding generation process designed as
p(k: p MMM) (N 1 ā:-1 I):
where ya (Mr) = Vas ke Ci--Vi-ae Ex ing model ()
8. 1)) and the term éx 1) is predicted by the denois-
To confine the structure under a desired space group constraint, we only diffuse and generate the unconstrained dimensions of k while preserving the constrained value kg as outlined Table I To optimize the denoising term éx,M,, t), we first sample E -V00, I) and reparameterize k; as k m (vā,ko + vI- â,E+) (1 (1-m) m)) @ kg, where the mask is given by m E {0, 1)6 m, indicates the i-th basis is unconstrained, and O is the element-wise multiplication. The objective on is finally computed by
Lb = N O - (M, t)][2)
(5)
Diffusion on F. In unit cell, the fractional coordinates F R3xN of N atoms can be arranged the Wyckoff positions of N' basic fractional coordinates F' R3*N'' Hence we only focus on the generation of F", and its forward process is conducted via the Wrapped Normal distribution following Jiao et al. (203) maintain periodic translation invariance:
q( = (F. (F(
(6)
'In practical implementation. Wyckoff positions are identified by combination of a number and a letter, where the number is the multiplicity, and the letter is to distinguish the Wyckoff position type in a dictionary order corresponding the ascending order of the multiplicity.
5

---

Published as a conference paper at ICLR 2024
The forward sampling can be implemented F w(Fi + oFep [M,,t) t)). where w(-) retains the fractional part of the imput. For the backward process, we first acquire Fy from the uniform initialization, and sample Fú via the predictor-correcte sampler with the denoising term (M,, output by the model ((M
Note that the basic coordinates do not always have degrees of freedom. The transformation ma- trice R., could be singular, resulting in Wyckoff positions located in specific planes, axes, or even reduced to fixed points. Hence we project the noise term EF onto the constrained subspaces via the least square method as e'p li, s) R ep s), where R is the pseudo-inverse of R.g. The training objective on F" is
F (F)))
A Fy log o/ (F) - é (M,, 1112
(7)
where A, = E-1 (IV log on (0, 1 is the pre-computed weight, and q' is the projected distribu- tion of induced by cpp.. Further details are provided in Appendix.
Diffusion on A. Since the atom types remain consistent with the Wyckoff positions, we can also only focus on the basic atoms A' C A. Considering A' E Rxxx as the one-hot continues representation, we apply the standard DDPM based method by specifying the forward process as
q( A)) = N AAIlvoAA (1- (1-
(8)
And the backward process defined as
1 (M
p( (( = =N(Ai-
(9)
1-
where "A' (Me) is similar to (M,) in Eq. dicted by the model o(Me. t).
4
The denoising term (M,,) E RAXN is pre-
The training objective is
CA = Wia.),.a - -EA, (M,, 1)]
(10)
The entire objective for training the joint diffusion model of M is combined as
LM X + LF + AA
(11)
4.4 DENOISING MODEL
In this subsection, we introduce the specific design of the denoising model d(Me: to obtain the three denoising terms Ék. éF ÉA' under the space group constraint, with the detailed architecture illustrated Figure 4 Appendix B.I We omit the subscript in this subsection for brevity.
We first fuse the atom embeddings farom (A) and the sinusoidal time embedding fime (t) fune))))))) (t)), where Pin is an MLP
acquire the input node features H The message passing from node to i in the 1-th layer is designed as Eq. 12 13
*Pi([Jumm
where Pm and &h are MLPs, and t'rr : m = pm (h
(1-1) .nj ((-1) BTT f f.)). (12)
(-1,1)3 [-1, 131K is the Fourier trans- formation with K bases to periodically em- bed the relative fractional coordinate f. f- f. Note that here we apply & in Eq. (12) as the unique -33)-invariant representation of L instead of the inner product L in Dif- fCSP (Jiao et al. 2023). and its reliability is validated in $ 5.4 After L layers of mes- sage passing, we get the invariant graph- and node-level denoising terms as Eq. (14 15). where ipk: SP: AA are MLPs. To align with the constrained diffusion framework proposed in $ 4 the denosing terms are re- quired to maintain the space group constraints, which is not considered in the original DifICSP. The constrained denoising terms are finally projected as Eq. 18). 18). where Ep e
N
h ( = h h + (h
1111)
11-1) Σ m
) (13)
1=1
N
Σ
(1)
Ek unncumstrainedd = 94
h
(14)
N
1rl.,),eAl:1 (p ).PA
(15)
ék m ① Ek umcoossstraie
(16)
(17)
Č WyckoffMean(EAl
(18)
the projected denoising term towards the subspace of the Wyckoff positions, and WyckoffMean computes the average of atoms belonging to the same Wyckoff position.

---

Published as a conference paper at ICLR 2024
Table 2: Results crystal structure prediction task. MR stands for Match Rate.
Perov-5
MP-20
MPTS-52
MR (%)
RMSE
MR (%)
RMSE
MR (%)
RMSE
RS
36.56
0.0886
11.49
0.2822
2.68
0.3444
BO
55.09
0.2037
12.68
0.2816
6.69
0.3444
PSO
21.88
0.0844
4.35
0.1670
1.09
0.2390
P-cG-SchNet (Gebauer et al. 2022
48.22
0.4179
15.39
0,3762
3.67
0.4115
CDVAE Xie et al. 2021
45,31
0.1138
33,90
0.1045
5.34
0,2106
DiffCSP Uiao et al. 2023
52.02
0.0760
51.49
0.0631
12.19
0.1786
DiffCSP++ (w/ CSPML)
52.17
0.0841
70.58
0.0272
37.17
0.0676
DifICSP++ (w/ GT)
GT)
98.44
0.0430
80.27
0.0295
46.29
0.0896
EXPERIMENTS
5.1
SETUP
In this section, evaluate method over various tasks. We demonstrate the capability and explore the potential upper limits on the crystal structure prediction task in $ 5.22 Additionally, we present the remarkable performance achieved in the ab initio generation task in 5.3 We further provide adequate analysis in 55.4
Datasets. We evaluate method four datasets with different data distributions. Perov- (Castelli et al.) 2012) encompasses 18,928 perovskite crystals with similar structures but distinct compositions. Each structure has precisely 5 atoms unit cell. Carbon-24 (Pickard (2220) com- prises 10,153 carbon crystals. All the crystals share only one element, Carbon, while exhibiting diverse structures containing 45,231 materials sourced from Material Projects with diverse compositions and structures. These materials represent the majority of experimentally generated crystals, each consisting of no more than 20 atoms in unit cell. MPTS-52 serves as more challenging extension of MP-20, consisting of 40,476 structures with unit cells containing up to atoms. For Perov-5, Carbon-24 and MP-20, follow the 60-20-20 split with previous works (Xie et al., 2021). For MPTS-52, 52, perform chronological split, allocating 77,380/5,000//
24 atoms within unit cell. MP-20 (Jain et al., 2013) contains
crystals for training/aalidattoon/estnnn
Tasks. We focus two major tasks attainable through method. Crystal Structure Prediction (CSP) aims at predicting the structure of a crystal based on its composition. Ab Initio Generation requires generating erystals with valid compositions and stable structures. We conduct the CSP experiments on Perov-5, MP-20, and MPTS-52, as the structures of carbon crystals vary diversely, and it is not reasonable match the generated samples with specific reference Carbon- 24. The comparison for the generation task is carried out using Perov-5, Carbon-24, and MP-20, aligning with previous works.
5.2 CRYSTAL STRUCTURE PREDICTION
To adapt our method to the CSP task, we keep the atom types unchanged during the training and generation stages. Moreover, the proposed DiffCSP++ requires the provision of the space group and the Wyckoff positions of all atoms during the generation process. To address this requirement, we employ two distinct approaches to obtain these essential conditions.
For the first version of our method, we select the space group of the Ground-Trus (GT) data as input. However, it's worth noting that these conditions are typically unavailable in real-world sce- narios. Instead, we also implement our method with CSPMI. (Kusaba 2022) 2022). ) metric learning
technique designed to select templates for the prediction of new structures. Given composition, first identify the composition in the training set that exhibits the highest similarity, Subsequently. we employ the corresponding structure template and refine it using DiffCSP++ after proper element substitution. We provide more details in Appendix B.2
For evaluation, match the predicted sample with the ground truth structure. For each composition within the testing set, we generate one structure and the match determined by the Structure class in pymatgen (Ong et al. 2013) with thresholds stol=0.5, angle tol=10, Itol=0.3, in accordance

---

Published as a conference paper at ICLR 2024
Table 3: Results ab initio generation task. The results of baseline methods are from Xie et al. (021); Jiao et al. al.) (2023).
Validity (%) f
Coverage (%) 1
Property dg
Data
Method
Struc.
Comp.
COV-R
COV-P
dp
detem
Perov-5
FTCP (Ren et al.
2021
0.24
54.24
0.00
0.00
10.27
156.0
0.6297
Cond-DIFC: Court
2020
73.60
82.95
73.92
10.13
2.268
4.111
0.8373
G-SchNel (Gebauer
99.92
98.79
0.18
0.23
1.625
4.746
0.0368
P-G-SchNel (Gebauer
2019
79.63
99.13
0.37
0.25
0,2755
1,388
0.4552
CDVAE Xie cl al. 2021
100.0
98. 59
99.45
98.46
0.1258
0.0264
0.0628
DifCCSP Jino et al.
100.0
98. 85
99.74
98.
0.1110
0.0263
0.0128
DitCCSP+
100.0
98.77
99.60
98.80
0.0661
0.0405
0.0040
C2 FTCP (Ren et al. 2021
0.08
0.00
0.00
5.206
19.05
G-SchNct (Gbanee al. 2019
99.94
0.00
0.00
0,9427
1.320
P-G-SchNeT (Cchater
2019
48.39
0.00
0.00
1,533
134.7
CDVAE Xie cl al 2121
100.0
99.80
83.08
0.1407
0.2850
DifCCSP Jaa el al
100.0
99.90
97.27
0.0805
0.0820
DitCCSP+
99.99
100.0
88.28
0.0307
0.0935
MP-20
FTCP Ren et al. 2021
1.55
48.37
4.72
0.09
23.71
160.9
0.7363
G-SchNer Gehauer
2019
99.65
75.96
38.33
99.57
3.034
42.09
0.6411
P-G-SchNeT (1 rchaaue
2019
77.51
76.40
41.93
99.74
4.04
2.448
0.6234
CDVAE Xie el al 2021
100.0
86,70
99.15
99.49
0.6875
0.2778
1.432
DifCCSP Jiao et al. 2023
100.0
83.25
99.71
99.76
0,3502
0.1247
0.3398
DifCCSP++
99.94
85.12
99.73
99.59
0.2351
0.0574
0.3749
with previous setups. The match rate represents the ratio of matched structures relative to the total number within the testing set, and the RMSD is averaged over the matched pairs, and normalized by vV/N where V the volume of the latrice.
We compare our methods with two lines of baselines. The first line is the optimization- bbssd meth- ods (Cheng et al., 2022) including Random Search (RS), Bayesian Optimization (BO), and Particle Swarm Optimization (PSO). The second line considers three types of generative methods. P-eG- SchNet (Gebauer et al. 2022) is an autoregressive model taking the composition as the condition. CDVAE (Xie et al. 2021) proposes a VAE framework that first predicts the invariant lattice param- eters and then generates the atom types and coordinates via a score-based decoder. DiffCSP (Jiao) et al. 2023) jointly generates the lattices and atom coordinates. All the generative methods do not consider the space group constraints.
The results are shown in Table where we provide the performance of the templates mined by CSPML and directly from GT. We have the following observations. 1. DiffCSP++, when equipped with GT conditions, demonstrates a remarkable superiority over other methods. This indicates that incorporating space group symmetries into the generation framework significantly enhances its abil- ity predict more precise structures. 2. When combined with CSPML templates, our method continues to surpass baseline methods. Given that the ground truth (GT) space groups are not ac- cessible in real-world CSP scenes, our method offers a practical solution for predicting structures with high space group symmetry. 3. Notably, there remains a gap between match rates under space group conditions derived from mined templates and those from GT conditions (70.58% vs. 80.27% on MP-20). This suggests that an improved complateffindigg algorithm could potentially enhance performance, which we leave for further studies.
5.3 AB INITIO GENERATION
For each dataset, we first sample 10,000 structures from the training set with replacement as tem- plates, and conduct the ab initio generation on the extracted templates. We focus on three lines of metrics for evaluation. Validity. We requires both the structures and the compositions of the generated samples are valid. The structural valid rate is the ratio of the samples with the minimal pairwise distance larger than 0.5A while the compositional valid rate is the percentage of samples under valence equilibrium solved by SMACT (Davies et al. 2019). Coverage. The coverage recall (COV-R) and precision (COV-P) calculate the percentage of the crystals in the testing set and that in generated samples matched with each other within a fingerprint distance threshold. Property statistics. We calculate three Wasserstein distances between the generated and testing structures, specifically focusing on density, formation energy, and the number of elements (Xic et al. [02T), denoted as d,. dr, and delem respectively. To execute this evaluation, apply these validity and
8

---

Published as a conference paper at ICLR 2024
coverage metrics to all 10,000 generated samples, and the property metrics computed on subset of 1,000 valid samples.
We compare method with previous generative methods FTCP Ren et al. 2021), CFnd-DFC-
VAE (Court et al. 2020), G-SchNet (Gebauer et al. 2019). P-G-SchNet, CDVAE (Xie et al. 2021) and DiffCSP TTio al. 2023). Table 3 depicts the results. We find that method yields compa- rable performance on validity and coverage metrics, while showcasing a substantial superiority over the baselines when it comes to property statisties, indicating that the inclusion of space group con- straints contributes to the model's ability to generate more realistic crystals, especially for complex structures like MP-20.
5.4 ANALYSIS
In this subsection, we discuss the influence of the key components in our proposed framework.
Invariant Lattice Representation. In this work, we sub- stitute the coefficient vector k for the inner product term
Table 4: Ablation studies.
L L in DiftCSP (Jiao et 2023) serve as the O(3)- invariant representation of the lattice matrix. To assess the impact of this modification, we adapt the diffusion process and representation from L to & in DifCCSP, with- out imposing extra space group constraints. This variant is denoted as DiffCSP-k: k. The performance of DiffCSP-k is substantiated by the results in Table 4 as being on par with the original DiffCSP, validating l as dependable invariant representation of the lattice matrix.
MP-20
MR (%)
RMSE
Invariant Lattice Representation
DiffCSP
51.49
0.0631
DifiCSP-k
50.76
0.0608
Pre-Average vs. Post-Areruge
DifICSP++ (Pre)
78.75
0.0355
DifICSP+ (Post) 80.27
0.0295
Pre-Average Post-Average. In Eq. (7). we average the denoising outputs F the base nodes for each Wyckoff position, and calculate the losses on the base node in Eq. (7). Practically, the loss function can be implemented in two forms, named pre-average and post-average, extended in Eq. 19 20 respectively.
CF te >o V log4 (F) Mcan Mcan(E'r)13 LF = Mean Fr log q' (F) (/)11)
(19)
(20)
Intuitively, the pre-average loss enforces the average output of each Wyckoff position match with the label the base node, while the post-average loss minimizes the T.2-distances of each atom. Table reveals the superior performance of the post-average model, which we adopt for all subsequent experiments.
Hinenoiie 125
Nirene
Towards structures with customized symmetries Our method enables structure generation under given space group constraints, hence allowing the creation of diverse structures from the same composition but based on different space groups. To illustrate the versatility of our approach, we visualize some resulting structures Figure 3 which demonstrates the distinct structures generated under various space group constraints.
Figure 3: Generation under different space groups.
6 CONCLUSION
In this work, we propose DifiCSP++, a diffusion- approach for crystal generation that effec- tively incorporates space group constraints. We decompose the complex space group constraints into invariant lattice representations of different crystal families and the symmetric atom types and coordinates according to Wyckoff positions, ensuring compatibility with the backbone model and the diffusion process. Adequate experiments verify the reliability of DiffCSP++ on crystal structure prediction and ab initio generation tasks. Notably, our method facilitates the generation of struc- tures from specific space groups, opening up new opportunities for material design, particularly in applications where certain space groups or templates are known exhibit desirable properties.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
This work is supported by the National Science and Technology Major Project under Grant 2020AAA0107300, the National Natural Science Foundation of China (No. 61925601, 62376276),
Beijing Nova Program (20230484278), and Alibaba Damo Research Fund.
REFERENCES
Ivano E Castelli, David D Landis, Kristian S Thygesen, Soren Dahl, Ib Chorkendorff, Thomas Jaramillo, and Karsten Jacobsen. New cubic perovskites for one-and two-photon water splitring using the computational materials repository. Energy & Environmental Science, 5(10))9034- 9043, 2012.
Lei Chen, Chandan Setty, Haoyu Hu, Maia G Vergniory, Sarah E Grefe, Lukas Fischer, Xinlin Yan, Gaku Eguchi, Andrey Prokofiev, Silke Paschen, et al. Topological semimetal driven by strong correlations and crystalline symmetry. Nature Physics, 18(1 18(11):1341-1345 2022.
Guanjian Cheng, Xin-Gao Gong, and Wan-Jian Yin. Crystal structure prediction by combining graph network and optimization algorithm. Nature communications, 13(1):1-8, 2022.
Callum J Court, Batuhan Yildirim, Apoorv Jain, and Jacqueline M Cole. 3-d inorganic crystal structure generation and property prediction via representation learning. Journal of chemical information and modeling, 60(10):4518 4438,,45555550 2020.
Daniel Davies, Keith Butler, Adam Jackson, Jonathan Skelton, Kazuki Morita, and Aron Walsh. Smact: Semiconducting materials by analogy and chemical theory. Journal of Open Source Software, 4(38);1361, 2019.
Scott Fredericks, Kevin Parrish, Dean Sayre, and Qiang Zhu. Pyxtal: A python library for crystal structure generation and symmetry analysis. Computer Physics Communications, 261:107810, 2021. ISSN 0010-4655. doi: http://////a//tta//////////00000
10780...0. URL httt:://www
sci ecceenedio rect com sc wwweeeeeeetecee
Niklas Gebauer, Michael Gastegger, and Kristof Schütt. Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, Fox, and R. Garnett (eds.), (eds.), Advances in Neural Information Processing Systems 32, pp. 7578. Curran Associates, Inc., 2019.
Niklas WA Gebauer, Michael Gastegger, Stefaan Hessmann, Klaus-Robert Müller, and Kristof T Schütt. Inverse design of 3d molecular structures with conditional generative neural networks. Nature communications, 13(1):1-11, 2022.
Brian C Hall. Lie groups, Lie algebras, and representations. Springer, 2013.
Howard Hiller. Crystallography and cohomology of groups. The American Mathematical Monthly, 93(10):765-779 1986.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
Jordan Hoffmann, Louis Maestrati, Yoshihide Sawada, Jian Tang, Jean Michel Sellier, and Yoshua Bengio. Data-driven approach to encoding and decoding 3-d crystal structures. arXiv preprint 170,009999 2019.
Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling, Argmax flows and multinomial diffusion: Learning categorical distributions. Advances in Neural Information Processing Systems, 34:12454-12466 2021.
Jianjun Hu, Wenhui Yang, and Edirisuriya M Dilanga Siriwardane. Distance matrix-based crystal structure prediction using evolutionary algorithms. The Journal of Physical Chemistry A, 124 1199919909- 2020.
10

---

Published as a conference paper at ICLR 2024
Jianjun Hu, Wenhui Yang, Rongzhi Dong, Yuxin Li, Xiang Li, Shaobo Li, and Edirisuriya MD Siriwardane. Contact map based crystal structure prediction using global optimization. Crys- 1EngComm, 23(8):1765-1770 2021.
John Ingraham, Max Baranov, Zak Costello, Vincent Frappier, Ahmed Ismail, Shan Tie, Wujie Wang, Vincent Xue, Fritz Obermeyer, Andrew Beam, et al. Illuminating protein space with a programmable generative model. BioRxiv, pp. 2022-12, 2022.
Anubhav Jain, Shyue Ping Ong, Geoffroy Hautier, Wei Chen, William Davidson Richards, Stephen Dacek, Shreyas Cholia, Dan Gunter, David Skinner, Gerbrand Ceder, et al. Commentary: The ma- terials project: A materials genome approach to accelerating materials innovation. APL materials, ([I):010022, 2013.
Rui Jiao, Wenbing Huang, Peijia Lin, Jiaqi Han, Pin Chen, Yutong Lu, and Yang Liu. Crystal struc- ture prediction by joint equivariant diffusion on lattices and fractional coordinates. In Workshop on "Machine Learning for Materials" ICLR 2023, 2023.
Forum?id-VPByphhaee
Sungwon Kim, Juhwan Noh, Geun Ho Gu, Alan Aspuru-Guzik, and Yousung Jung. Generative adversarial networks for crystal structure prediction. ACS central science, 6(8): 1420,, 2020.
Astrid Klipfel, Yačl Fregier, Adlane Sayede, and Zied Bouraoui. Vector field oriented diffusion model for crystal material generation. In Proceedings of the AAAI Conference on Artificial Intel- ligence, volume 38, pp. 22193 22201, 2024.
Minoru Kusaba, Chang Liu, and Ryo Yoshida. Crystal structure prediction with machine learning- based element substitution. Computational Materials Science, 211:111496, 2022.
Ron Larson. Elementary linear algebra. Cengage Learning, 2016.
Chang Liu, Erina Fujita, Yukari Katsura, Yuki Inada, Asuka Ishikawa, Ryuji Tamura, Kaoru Kimura, and Ryo Yoshida. Machine learning to predict quasicrystals from chemical compositions. Ad- vanced Materials, 33(36):2102577, 2021.
Ke Liu, Shangde Gao, Kaifan Yang, and Yuqiang Han. Pevae: A physies-informed neural network for determining the symmetry and geometry of crystals. In 2023 International Joint Conference on Neural Networks (IJCNN) pp. 1-8. IEEE, 2023.
Yue Liu, Tianlu Zhao, Wangwei Ju, and Siqi Shi. Materials discovery and design using machine learning. Journal of Materiomics, 3(3):159-177, 2017.
Shitong Luo, Yufeng Su, Xingang Peng, Sheng Wang, Jian Peng, and Jianzhu Ma. Antigen- specifccc antibody design and optimization with diffusion- generative models for protein structures. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pp. 8162-8171. PMLR, 2021.
Juhwan Noh, Jachoon Kim, Helge S Stein, Benjamin Sanchez-Lengeling John M Gregoire, Alan Aspuru-Guzik, and Yousung Jung. Inverse design of solid-state materials via a continuous repre- sentation. Matter, 1(5):1370-1344 2019.
Asma Nouira, Nataliya Sokolovska, and Jean-Claude Crivello. Crystalgan: learning to
discover crystallographic structures with generative adversarial networks. arXiv preprint 0rXiv:1810.11203. 2018.
Artem R Oganov, Chris Pickard, Qiang Zhu, and Richard J Needs. Structure prediction drives matcrials discovery. Nature Reviews Materials, 4(5):331-348 2019.
Shyue Ping Ong, William Davidson Richards, Anubhav Jain, Geoffroy Hautier, Michael Kocher, Shreyas Cholia, Dan Gunter, Vincent L Chevrier, Kristin A Persson, and Gerbrand Ceder. Python materials genomics (pymatgen): A robust, open-source python library for materials analysis. Computational Materials Science, 68:314-318 2013.

---

Published as a conference paper at ICLR 2024
Chris J. Pickard. Airss data for carbon at 10gpa and the c+n+h+o system at Igpa, 2020.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022.
Zekun Ren, Siyu Isaac Parker Tian, Juhwan Noh, Felipe Oviedo, Guangzong Xing, Jiali Li, Qiao- hao Liang. Ruiming Zhu, Armin G. Aberle, Shijing Sun, Xiaonan Wang. Yi Liu, Qianxiao Li, Senthilnath Jayavelu, Kedar Hippalgaonkar, Yousung Jung, and Tonio Buonassisi. An invert- ible crystallographic representation for general inverse design of inorganic erystals with targeted properties. Matter, 2021. ISSN 2590-2385. doi: https://doi.org//0. 101.0.011166.11 /016//.matt..22. 20....122...1...
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models, 2021.
Feng Tang, Hoi Chun Po, Ashvin Vishwanath, and Xiangang Wan. Comprehensive search for topo- logical materials using symmetry indicators. Nature, 566(7745):486-488 489, 2019.
Tian Xie and Jeffrey C. Grossman. Crystal graph convolutional neural networks an accurate and interpretable prediction of material properties. Phys. Rev. Lett., 120:145301, Apr 2018. doi: 10. 10.1100//hssee..eet 120. 145301.
Tian Xie, Xiang Fu, Octavian Dotavian-Eugen Ganea, Regina Barzilay, and Tommi S Jaakkola. Crystal diffusion variational autoencoder for periodic material generation. In International Conference on Learning Representations, 2021.
Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. Geodiff: A geo- metric diffusion model for molecular conformation generation. In International Conference on Learning Representations, 2021.
Wenhui Yang, Edirisuriya M Dilanga Siriwardane, Rongzhi Dong, Yuxin Li, and Jianjun Hu. Crystal structure prediction of materials with high symmetry using differential evolution. Journal of Physics: Condensed Matter, 3354524 2021.
Claudio Zeni, Robert Pinsler, Daniel Zügner, Andrew Fowler, Matthew Horton, Xiang Fu, Sasha Shysheya, Jonathan Crabbé, Lixin Sun, Jake Smith, et al. Mattergen: a generative model for inorganic materials design. arXiv preprint arXiv:2312. 231233322.777.. 2023.
Yong Zhao, Edirisuriya Dilanga Siriwardane, Zhenyao Wu, Nihang Fu, Mohammed Al-Fahdi, Ming Hu, and Jianjun Hu. Physics guided deep learning for generative design of crystal materials with symmetry constraints. npj Computational Materials, 9(1):38, 2023.
Nils ER Zimmermann and Anubhav Jain. Local structure order parameters and site fingerprints for quantification of coordination environment and crystal structure similarity. RSC advances, 10 (10):60636088 2020.
12

---

Published as a conference paper at ICLR 2024
A THEORETICAL ANALYSIS
A.1 PROOF OF PROPOSITION
The proposition is rewritten and proved as follows.
Proposition (Polar Decomposition), An invertible matrix E 123x3 can be uniquely decomposed into L Q SSp(S), where Q R3x3 is orthogonal matrix, S E R3x3 is symmetric matrix and exp S) E भा. defines the exponential mapping of S.
8"
Proof. Given an invertible matrix L 3xx3 first calculate the inner product As J is symmetric, we can formulate its eigendecomposition as J = UAU where U E 3 the square matrix composed by eigenvectors of J and A R diagonal matrix with eigenvalues of J as diagonal elements. The required symmetric matrix can be achieved by S 2 log(AAUT.
L L.
As S is obviously symmetric, we need to prove that Q L exp(S)) orthogonal, i.e. QTQ I.
To this, have
Q = =-==""
Q exp((S)-
Q L Lexp ( log(A)U
LU exp loU(A))UT
=Q'LUVA-TOT
-LUVA-UUT) LUVA-
LUVA-U LUVA-U
-UVA-U = -VVA-AVA- TUT =I.
From the above construction, we further have that the decomposition unique, as exp(S) is positive definite.
A.2 PROOF OF PROPOSITION 2
We begin with the following definition.
Definition 1 (Frobenius Inner Product in Real Space). Given A, B RR333 the Frobenius inner product is defined as (A, B) = tr(A [),, where tr(.) denotes the trace of the matrix.
The proposition 2 is rewritten as follows.
Proposition 2. AS 3x3 S ST, "T,ak= (Ki, S.I. Ei B.
Proof. Based on the above definition, we can easily find that B)) = 0, Vi,j = 1, 6 and * j. meaning that the bases defined in 4.1 are orthogonal bases, and the coeffi- cients of the linear combination S >i B; can be formed as
B))r
k; (B,,
(21)
As the 3D symmetric matrix S E R3x3 has 6 degrees of freedom LLarson 2016). can be uniquely represented by the coefficient vector k (ki, ... ko).
13

---

Published as a conference paper at ICLR 2024
A.3 CONSTRAINTS FROM DIFFERENT CRYSTAL FAMILIES
In Crystallography. the lattice matrix L = 11, l2, la] can also be represented by the lengths a, 4, and angles a, 8.7 7 of the parallelepiped. Specifically, have
a | 1|
= [22 12112
a arccos (La.da)
(22)
bre 3 arccos [g)
(t..4)
arccos
a-b
Based on such notation, the inner product matrix J can be further formulated as
a² ab cos y cos 3
ab cos y ba bocos a
ac
J ==LL
UAU
(23)
bocos c²
Moreover, according to Appendix A.2 space as
can formulate the corresponding matrix in the logarithmic
6 S
T4 +
his
k1
l2
1 U log (AU
k; B, kol +
kn
ks
k4
k3
(24)
h2
K3
22g
2
1=1
We will specify the cases of the crystal families separately follows. Note that different with previous works which directly applies constraints on lattice parameters (Liu et al. 2023). we focus on the constraints in the logrithmic space, which plays significant role designing the diffusion process.
Trielinic. As discussed in Appendix A.2 a triclinic lattice formed by an arbitrary invertible lattice matrix can be represented as the linear combination of 6 bases under no constraints.
Monoclinic, Monoclinic lattices require a 7 -00; where J can be simplified as
a²
0
ac cos 3
J M
0
12
0
(25)
cos
0
c²
Obviously, JM has an eigenvector Gg (0, 1) as JM 1Fez. As J and S have the same
eigenvectors, we have S = (k1, h2 + Kg k4 kig)
AMey for some AM- Hence we directly
have Am = kg + Kg K4 and k = kg
Orthorhombic. Orthorhombic lattices require
= -00, where we have
a²
0
0
Jo =
0
b²
0
(26)
0
2
As So loo(Jo) diag(log(a), log (), log(c)),
directly achieve the solution of k
k 2₂ = K3 = 0,
h4 log(a/b)/2,
(27)
k; log(ab/c2)/6
kg log(abc)/3.
Tetragonal. Tetragonal lattices have higher symmetry than orthorhombic lattices with a b. We have ha by substituting into Eq. (27).
14

---

Published as a conference paper at ICLR 2024
Hexagonal. Hexagonal lattices constrained by as
3 = 90", 7 120", b, which formulate
a²
2
v2
a
v2
0
2
2
Ju
a
22
0
2
2
22
0
(28)
0
0
1
0
0
0
1
And for S, we have
V2
v2
log(a) log({)
v2
0
2
Sи
V2
0
0
log(a) + z log(
/2
6
0
log(c)
0
1.
(29)
log(a) (a) + log(i)
log(3)
0
1 log(3)
log(a) ))
log(
0
(30)
0
0
log(c)
Combine Eq.
24 and Eq.
30
we have the solution
2 k3 his 0,
h log (3) 4,
(31)
k log( v'3a² 1/6,
2x"
hg log( /²c//3.
Cubic. Cubie lattices extend tetragonal lattices b C, changing the solution in Eq. 27 into has = kss 0, kg = log(a).
k
k3
B IMPLEMENTATION DETAILS
B. ARCHITECTURE OF THE DENOISING BACKBONE
We illustrate the architecture of the model described 4.4 Figure embedding 1/FT is defined as
The Fourier coordinate
[sin(2 fe), = 2m,
Ver Ver(()),,
(32)
ccos(2Tm k 2m +
B.2 COMBINATION WITH SUBSTITUTION BASED ALGORITHMS
Crystal structure prediction (CSP) requires predicting the crystal structure from the given composi- tion. To conduct our method on the CSP task, we must initially select appropriate space group and assign each atom a Wyckoff position. We achieve this goal via a substitution bobstttution-basedd method, CSPML (Kusaba et al. 2022), which first retrieves template structure from the training set accord- ing to the query composition, and then substitutes elements in the template with those the query. We depict the prediction pipeline in Figure including the following steps.
Template Retrieval. Given a composition as a query, CSPML initially identifies all structures within the training set that share the same compositional ratio (for instance, 1:1:3 for CaTiO,). The retrieved candidates are then ranked using a model based on metric learning. This model is trained on pairwise data derived from the training set. For structures Mi, obtain compositional finger- prints Fp(c, i), F)(c.j) via XenonPy (Liu et al. 2021) and structural fingerprints Fp(s, i). F) CrystalNN ZZimmermann & Jain 2020p. The model is trained via the binary classification loss
LCSPMI = BCCE((()
Fp(c.j))), 1 (oppd..)-F((s.)lll33
Here, a threshold used to determine if the structures M; and M, are similar. We adopt 8 0.3 in line with the setting of Kusaba et al. (2022). The ranking score defined as (F
15

---

Published as a conference paper at ICLR 2024
Epp
Projection Module
Wyckoff Mean
Wyckoff Mean
RT
uucoosttriined
Output Module
MLP
MLP
MLP
E(//
H
Message-Passing Module
()(-
XL
MLP
(+)
h;
(0)
f: fi
Embedding Module
MLP
Atom Embedding
Time Embedding
A
t
Figure 4: Architecture the denoising model.
Fp(c,k)) for query composition A, and candidate composition At, implying the probability of similarity.
Element Substitution. The second step is to assign the atoms in the query composition to the template with the corresponding element ratio (1/5 for Ca in CaTiOg). For the elements with the same ratio (Ca and Ti), we solve the optimal transport with the Ly-distance between the element deseriptors as the cost.
Refinement. Finally, refine the structure via DiffCSP++ by adding noise to timestep and apply the generation process under the constraints provided by the template. Practically, we select 50 for MPTS-52 and 100 for Perov-5 and MP-20.
16

---

Published as a conference paper at ICLR 2024
Template Retrieval
Query
X
Element Substitution
Refinement
D
Figure 5: Pipeline of DiffCSP++ combined with CSPML.
Table 5: CSP results of the CSPML templates. MR stands for Match Rate.
Perov-S
MP-20
MPTS-52
MR (%)
RMSE
MR (%)
RMSE
MR (%)
RMSE
CSPML (Kusaba et al. 2022 DifICSP+ (w/ CSPML)
51.84
0.1066
70.51 70.58
0.0338 0.0272
36.98 37.17
0.0664
52.17
0.0841
0.0676
More Results. We further provide the performance of the CSPML templates in Table DiffCSP++ exhibits generally bigher match rates and lower RMSE values upon CSPML templates. This un- derscores the model's proficiency in refining structures. Note that the refinement step is independent of the template- finding method, and more powerful ranking models or substitution algorithms may further enhance the CSP performance.
B.3 HYPER-PARE PAIEER--AAMMTERS AND TRAINING DETAILS
We follow the same data split as proposed CDVAE (Xie et al. 2021 and DiffCSP (Jiao et al. 2023). For the implementation of the CSPML ranking models, we construct 100,000 positive and 100,000 negative pairs from the training set for each dataset to train 3-layer MLP with 100 epochs and a 10-3 learning rate. To train the DiffCSP++ models, 6 layers, 512 hidden states, and 128 Fourier embeddings for each task and the training epochs are set to 3500, 4000, 1000. 1000 for Perov-5, Carbon-24, MP-20, and MPTS-52. The diffusion step 1000. We utilize the cosine scheduler with s 0.008 to control the variance of
train a denoising model with
set to
the DDPM process & and A, and exponential scheduler with oi 0.005, 0.5 to
17

---

Published as a conference paper at ICLR 2024
340
Figure 6: Different structures generated upon the space group constraints.
control the noise scale on F. The loss coefficients are set as A
1, = 20. We
apply y 2 x 10 for Carbon-24, 1 x 10-5 for MPTS-52 and 5 10- for other datasets for the corrector steps during generation. For sampling from q' in Eq. (7) we first sample EF (O, I) select RAg for each Wyckoff position acquire e [=,
R f [E, and finally achieve Fý as F" = w/ w + ) where the operation w(.) preserves the fractional part of the input coordinates. To expand the atom types and coordinates of N' Wyckoff positions N atoms, we first ensure that all atoms in one Wyckoff position have the same type, i.e. as, a', and then determine the fractional coordinate of each atom via the basic fractional coordinate f and the corresponding transformation pair (R,,, ), meaning fs. Rs, ts,.
C MORE VISUALIZATIONNS
We provide visualizations in 5.4 from a CSP perspective to demonstrate the proficiency of our method in generating structures of identical composition but within varying space groups. Transi- tioning to the ab initio generation task, we attain an inverse objective, that is, to generate diverse structures originating from the same space group as determined by the template structure. This further illustrated in Figure
D CODES
Our code is available attpps///thuaacm///j/o////f/f/f
18


Reviews:
[{"review_id": "4pf1F6PJHu", "reviewer": "Reviewer_GbgC", "summary": "The authors propose DiffSCP++ - building upon prior work DiffSCP (Jiao et al., 2023), to incorporate inductive biases of space group symmetries in a computationally tractable form for crystal generation. The primary goal in this work is to learn to sample (via diffusion) from a conditional distribution (given the finite space group) rather than an unconditional generation combined with E(n) equivariant networks. The authors do this by decomposing into two parts -  constraints based on an orthogonal group invariant exponential subspace and constraints of fractional coordinates. Subsequently, to tackle the crystal generation problem, the authors employ diffusion models to jointly generate the lattice, the fractional coordinates and atom types conditioned on the obtained constraints pertinent to the space group. The authors then present results on 4 different datasets, for two different tasks (crystal structure prediction and ab-initio crystal generation) where their model outperforms the compared baselines\n\n\nNote to AC: My lower confidence rating is because of lack of expertise with literature relevant to crystal generation.", "questions": "Please address the weakness.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Most parts of the paper are well written and easy to comprehend for someone not familiar with crystal generation literature.\n2. Employs a conditional generation model - which takes into account the space group constraints rather than treating the generation and the invariances as separate modules.\n3. Compares to other recent works, such as PGCGM (Zhao et al., 2023) which also incorporate the affine matrices of the space\ngroup as additional input into a Generative Adversarial Network (GAN) model. The big plus of DiffSCP++ is that it is more widely applicable without being constrained to ternary systems.\n4. Strong experimental results in comparison to DiffSCP (out of which the model was built out of) and other baselines in the crystal structure prediction task.", "weaknesses": "1. Proposition 1 and 2 - are simple extensions of two known results from linear algebra - why is this presented without citations - to linear algebra books from Strang, Roman, etc.?\n2. Novelty is definitely present - but not something completely unexpected and draws and builds upon existing literature", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ehzo5uq1cS", "reviewer": "Reviewer_GFzi", "summary": "A novel approach for crystal structure generation and ab-initio\ncrystal generation. The approach complements a recent solution based\non diffusion models with the support for space group\nconstraints. Experimental results confirm the potential of the\nproposed solution.", "questions": "Does DiffCSP++ in Table 3 use GT space group? is this to be expected? \n\nAlso, see weaknesses for clarification requests.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "Tackling a relevant application which requires non-trivial changes to existing solutions.\n\nA sound and well-motivated approach.\n\nWell-written and well organized manuscript.", "weaknesses": "The relationship of the method with DiffCSP, on which it builds,\nshould be better clarified, so as to more clearly highlight the\ncontribution of the space group constraint. \n\nI am not entirely happy with how results are reported in Table 2. My\nunderstanding is that given that the space group of the GT data is\ntypically not available, the real method is what you call DiffCSP++\n(w/ CSPML) (which I would name DiffCSP++), and this is the method you\nshould compare with the competitors (with boldface for best performing\nmethod etc).  The approach using GT space group (that you name\nDiffCSP++) is an upper bound on the achievable performance, and should\nbe reported as such (including the discussion on future work about\nbetter template-finding algorithm, that you already have in the paper).\n\nThis also affect table 3, in case DiffCSP++ uses GT space group and\nthis is not a kind of knowledge to be expected in ab-initio generation\n(I do not know the domain).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WjB7Z5cZJy", "reviewer": "Reviewer_34Zd", "summary": "This paper proposed to consider crystal symmetry group constraints during crystal structure prediction, and proposed decomposition technic to control the symmetry group for lattice and fractional coordinates of atoms in the cell. The idea is interesting and the performances beyond DiffCSP is reasonable and significant.", "questions": "As listed in the weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The first crystal structure prediction method consider crystal symmetry groups.\n- SOTA performances beyond DiffCSP.\n- Interesting use of crystal symmetries, with controls that can be used during generation.", "weaknesses": "- The model details are not provided, some details about the model are missing. This makes me a little confused about what kind of model they are using, e.g., invariant ones or Equivariant ones.\n- How they construct the crystal structures from reduced number of atoms N' is not clear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)