Periodic Materials Generation using Text-Guided Joint Diffusion Model
Authors: KISHALAY DAS
Subhojyoti Khastagir
Pawan Goyal
Seung-Cheol Lee
Satadeep Bhattacharjee
Niloy Ganguly
Venue: ICLR 2025 Poster (2025)
Keywords: ML4Materials
Diffusion Models
Crystal Materials
Material Generation
AI4Science
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We introduce a novel text-guided diffusion model for generating periodic materials. Our model jointly produces atom fractional coordinates, types, and lattice structures using a periodic E(3)-equivariant graph neural network (GNN).

Abstract: Equivariant diffusion models have emerged as the prevailing approach for generat-
ing novel crystal materials due to their ability to leverage the physical symmetries
of periodic material structures. However, current models do not effectively learn the
joint distribution of atom types, fractional coordinates, and lattice structure of the
crystal material in a cohesive end-to-end diffusion framework. Also, none of these
models work under realistic setups, where users specify the desired characteristics
that the generated structures must match. In this work, we introduce TGDMat, a
novel text-guided diffusion model designed for 3D periodic material generation.
Our approach integrates global structural knowledge through textual descriptions
at each denoising step while jointly generating atom coordinates, types, and lattice
structure using a periodic-E(3)-equivariant graph neural network (GNN). Extensive
experiments using popular datasets on benchmark tasks reveal that TGDMat out-
performs existing baseline methods by a good margin. Notably, for the structure
prediction task, with just one generated sample, TGDMat outperforms all baseline
models, highlighting the importance of text-guided diffusion. Further, in the genera-
tion task, TGDMat surpasses all baselines and their text-fusion variants, showcasing
the effectiveness of the joint diffusion paradigm. Additionally, incorporating textual
knowledge reduces overall training and sampling computational overhead while
enhancing generative performance when utilizing real-world textual prompts from
experts. Code is available at https://github.com/kdmsit/TGDMat

Published as a conference paper at ICLR 2025
PERIODIC MATERIALS GENERATION USING TEXT- GUIDED JOINT DIFFUSION MODEL
Kishalay Das Subhojyoti Khastagir's Pawan Goyal', Seung-Cheol Lee",
Satadeep Bhattacharrje", Niloy Ganguly' Indian Institute of Technology, Kharagpur, India
2 Indo Korea Science and Technology Center, Bangalore, India
ABSTRACT
Equivariant diffusion models have emerged the prevailing approach for generat- ing novel crystal materials due to their ability to leverage the physical symmetries of periodic material structures. However, current models do not effectively learn the joint distribution of atom types, fractional coordinates, and lattice structure of the crystal material in cohesive end-to-end diffusion framework. Also, none of these models work under realistic setups, where users specify the desired characteristics that the generated structures must match. In this work, we introduce TGDMat, a novel text-guided diffusion model designed for 3D periodic material generation. Our approach integrates global structural knowledge through textual descriptions at each denoising step while jointly generating atom coordinates, types, and lattice structure using a periodic-E(3)-equuull experiments using popular datasets on benchmark tasks reveal that TGDMat out- performs existing baseline methods by a good margin. Notably, for the structure prediction task, with just one generated sample, TGDMat outperforms all baseline models, highlighting the importance of text-guided diffusion. Further, in the genera- tion task, TGDMat surpasses all baselines and their text-fusion variants, showcasing the effectiveness of the joint diffusion paradigm. Additionally, incorporating textual knowledge reduces overall training and sampling computational overhead while enhancing generative performance when utilizing real- real-world textual prompts from experts. Code is available at
graph neural network (GNN), Extensive
com/ kdmsit//GDMa..
INTRODUCTION
Screening 3D periodic structures and their atomic compositions to identify novel crystal materials with specific chemical properties remains long- oon-standing challenge in the materials design community. These materials have been fundamental to key innovations such as the development of batteries, solar cells, semiconductors etc. [Butler et al. 20018 Desiraju 2002). Historically, there have been attempts to generate novel materials by conducting resource-intensive and time-consuming simulations based on Density Functional Theory (DFT) (Kohn & Sham 1965). Recently, the equivariant diffusion
models (Jiao et al. 2023 Luo et al. 2023b; Xie et al. 202 have demonstrated great potential to generate stable 3D periodic structures of new crystal materials.
However, these models possess several inherent limitations. 1) None of these existing SOTA models learns the joint distribution of atom coordinates, types, and lattice structure of the material through an end-to-end diffusion network. Existing models like CDVAE (Xie et al.) 202T) and SyMat (Luo) et al.) 2023b) learn lattice parameters and atom types separately using VAE model and further use a score network to learn the conditional distribution of atom coordinates given atom types and lattice. DifICSP (Jiao et al.) (023), on the other hand, focuses primarily on structure prediction task where it assumes atom types are given and predict the stable crystal structure (lattice and coordinates). 2) Furthermore, these models use (E(3)-equvvariant GNNs as backbone denoising network, which largely relies on messages passing around the local neighborhood of the atoms. Hence they fail to incorporate global structural knowledge into the diffusion process, which can enhance the diffusion performance. 3) Finally, these models are unconditional by design. From initial noisy structures without any external constraints, they generate stable erystal structures, which are distributionally
+Correspondence Kishalay: kishalaydas @

---

Published as a conference paper at ICLR 2025
BaPGz is Cable structured and crystaliizss the cubic
Bar 11 bond lenghts 3.37 Pd) bonded to equivatent Bot 1) and sis equivoient Pal atouse to form moxture of foce, edge and corner-ahart cubec teoghts ()- eneray
ceardiuste
eouivoient 1o
materall chemical formula DaPozi The elements formation energy per atom is negative The band gap is zern. The eneray above the corvexx hull 281o. The space groun number 227. The crystal system subIc Generate the
The
0.578, band
bull is o.o. Generate the material
Detailed Description by Rabeerystallograheer
Short Prompt by
Unit Cell Structure
Figure 1: Detailed textual description generated by Roboerysttlllogapher, less-detailed prompts by domain experts, and crystal unit cell structure of BaPdz.
similar to structures of the training dataset. This setup may have limited utility real-world scenarios, as it lacks a mechanism for users to specify criteria for the material to be generated. In realistic setup, users would want to specify certain key details about the target material, like the chemical formula, space group, crystal symmetry, bond lengths, chemical properties, etc as input to the diffusion model, which the generated structure must then match.
In this paper, we propose, TGDMat, novel Geidee Diffussio Model for Material Generation that mitigates the limitations mentioned above and enhances the generation capability. Though
Text Guided Diffusion Models (TGDMs) produce impressively high-qualit data in the form of
images (Nichol et af., 2021 Ramesh et al., 2022 Rombach et al., 2022] Saharia et al. 2022).
audio (Kreuk et al. [022] Yang al. 2022), video (Du | (2024), molecules (Gong al. 2024
Luo et 2022aa) etc, it remains largely unexplored in periodic material generation. Text-guided
diffusion model for new material generation has some key benefits. First, we can leverage popular tools like Roboerystallographe (Ganose & Jain 2019) to generate a textual description of the material which provides a rich and diverse set of global structural knowledge like chemical formula, lattice constraint, space group number, crystal symmetry, chemical properties, etc. We believe this additional information is helpful for diffusion models learning underlying crystal geometry. Second, provides end users the flexibility to use custom prompts to guide the material generation process, ensuring that the resulting material aligns with the user's 's provided description. Towards that goal, we first develop diffusion model that jointly generates the atom coordinates, atom types, and lattice structure of crystal materials using a periodic E(3)-equivariant denoising model, satisfying periodic E(3) invariance properties of learned data distribution. Subsequently, we fuse textual information into the reverse diffusion process, which guides the denoising process in predicting material structure as specified by the textual description. To sum up, our novel contributions in this work are as follows:
To the best of our knowledge, we are the first explore text-guided diffusion for material generation. Our proposed TGDMat bridges the gap between natural language understanding and material structure generation.
Unlike prior models, TGDMat conducts joint diffusion on lattices, atom types, and co- ordinates, enhancing its ability to accurately capture the crystal geometry. Additionally, incorporating global structural knowledge through textual descriptions at each denoising step improves TGDMat's ability generate plausible materials with valid and stable structures.
Extensive experiments using popular datasets on benchmark tasks show TGDMat outper- forms baseline models with good margin. Notably, CSP task, with just one generated sample, TGDMat outperforms all baseline models, highlighting the importance of text- guided diffusion. Moreover, in the generation task, TGDMat outperforms all baselines and their text-fusion variants, showcasing the effectiveness of the joint diffusion paradigm.
Fusing textual knowledge reduces the overall computational cost for both training and infer- ence of the diffusion model. Moreover, when applied to real-world custom text prompts by experts, TGDMat demonstrates rich generative capability under general textual conditions.
2

---

Published as a conference paper at ICLR 2025
2 PRELIMINARIES CRYSTAL STRUCTURE REPRESENTATION
Crystal material can be modeled by minimal unit cell, which gets repeated infinite times in 3D space on regular lattice to form the periodic crystal structure. Given a material with N number of atoms in its unit cell, we describe the unit cell by two matrices: Atom Type Matrix (A) and Coordinate Matrix (X). Atom Type Matrix A = [a1 dd.aa......,,
E IR N denotes set of atomic type in one hot representation (k: maximum possible atom types). On the other hand, Coordinate Matrix X =
NN]" E RR-x3 denotes atomic coordinate positions, where x; RR3 corresponds to coordinates of jth atom in the unit cell. Further, there is an additional Lattice Matrix L = !2.
R3x3, which describes how a unit cell repeats itself in the 3D space towards 4 la and 13 direction form the periodic 3D structure of the material. Formally, a given material can be defined as M = (A, X, L) and we can represent its infinite periodic structure X
{ X, Σ} k24 Å = {ā, â at) where k1, f2, h3, i Z, S i N.
Invariances in Crystal Structure. The basic idea of using generative models for crystal generation is to learn the underlying data distribution of material structure p(M). Since crystal materials satisfy physical symmetry properties (Dresselhaus al., 20077 Zee 2016), one of the major challenges here is the learned distribution must satisfy periodic E(3) invariance i.e. invariance
to permutation, translation, rotation, and periodic transformations. A formal definition of these
invariance properties is provided in Appendix C
RELATED WORK: PERIODIC MATERIAL GENERATION
Recently, the majority of the research on material generation focuses on using popular generative
models like VAEs (Kingma & Welling 2013),
GANs (Goodfellow et al., 2014) or Diffusion
Models (Song& & Ermon 2019 2020; Ho et al.
2020) to generate 3D periodic structures of
materials (Hoffmann et al.
2019 Noh al. 2019
Ren et al. 2020 Kim et al.] 2020 Court et al.
2020 Long et al. 2021 Zhao et al. |222 Xie et al.
2021 Jiao et al. 2023 Luo et al. 2023b Zeni
al. 2023 Yang al.
2023 Jiao al. 2024|
Miller et al. 2024).
In specific, state-of-the-at
models like CDVAE (Xie et al. 2021) and SyMat (Luo et al.] 2023b) combine VAEs and score- score-based
diffusion models work directly with atomic coordinates, ensuring euclidean and periodic invariance
using equivariant graph neural networks(GNNs). Moreover, DifiCSP (Jiao et al., 2023) focuses on structure prediction, jointly optimizing atom coordinates and lattice using a diffusion framework given atomic composition. We provided comprehensive literature review of other related works in Appendix
Relations with Prior Methods, Among existing models, DiffCSP comes close to our methodology, however, our work differs in multiple ways. DiffCSP primarily focuses only on the Structure Prediction (CSP) task and they didn't explore the Random Generation (Gen) task, whereas TGDMat focuses both tasks. Moreover, unlike DiffCSP, TGDMat can leverage the informative textual descriptions during the denoising process and can jointly learn lattices, atom types, and coordinates, which makes TGDMat a more flexible and robust generative model for new material generation.
METHODOLOGY
4.1 PROBLEM FORMULATION
In this work, given the textual description, we focus on generating a stable crystal structure that aligns with the provided textual description. Formally, given a dataset M [M,, T.J. containing crystal structure M, = (A,; X,, L.) and its text description (T:), the goal of text guided crystal generation problem is to capture the underlying conditional data distribution M(M via learning generative model fo (M T), where 0 is a set of learnable parameters. While training, we need fe to ensure that the learned distribution is invariant to different symmetry transformations mentioned in Section 2 Once trained, given a text description of a plausible material, the learned generative model can sample valid and stable structure of the material, that is invariant to different symmetry transformations.
4.2 TEXTUAL DATASETS
Leveraging textual information to guide the reverse diffusion process remains unexplored in the material design community. To the best of our knowledge, there is currently no text data available
3

---

Published as a conference paper at ICLR 2025
for materials in benchmark databases (mentioned in Section (.Il. Hence, we first curate the textual data of these material databases. Specifically, we propose two approaches for generating textual descriptions of materials, which are easy to follow. First, we utilize freely available utility tool, hoboerystallographer (Cianose & Jain 2019) to generate detailed textual descriptions about the periodic structure of crystal materials. These descriptions encompass local compositional details like atomic coordination, geometry, etc. as well as global structural aspects like crystal formula, mineral type, space group information, etc. Secondly, we utilized shorter and less detailed prompts that are more easily interpretable by users. We extend the prompt template proposed by (Gruver et al., 2024), which encodes minimal information about the material like its chemical formula, constituent elements, crystal system it belongs to, and its space group number. Further, we specify a few chemical properties, and instead of mentioning their actual values, we provide generic information like negative/positive formation energy, zero/nonzero band gaps, etc. Detailed information regarding the two textual datasets, including their curation process is provided in Appendix D We have publicly shared the textual datasets for both benchmark material databases with the community for future use.
4.3 PROPOSED METHODOLOGY TGDMAT
Our proposed model, TGDMat (Fig. 2. uses an equivariant diffusion model guided by contextual representation of the textual description (Cp) to generate a new crystal structure M = (A, X.L., L). Unlike prior methods (Jiao et al. 2023 Luo et al. [023b] Xie et al. (02I)) our method jointly diffuses A, X. L to learn the underlying data distribution of erystal structure p piM CC Diffusion models (Ho) et al. 2020, Song & Ermon] 2019: 2020) are popular generative models that are formulated using a T steps Markov Chain. Given an input crystal material Mo = (Ao.Xo Lo), the forward process gradually add noise to Ao, Xo, Lo independently over T steps and the reverse denoising process samples a noisy structure M = (Ar.Xy Xr.Lr) from a prior distribution and reconstruct back Mo using some GNN model. At each hh step of denoising (T 2 t 2 0), the contextual representation of the crystal textual description (C,) will guide the diffusion process so that the intermediate structure M, aligns the target 3D structure constrained on textual conditions. Moreover, the learned distribution of material structure must satisfy periodic E(3) invariance. It is well studied in the literature (Xu al. 2022) that if the prior distribution p(x) is invariant to a group and the transition probabilities of a Markov chain y p(y]ar) exhibit equivariance, the marginal distribution of at any given time step also remains invariant to group transformations. Hence the learned distribution p(Mo) of the denoising model will satisfy periodic E(3) invariance if the prior distribution p(Mrr is invariant and the neural network used to parameterize the transition probability q M,) is equivariant to permutational, translation, rotational, and periodic transformations. To satisfy that, we use periodic- B(3)-equivariant GNN model as a backbone denoising network to guide the denoising process. Next in this section, we first explain diffusion on M in 4. then demonstrate the text- denoising network in 4.3.2 and finally training details in 4.4
4.3.1 JOINT EQUIVARIANT DIFFUSION ON M
Diffusion on Lattice (L). Since the Lattice Matrix L = [1, [2)T E RR333 is in continuous space, we leverage the idea of the Denoising Diffusion Probabilistic Model (DDPM) (Ho et a 2020) for diffusion on L. Specifically, given input lattice matrix Lo ~ p(L). at each th step, the forward diffusion process iteratively diffuses it through a transition probability (Lo( which
(1 II) where, a; I Ogg 0t = 1 3. and {Be E (0, 1)] controls the variance of diffusion step following certain noise scheduler. By reparameterization, we can rewrite L, = vārLo + vT- âze! where, c is noise sampled from N(O,, 1). added with Lo at tth step to generate L, After T such diffusion steps, noisy lattice matrix L1 ~ (O, I) is generated. During reverse denoising process, given noisy Lr NN I) we reconstruct true lattice structure Ly thorough iterative denoising step via learning reverse conditional distribution, which we formulate as p(L+-1 MM,, Gp) = NV CM,, (I-a)) ( where (M, Cp) = vaz (L. -=a
can be derived as (LL,)oo) = (VL
l-ae (M,, Cp, t)). Intuitively, ét needs to be subtracted from L, to VI-ae
generate L,-1 and textual representation Cp will steer this reverse diffusion process. We use text- guided denoising network & (A,A,, X,, Ly, t, C,) to model the noise term e (M,, Cp. t). Following the simplified training objective proposed by (Ho et al. 2020 we train denoising model using 12 loss between él and L
(I) Diffusson on Atom Types (A). Prior studies (Jioo et al. 2023 Xie et al. 2021) consider Atom Type Matrix A as the probability distribution for k classes E X (continuous variable) and apply
Llattice = Ee (0,,,,)))
4

---

Published as a conference paper at ICLR 2025
PONEEEO S M0ny
Input Structure
Forward Process (Nolsing)
Reverse Pracess (denolsing)
(00.)
M, (A, L,)
M (A-1)X1;;144))
Mo (A,X,
SE(3)- Equuvvaaatt GNN model
L,
L
A-1
MatSciBERT
X₁
X-11
Textual Description
Figure 2: Model Architecture of our proposed text guided diffusion model TGDMat. At th step of reverse diffusion, given M, (A X,, Lt), we use periodie-E(3)-cquivval GNN model guided by contextual representation of the textual prompts (C,) to generate M,-1 = X
DDPM to learn the distribution. However for discrete data these models are inappropriate and produce suboptimal results (Austin et al., 2021 Camphell al. 2022). Hence consider as discrete variables belonging to classes and leverage discrete diffusion model (D3PM) (Austin al. 202I) for diffusion on A. In specific, with as the one-hot representation of atom a, the transition probability for the forward process is q(a: aa=1) Cat(ar;p 12)), where Cat(a; categorical distribution over a with probabilities p and Q, is the Markov transition matrix at time step t, defined as @el. q(a = i|a--1 j). Different choices of Q, and corresponding stationary distributions are proposed by (Austin et al., 2021) which provides flexibility to control data corruption and denoising process. We adopted the absorbing state diffusion process, introducing a new absorbing state [MASK] in e,- At each time step t, an atom either stays in its type state with probability 3r or moves to [MASK] state with probability 8, and once it moves [MASK] state, stays there. Hence, the stationary distribution of this diffusion process has all the mass on the [MASK] state. During denoising process, given textual representation Cp. first sample noisy ar and obtain ag thorough iterative denoising step via learning reverse conditional transition pa(a;-1
1 ar,, Cp) Eeo g(a,- 1; (an Pe (aolar, C,). We use the text-guided denoising network bb (A, X Las Cp) to model this denoising process, which is trained using following loss function Lv + XLCE
Ltype
(2)
where Lv and LCE is the variational lower bound and cross-entropy loss respectively and is hyperparameter. Details about the diffusion process and the losses LV LCE in Appendix E
Diffusion on Atom Coordinates (X). We can diffuse the Coordinate Matrix X X2, XN]
I N in two ways: either by diffusing cartesian coordinates or fractional coordinates. Prior works like CDVAE Xie et al. 2021) and SyMat (Luo al. 2023b) diffuse cartesian coordinates whereas DiffCSP (Jiao et al.) 2023) diffuses fractional coordinates. In our setup, we are jointly learning atom coordinates and lattice matrix, hence we follow DiffCSP and diffuse fractional coordinates. Fractional coordinates in crystal material resides in quotient space RR N x3 induced by the crystal periodicity. Since the Gaussian distribution used in DDPM unable to model the cyclical and bounded domain of X, it is not suitable to apply DDPM to model X. Hence at each step of forward diffusion, we add noise sampled from Wrapped Normal (WN) distribution (De Bortoli et al. 2022) to X and during denoising leverage Score Matching Networks (Song & Ermon 2019 2020
5

---

Published as a conference paper at ICLR 2025
to model underlying transition probability (x) (X In specific, each tth step of diffusion, we derive X, as X, = fu (Xo + oTc") where, ex ~ (N,, I), of is the noise scheduler and (.) ) is truncation function. Given a fractional coordinate matrix X, truncation function (X) = (X
X returns the fractional part of each element of X.
As argued in (Jiao al. 2,23), ((Xe/Xo) is periodic translation equivariant, and approaches uniform distribution ((( 1) for sufficiently large values of OT. Hence during the denoising process, we first sample XT ~ (0, 1) and iteratively denoise via score network for T steps to recover back the true fractional coordinates Xp. We use the text- sext-guided denoising network Do (A X Lr, t, Cp) to model the denoising process, which trained using the following score- motching objective function:
Leoord = Ex Vx, logglX,
ex (M,, Cp. 1)112
(-(112)
where Vx, (og x EkEZNx exp(- [|x,-Xo Xo is the score function of transitional distri- 20:
bution and ex (M:, Cp.t) denoising term. More Details are provided in Appendix
4.3.2 TEXT GUIDED DENOISING NETWORK
In this subsection, we will illustrate the detailed architecture of our proposed Text Guided Denoising Network ( X, Lt, f. Cp), which we used during denoising process to generate A, X and As mentioned in 2 the learned distribution of material structure p(M) must satisfy periodic E(3) Graph Neural Network (GNN) integrated with pre-trained textual encoder to model the denoising process. In particular, as a text encoder, we adopt a pre-trained MatSciBERT (Gupta et al.) 2022) model, which is a domain-specifee language model for materials science, followed by a projection layer. MatSciBERT is effectively a pre-trainn pre-trained SeiBERT model on a scientific text corpus of 3.17B words, which is further trained on a huge text corpus of materials science containing around 285M words. We feed textual description of material T into MatSciBERT and extract embedding of [CLS] token hCLs as a representation of the whole text. Further. we feed hcLs through projection layer generate the contextual textual embedding for the material Cp E Rd, which we pass to the equivariant GNN model to guide the denoising process. Practically. as the backbone network for the denoising process, we extend CSPNet architecture (Jiao et al.] (023). originally developed for crystal structure prediction (CSP) task. CSPNet is built upon EGNN (Satorras al., 202T), satisfying periodic E(3) invariance condition on periodic crystal structure. At the 1n layer message passing, the Equivariant Graph Convolutional
invariance. Hence we leverage a
Layer (EGCL) takes input the set of atom embeddings h² [h" Ah : ..., J atom coordinates x = [xf x x "......] and Lattice Matrix L and outputs transformation on h ++1 Formally, we can define the 4th layer message passing operation as:
mij = (h h1, L''L, (
ma.i h k+1
h Ph m:)
(4)
where Pm: Ph are MI.s and vor is a Fourier Transformation function applied on relative difference between fractional coordinates x² x². Fourier Transformation is used since it is invariant to periodic translation and extracts various frequencies of all relative fractional distances that are helpful for crystal structure modeling (Jiao et al.) (023).
We fuse textual representation Cp into input atom feature h. as
(5) where is the timestamp of the diffusion model, fpos (-) is sinusoidal positional encoding (Ho et al. 2020 Vaswani et al. 2017). fatom (.) learned atomic embedding function and || is concatenation operation. Input atom features h² and coordinates x² are fed through K. layers of EGCL to produce él, p(A, MM,) and ex as follows:
= fatom (a:) fpo (t) Cp}
1 e =pr N Σ
9; P(A, )()
M.) PA(( ex px
(6)
where PL, PX are MLPs on the final layer embeddings. Intuitively, we feed global structural knowledge about the crystal structure into the network by injecting contextual representation C, into input atom features. This added signal participates through message-passing operations in Eq. aldd guides in denoising atom types, coordinates, and lattice parameters such that it can capture the global crystal geometry and aligned with the input stable structure specified by textual description.

---

Published as a conference paper at ICLR 2025
Perov-5
Carbon-24
MP-20
Method
Samples
Match Rate
RMSE
Match Rate
RMSE
Match Rate T
RMSE J
48.22
0.4179
17.29
0.3846
15.39
0.3762
P-eG-SchNeet
20
97.94
0,3463
55,91
0,3551
32.64
0,3018
45.31
0,1138
17.09
0.2969
33,90
0.1045
CDVAE
20
88.51
0.0464
88.37
0.2286
66.95
0.1026
52.02
0.0760
17.54
0.2759
$1.49
0.0631
DifCCSP
20
98.60
0.0128
88.47
0.2192
77.93
0.0492
TGDMat
56.54
0.0583
24.13
0.2424
52.22
0.0597
(Short)
20
98.25
0.0137
88.28
0.2252
80.97
0.0443
TGDMat
90.46
0.0203
44.63 95.27
0.2266
55.15
0.0572
(Long)
20
98.59
0.0072
0.1534
82.02
0.0483
Table I: Summary of results on CSP task. We highlight the best and second-best performances in bold and underlined, respectively.
4.4 TRAINING AND SAMPLING
TGDMat is trained using the following combined loss:
X1 Liattice + A + Ax Looord
(7)
where Llattices Ctype and Leoord are lattice 12 loss (Eq, D type loss (Eq. and coordinate score matching loss (Eq. 3) respectively and ÅL, ÅA. Xx are hyperparameters control the relative weightage between these different loss components. During training, we freeze the MatSciBERT parameters and do not tune them further. During sampling, we use the Predictor-Corrector (Song et al., 2020) sampling mechanism to sample An, Xo and Lo- Training/Sampling algorithms are provided in Appendix E.5
EXPERIMENTS
We provide a comprehensive evaluation of our method against several baselines on two benchmark tasks. First, in 5.1 provide brief overview of the experimental setup, including benchmark tasks, and datasets. Next, in 5.2] we demonstrate how textual data enhances the prediction stable crystal structures. Following that, in [5.3] highlight the effectiveness of proposed joint diffusion paradigm in enhancing its ability to generate novel crystal materials. Additionally, we present the correctness of our generated materials 5.4 the computational cost of training and inference and few ablation studies 5.6 Appendix describes additional experiments and more ablations studies.
5.1 BENCHMARK TASKS AND DATASETS
Following the prior works (ie et al. 2021 Jiao et al., 2023). we evaluate our proposed model TGDMat on two benchmark tasks for material generation, Crystal Structure Prediction (CSP) and
Random Material Generation (Gen), using three popular material datasets: Perov-5 (Castelli et al. 2012a b). Carbon-24 (Pickard.) 2020) and MP-20 (Jain al., (013b). curated textual data for these datasets with textual description of each material. Specifically, we generate both long detailed textual descriptions and shorter prompts using approaches mentioned in 4.2 While training TGDMat, split the datasets into the train, test, and validation sets following the convention of 60:20:20 Xie
et al. 2021 More details about the dataset and the experimental setup are Appendix E.
5.2
CRYSTAL STRUCTURE PREDICTION (CSP)
Setup. In CSP task, the goal is to predict the crystal structure (atom coordinates and lattice) given atom types. In text guidance setup. TGDMat utilizes textual descriptions during the denoising process and jointly predicts atom coordinates and lattice parameters from randomly sampled noise. To assess TGDMat's effectiveness, we choose three SOTA generative models: P-eG-SchNet (Gebauer et 2022). CDVAE (Xie et al., 2021), and DiffCSP (Jiao et al. 2023). Following prior works (Tiao et al. 2023 Xie et al. 2021). we evaluate the performance all competing models using standard metrics Match Rate (MR) and RMSE, by matching the generated structure and the ground truth structure in the test set. particular, we generate samples for each material structure in the test set, and determine the matching metrics (MR and RMSE) if at least sample aligns with the ground truth structure, Details about evaluation metrics in Appendix F2
7

---

Published as a conference paper at ICLR 2025
Validity 1
Coverage 1
Property Statisties (EMD)
Dataset
Method
Compositional
Structural
COV-R
COV-P
Element
CDVAE
98.59
100
99.45
98.46
0.0628
0.1258
0.0264
CDVAE+
98.45
99.8
99.53
99.09
0.0609
0.1276
0.0223
SyMat
97.40
100
99.68
98.64
0.0177
0.1893
0.2364
SyMat+
97
99.9
99.70
98.79
0.0172
0.1755
0.2566
Perov-5
DifíCSP
98.85
100
99.74
98.27
0.0128
0.1110
0.0263
DifCCSP+
98 18.44
100
99.85
98.53
0.0119
0.1070
0.0241
(horr))
100
99.7
99,
0.0108
0.0257
TGDMai(Long
98.63
100
99.83
99.52
0.0090
0.0497
0.0187
CDVAE
100
99.8
83.08
0.1407
0.285
CDVAE+
100
84.76
0.1377
0.266
SyMat
100
99.9
97.59
0.1195
3,9576
SyMal+
100
97.63
0.1171
Carbon-24
DifCCSP
100
99.9
97.27
0,0805
0.082
DiffCSP+
100
99,9
97.33
0.0763
0.085
(GoMat(Shorr)
100
99, 8
91.77
0.0681
0.087
TGDMat(Long)
100
99.9
92.43
0.043
0.063
CDVAE
86.70
100
19.15
99.49
1.432
0.6875
0.2778
CDVAE+
87.42
100
99.57
99.81
0.972
0.6388
0.2977
SyMat
100
98.97
99.97
0.5067
0.3805
0.3506
MP-20
DiffCSP
SyMar+
83.25
88.47
99.9
100
99.01
99.71
99.95
99.76
0.3398
0.4865
0.3502
0.3879
0.3489
0.1247
DiffCSP+
85.07
100
99.8
99.89
0.3122
0.3799
0.1355
(GDMar(Short) 86.60
100
99.79
99.88
0.3337
0.3296
0.1154
TGDMai(Long)
92.97
100
99.89
99.95
0,2890
0.3382
0.1189
Table 2: Summary of results on Gen task, with the best and second-ber bestt performances in bold and underlined, respectively. The table contains "1" values for metrics that don't apply to certain datasets.
Results and Discussions. We report the Match Rate and RMSE all the baselines and TGDMat for three benchmark datasets in Table [ We trained TGDMat using both detailed textual descriptions and short prompts, as outlined in 4.2 and report them as TGDMat(Lonng) and TGDMat(Short) respectively in Table [ We observe that both variants of TGDMat surpass all the baseline models with a good margin across three datasets, which shows the rich capability of text-guided joint diffusion predict stable crystal structure. Notably, while prior diffusion models demonstrate improved Match Rates and lower R.SE when generating 20 samples per test material, they largely fail in both metrics when generating only one sample per test material. However, generating 20 samples per test material to match the structures is unrealistic and computationally burdensome. This highlights the importance of text-guided diffusion: incorporating textual knowledge during reverse diffusion aids in aligning the noisy structure with the 3D geometry of stable realistic materials. Specifically, with just one generated sample (k = 1) per test material, both the variants of TGDMat outperform all baseline models, thereby reducing computational overhead. Moreover, even with 20 generated samples (k- = 20) performance of TGDMat significantly better for Carbon-24 24 and MP-20, whereas comparable with DifICSP for Perov-5. In general, due the extensive metadata provided by the detailed description about the crystal structure, the performance enhancement in the long variant surpasses that of the short one. However, these findings collectively demonstrate the effectiveness of text-guided diffusion in the stable crystal structure prediction task.
5.3 RANDOM MATERIAL GENERATION (GEN)
Setup. In Gen task, the goal is to generate novel stable materials (both structure and atom types), that are distributionally similar to the materials in the test dataset. In TGDMat, by design choice, we use the textual description of crystal materials in the test dataset during the reverse diffusion process to enhance the generation capability. To evaluate performance of TGDMat in this task, we choose three popular state-of-the-as generative models: CDVAE (Xie al. 202I). SyMat (Luo et al.) 2023b). and DifCCSP (Jioo et al. (002)). For a fair comparison, we also consider their text-guided variants such as CDVA SyMatt and DiffCSP+ respectively, where we fuse the contextual representation of (Long) text data into those models using the same process described in 4.3.2 Following (Xie et al.)
2021). for evaluating all the competing models, we use seven metrics under three broad categories: Validity, Coverage, and Property Statistics (Details in Appendix F.2 To ensure fair comparison regarding sample size, we generate a number of samples equal to the test data size for all baseline models, both unconditional and text-guided variants, and evaluate their performance accordingly.
Results and Discussions. We report the result of TGDMat (Long and Short) and all the
8

---

Published as a conference paper at ICLR 2025
Long (Detailed) Description
Short Prompl
Ground truth
Generated Sumples
LaNi2Ge2 crystallizes in the tetragonal Below is description of bulk mate- H/mmm space group. La bonded in a 16- rial. The chemical formula La(NiGe)2 coordinate goometry to eight equivalent Ni The elements are La, Ni, Ge. The forma- and eight equivalent bond lengths are 3.25 A.... The Ge-Ge bond zero. The energy above the conves hull length 2.66 The formation energy is zero. The quacegroup number 138. 0.691. The band gap is O.O. The energy above The crystal system is tetrugonal. Gener- the convex hull 0.O.
atoms.
energy negative. The gap is
alc
HgS:NOF is alpha Rhenium trioxide-derieee derived Below is description of a bulk material. structured and crystallizes the ortherhamble The chemical fomuula HgS:NOF, The Pmmm space group. The structure consists elements are Sc....N.O.F. The forma- of one cluster inside ScNOF... linear energy positive. The spacegroup geometry to two equivalent Se atoms. The number is 46. The crystal system is or- formation energy 1.1428.
tburborbicc Generalee the material.
Table 3:
Visualization of the generated structures given textual description. Note that our model pro- duces rotated or translated versions of the ground truth material, owing to the periodic-E(3)) Jivvarriee
baseline models in Table We observe that both variants of TGDMat consistently enhances performance across almost all metrics across the benchmark datasets. Particularly on the Perov-5 dataset, TGDMat outperforms all baseline models across all metries except for compositional validity, where its performance is on par with state-of paat-off-the-at afe-the-rt results. In the Carbon-24 and MP-20 datasets, TGDMat exhibits performance improvements across all metrics except for COV-P. Additionally, our experiments indicate that utilizing shorter prompts results in slight decrease in overall performance compared to the longer variant. Nonetheless, the performance remains comparable to baseline models. Overall, TGDMat exhibits promising performance in the Random Material Generation task, indicating its effectiveness in integrating global textual knowledge into the reverse diffusion process to generate more stable periodic structures of 3D crystal materials. Moreover TGDMat outperforms text guided variant of baseline models CDVAE+, SyMat+ and DiffCSP+ as well, which highlights the effectiveness of using joint diffusion to learn A.X and L.
Additionally, we present visualizations of few generated materials based on the textual descriptions in Table 3 and compare them with the ground truth material structure. The generated samples exhibit clear matches to the ground truth structure, highlighting the generation capability of the TGDMat model given information in text form. More visualization results are in Appendix R.7
5.4 CORRECTNESS OF GENERATED MATERIALS
Setup. In this section, we investigate whether the generated material matches different features specified by the textual prompts. TGDMat has the capability to process textual prompts given by the user, enabling it to manage global attributes about crystal materials such as Formula, Space group, Crystal System, and different property values like formation energy, band-gap, etc. To ensure the fidelity of our model's outputs concerning these specified global attributes from the text prompt, We randomly generated 1000 materials (sampled from all three Datasets) based on their respective textual descriptions (both Long and Short) and assessed the percentage of generated materials that matched the global features outlined in the text prompt. In specific, we matched the Formula, Space group, and Crystal System of generated materials with the textual descriptions. Moreover, we examined whether properties such as formation energy and bandgap matched the specified criteria as per the text prompt (positive/negaaiiee, zero/nonzero).
Results and Discussions. We report the results for TGDMat using long prompt in Table and short prompts in Table 8 F.4. In general, using longer text, considering Perov-5 and Carbon-24 datasets, the generated material meets the specified criteria effectively. However, when dealing with the MP-20 dataset, which is more intricate due to its complex structure and composition, performance tends to decline. Additionally, when using shorter prompts, overall performance suffers across all datasets compared to longer text inputs. This is because the longer text, provided by the robocrystallographer, offers a comprehensive range of information, both global and local
5.5 COMPUTATIONAL COST FOR TRAINING AND SAMPLING
Integrating textual knowledge during reverse diffusion for crystal material generation offers a key advantage: it accelerates convergence towards realistic structures and reduces computational overhead. We observe, compared to other baseline models, TGDMat incurs substantially lower computation costs during training and sampling processes. Compared to baseline models, our approach notably

---

Published as a conference paper at ICLR 2025
Global Features in Text Prompt
% of Matched Matcrials
Perov-5
Carbon-24
MP-20
Formula
97.50
98.20
70.54
Space Group
87.00
80.79
67.88
TGDg)
Crystal System
92.60
91.55
73.54
Formation Energy
95.49
92.88
Band Gap
98.61
96.73
Table Correctness of generated materials matching conditions specified by the textual prompts.
Perov-5
Carbon-24
0:97
0.95
0.95
0.90
0.93
0.80
0.90
(a) Match Rate vs Running time
(b) Smoothness of M'DMaa
Figure 3: (a) Match Rate vs Running time (GPU Hours) for different variants of TGDMat(Long)
{50 Steps

---

Published as a conference paper at ICLR 2025
7 ACKNOWLEDGMENT
This work was funded by Indo Korea Science and Technology Center, Bangalore, India, under the project name "Generating Stable Periodic Materials using Diffusion Models". We thank the Ministry of Education, Govt of India, for supporting Kishalay with Prime Minister Research Fellowship (PMRF) during his Ph.D. tenure.
REFERENCES
Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems, 34:17981-17999 2021.
Keith T Butler, Daniel W Davies, Hugh Cartwright, Olexandr Isayev, and Aron Walsh. Machine learning for molecular and materials science. Nature, 559(7715):4475555 2018.
Andrew Camphell, Joe Benton, Valentin De Bortoli, Thomas Rainforth, George Deligiannidis, and Arnaud Doucet. A continuous time framework for discrete denoising models. Advances Neural Information Processing Systems, 35:28266-28279, 2022.
Ivano E Castelli, David D Landis, Kristian S Thygesen, Søren Dahl, Ib Chorkendorff, Thomas F Jaramillo, and Karsten W Jacobsen. New cubic perovskites for one-and two-photon water splitting using the computational materials repository. Energy & Environmental Science, 5(10):9034 90433403333 2012a.
Ivano E Castelli, Thomas Olsen, Soumendu Datta, David D Landis, Søren Dahl, Kristian S Thygesen, and Karsten W Jacobsen. Computational screening of perovskite metal oxides for optimal solar light capture. Energy & Environmental Science, 5(2):8814 581999999 2012b.
Chi Chen, Weike Ye, Yunxing Zuo, Chen Zheng, and Shyue Ping Ong. Graph networks as universal machine learning framework for molecules and crystals. Chem. Mater, (1()):3564 2019.
Kamal Choudhary and Brian DeCost. Atomistic line graph neural network for improved materials property predictions, npj Computational Materials, 7(1):1-8, 2021.
Callum J Court, Batuhan Yildirim, Apoorv Jain, and Jacqueline M Cole. 3-d inorganic crystal structure generation and property prediction via representation learning. Journal of Chemical Information and Modeling, 60(10):451) 4535, 2020.
Kishalay Das, Bidisha Samanta, Pawan Goyal, Leung-Cheol Lee, Satadeep Bhattacharjee, and Niloy Ganguly. Crysxpp: An explainable property predictor for crystalline materials. npj Computational Materials, 8(1):43, 2022.
Kishalay Das, Pawan Goyal, Seung-Cheol Lee, Satadeep Bhattacharjee, and Niloy Ganguly. Crys- mmnet: multimodal representation for crystal property prediction. In Uncertainty Artificial Intelligence, pp. 507-517. PMLR, 2023a.
Kishalay Das, Bidisha Samanta, Pawan Goyal, Seung- Lee, Satadeep Bhattacharjee, and Niloy Ganguly. Crysgnn: Distilling pre-trained knowledge to enhance property prediction for crystalline materials. arXiv preprint arXiv:2301.00 05555,, 2023b.
Daniel Davies, Keith Butler, Adam J Jackson, Jonathan M Skelton, Kazuki Morita, and Aron Walsh. Smact: Semiconducting materials by analogy and chemical theory. Journal of Open Source Software, 4(38):1361, 2019.
Valentin De Bortoli, Emile Mathieu, Michael Hutchinson, James Thornton, Yee Whye Teh, and Arnaud Doucet. Riemannian score-based generative modelling. Advances in Neural Information Processing Systems, 35:2406-2422. 2022.
Gautam Desiraju. Cryptic crystallography. Nature materials, 1(2):77-79, 2002.
Jacob Devlin, Ming-Wor Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv: /0/0v 180.00008 2018.
11

---

Published as a conference paper at ICLR 2025
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. Advances in neural information processing systems, 34:8780-8799 2021.
Mildred S Dresselhaus, Gene Dresselhaus, and Ado Jorio. Group theory: application to the physies of condensed matter. Springer Science & Business Media, 2007.
Yilun Du, Sherry Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Josh Tenenbaum, Dale Schuurmans, and Pieter Abbeel. Learning universal policies via text-guided video generation. Advances in Neural Information Processing Systems, 36, 2024.
Octavian Ganea, Lagnajit Pattanaik, Connor Coley, Regina Barzilay, Klavs Jensen, William Green, and Tommi Jaakkola. Geomol: Torsional geometric generation of molecular 3d conformer ensembles. Advances in Neural Information Processing Systems, 34:13757-13769, 2021.
Alex M Ganose and Anubhav Jain. Roboerystallographer automated erystal structure text descrip- tions and analysis. MRS Communications, 9()):8748881, 2019.
Niklas WA Gebauer, Michael Gastegger, Stefaan SP Hessmann, Klaus-Robert Müller, and Kristof T Schütt. Inverse design of 3d molecular structures with conditional generative neural networks. Nature communications, 13(1):973, 2022.
Haisong Gong, Qiang Liu, Shu Wu, and Liang Wang. Text-guided molecule generation with diffusion language model. arXiv preprint 10X1v:2402.13044. 2024.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances neural information processing systems, 27, 2014.
Nate Gruver, Anuroop Sriram, Andrea Madotto, Andrew Gordon Wilson, C Lawrence Zitnick, and Zachary Ulissi. Fine-tuned language models generate stable inorganic materials as text. arXiv preprint arXiv:2402. 244.:24902.04779, 2024.
Tanishq Gupta, Mohd Zaki, N. M. Anoop Krishnan, and Mausam. MatSciBERT: A materials domain language model for text mining and information extraction. npj Computational Materials, 8(1):102, May 2022. ISSN 2057-3960. doi: 10. 00000///41544-22007788--003 URL https // //www. , nature. /ttpa//ccceeee1114422222-0000000
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance, 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-688 6851, 2020.
Jordan Hoffmann, Louis Maestrati, Yoshihide Sawada, Jian Tang, Jean Michel Sellier, and Yoshua Bengio. Data-driven approach to encoding and decoding 3-d crystal structures. arXiv preprint 000,009900,004,, 2019.
Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling, Argmax flows and multinomial diffusion: Learning categorical distributions. Advances in Neural Information Processing Systems, 34:12454-12465. 2021.
Anubhav Jain, Shyue Ping Ong, Geoffroy Hautier, Wei Chen, William Davidson Richards, Stephen Dacek, Shreyas Cholia, Dan Gunter, David Skinner, Gerbrand Ceder, and Kristin A. Persson. Commentary: The Materials Project: A materials genome approach to accelerating materials innovation. APL Materials. 1(1):0110022 07 2013a. ISSN 2166-532X. doi: 10. 00.106//..881323, htt.tttps//aol..g///00011111 4812322
Anubhav Jain, Shyue Ping Ong, Geoffroy Hautier, Wei Chen, William Davidson Richards, Stephen Dacck, Shreyas Cholia, Dan Gunter, David Skinner, Gerbrand Ceder, et al. The materials project: A materials genome approach to accelerating materials innovation, apl mater. API. Mater, 201 3b.
Rui Jiao, Wenbing Huang. Peijia Lin, Jiaqi Han, Pin Chen, Yutong Lu, and Yang Liu. Crystal structure prediction by joint equivariant diffusion. arXiv preprint 2309 04475, 2023.
12

---

Published as a conference paper at ICLR 2025
Rui Jiao, Wenbing Huang, Yu Liu, Deli Zhao, and Yang Liu. Space group constrained crystal generation. arXiv preprint arXiv:2402.03997 009.0.....9222.. 2024.
Sungwon Kim, Juhwan Noh, Geun Ho Gu, Alan Aspuru-Guzik and Yousung Jung. Generative adversarial networks for crystal structure prediction. ACS central science, (1)::1112-4220 2020.
Diederik P Kingma and Max Welling. arXiv:1312.61 2013.
Auto-encoding variational bayes.
arXiv preprint
Walter Kohn and Lu Jeu Sham. Self-consistent equations including exchange and correlation effects. Physical review, 140(4A):A1133) 1965.
Felix Kreuk, Gabriel Synnaeve. Adam Polyak, Uriel Singer, Alexandre Défossez, Jade Copet, Devi Parikh, Yaniv Taigman, and Yossi Adi. Audiogen: Textually guided audio generation. arXiv preprint arXiv:2209, /5352, 2022.
Meng Liu, Keqiang Yan, Bora Oztekin, and Shuiwang Ji. Graphebm: Molecular graph generation with energy-based models. arXiv preprint arXiv:2102.00 2021.
Teng Long, Nuno M Fortunato, Ingo Opahle, Yixuan Zhang, Ilias Samathrakis, Chen Shen, Oliver Gutfleisch, and Hongbin Zhang. Constrained crystals deep convolutional generative adversarial network for the inverse design of crystal structures. npj Computational Materials, 7(1):66, 2021.
Steph-Yver Yves Louis, Yong Zhao, Alireza Nasiri, Xiran Wang, Yuqi Song, Fei Liu, and Jianjun Hu. Graph convolutional neural networks with global attention for improved materials property prediction. Physical Chemistry Chemical Physics. 22(32):1141111111
2020.
Shitong Luo, Yufeng Su, Xingang Peng, Sheng Wang, Jian Peng, and Jianzhu Ma. Antigen-specific antibody design and optimization with diffusion-based generative models for protein structures. Advances in Neural Information Processing Systems, 35:9754-9767, 2022.
Yanchen Luo, Sihang Li. Zhiyuan Liu, Jiancan Wu, Zhengyi Yang. Xiangnan He, Xi- ang Wang, and Qi Tian. Text-guided diffusion model for 3d molecule generation. 2023a.
Youzhi Luo, Chengkai Liu, and Shuiwang Ji. Towards symmetry-aware generation of periodic materials. In Thirty-sevnnth Conference on Neural Information Processing Systems, 2023b. URL
net. worumoorooooooooooooo dd=jkcc 4vn1aZ
Benjamin Kurt Miller, Ricky TQ Chen, Anuroop Sriram, and Brandon M Wood. Flowmm: Generating materials with riemannian flow matching. arXiv preprint arXiv:2406.047/5 2024.
Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10744 2021.
Juhwan Noh, Jaehoon Kim, Helge S Stein, Benjamin Sanchez- John M Gregoire, Alan Aspuru-Guzik, and Yousung Jung. Inverse design of solid-state materials via a continuous repre- sentation. Matter, 155):1370-1384 2019.
Shyue Ping Ong, William Davidson Richards, Anubhav Jain, Geoffroy Hautier, Michael Kocher, Shreyas Cholia, Dan Gunter, Vincent L Chevrier, Kristin A Persson, and Gerbrand Ceder. Python materials genomics (pymatgen): robust, open-source python library for materials analysis. Computational Materials Science, 68:314-318 2013.
Cheol Woo Park and Chris Wolverton. Developing an improved crystal graph convolutional neural network framework for accelerated materials discovery. Physical Review Materials, 4(6), Jun 2020. ISSN 2475-9953, doi: 10. 10.1 4.06380. URL httt::////a doi org/0 1103/PhysRewWateii.k
/ttt::://xx.doooog//110
063801
Chris J. Pickard. Airss data for carbon at 10gpa and the ctnthto system at Igpa. alxcloud:2020.00260 2020.
materi-
13

---

Published as a conference paper at ICLR 2025
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and llya Sutskever. Learning transferable visual models from natural language supervision, 2021.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.061125, 1()):3 2022.
Zekun Ren, Juhwan Noh, Siyu Tian, Felipe Oviedo, Guangzong Xing, Qiaohao Liang, Armin Aberle, Yi Liu, Qianxiao Li, Senthilnath Jayavelu, al. Inverse design of erystals using generalized invertible crystallographic representation. arXiv preprint arXiv:2005.07609, 3(6):7, 2020.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684 10684-10695, 2022.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily I. Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic texi-to-image diffusion models with deep language understanding. Advances neural information processing systems, 35:36479-36494, 2022.
Victor Garcia Satorras, Emiel Hoogehoom, and Max Welling. E (n) equivariant graph neural networks In International conference on machine learning, pp. 9323-9332. PMLR, 2021.
Jonathan Schmidt, Love Pettersson, Claudio Verdozzi, Silvana Botti, and Miguel AL Marques. Crystal graph attention networks for the prediction of stable materials. Science Advances, 7(49): cabi7948, 2021.
Chence Shi, Shitong Luo, Minkai Xu, and Jian Tang. Learning gradient fields for molecular conformation generation. In International conference on machine learning, pp. 9558-9568. PMLR, 2021.
Jascha Sohl-Dicke Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning. pp. 2256-2265. PMLR, 2015.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Yang Song and Stefano Ermon. Improved techniques for training score- based generative models. Advances in neural information processing systems, 01112248-1244 2020.
Yang Song, Jascha Sohl- Sehl-Dickstten, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv: 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Jiaxiang Wu, Tao Shen, Haidong Lan, Yatao Bian, and Junzhou Huang. Se (3) equiivarai energy- based models for end-to-end protein folding. bioRxiv. pp. 2021-06, 2021.
Tian Xie and Jeffrey C Grossman. Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties. Phys. Rev. Lett., (20(14):145301 2018
Tian Xie, Xiang Fu, Octavian Eugen Ganea, Regina Barzilay, and Tommi Jaakkola. Crystal diffusion variational autoencoder for periodic material generation. arXiv preprint arXiv:2110.06199, 2021.
Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. Geodiff: A geometric diffusion model for molecular conformation generation. arXiv preprint arXiv:2203.029) 2022.
Keqiang Yan, Yi Liu, Yuchao Lin, and Shuiwang Ji. Periodic graph transformers for crystal material property prediction. In The 36th Annual Conference on Neural Information Processing Systems, 2022.
14

---

Published as a conference paper at ICLR 2025
D Yang, J Yu, H Wang. W Wang, C Weng, Y Zou, and D Diffsound Yu. Discrete diffusion model for text-to-sound generation. arxiv 2022. arXiv preprint arXiv:2207.09983, 2022.
Mengjiao Yang, KwangHwan Cho, Amil Merchant, Pieter Abbeel, Dale Schuurmans, Igor Mor- datch, and Ekin Dogus Cubuk. Scalable diffusion for materials generation. arXiv preprint urXiv:2331.09222 2023.
Anthony Zee. Group theory in nutshell for physicists, volume 17. Princeton University Press, 2016.
Claudio Zeni, Robert Pinsler, Daniel Zügner, Andrew Fowler, Matthew Horton, Xiang Fu, Sasha Shysheya, Jonathan Crabbé, Lixin Sun, Jake Smith, et al. Mattergen: a generative model for inorganic materials design. arXiv preprint arXiv:2312.03668, 2023.
Yong Zhao, Mohammed Al-Fahdi,, Ming Hu, Edirisuriya MD Siriwardane, Yuqi Song, Alireza Nasiri, and Jianjun Hu. High-throughput discovery of novel cubic crystal materials using deep generative neural networks. Advanced Science, 8(20):2100566, 2021.
Huaisheng Zhu, Teng Xiao, and Vasant G Honavar. 3m-diffusion: Latent diffusion for text-guided generation of molecular graphs, 2024.
15

---

Published as a conference paper at ICLR 2025
PERIODIC MATERIALS GENERATION USING TEXT-GUIDED JOINT DIFFUSION MODEL (TECHNICAL APPENDIX)
A LIMITATIONS AND FUTURE WORK
1) One of the major limitations and scope of the future work of proposed work is the lack of independent textual datasets for material generation tasks. In our experimental setup, our model relied on textual data extracted from existing datasets. Initially, we extracted text data from CIF files of materials in the test sets of Perov-5, Carbon-24, and MP-20, utilizing this data to evaluate our model and all baseline models. While the experimental results show promise, a more robust evaluation could have been achieved with an independent dataset containing only textual prompts. This would enable us to assess how effectively these models can generate the underlying 3D structure of materials through a text-guided diffusion process. Hence curating an independent textual dataset for material generation containing diverse set of meta-information will future scope for research.
2) Given text promptsddescription,, we generate contextual representation using a text en- coder in TGDMat, where we adopted a pre-trained MatSciBERT (Gupta et al. (2022) model, which is a domain-specifie language model for materials science. Also, while training TGDMat, we freeze the MatSciBERT parameters and do not tune them further. Moreover, during sampling, the user must follow a specific format (Long/Short) to provide the text description of the target material. This setup limits the expressive power of the textual representation. We investigated the robustness of TGDMat with much shorter prompts to sample from pre- tr-inaaidd TGDMat model in F.5 but observed performance degradation across all the benchmark dataset on Gen task. Hence, Exploring state-of-the-art LLMS and further fine-tuning them during training may create more powerful text conditional diffusion models and provide flexibility to process text prompts of different formats. However, that might create computational overhead as it will increase the number of parameters significantly. This provides scope for further investigation and we keep it as scope for future work.
B MORE RELATED WORK
B.1 CRYSTAL REPRESENTATION LEARNING
In recent times, graph neural network (GNN) based approaches have emerged as a powerful model in learning robust representation of crystal materials, which enhance fast and accurate property prediction. CGCNN (Xie & Grossman) 2018) is the first proposed model, which represents a 3D crystal structure an undirected weighted multi-edge graph and builds graph convolution neural network directly on the graph. Following CGCNN, there are a lot of subsequent studies (Chen et al.
2019: Choudhary DeCost [021; Das al.. 2023a Louis al. 2020 Park & WWolveronn [020 Schmidt et al., 2021), where authors proposed different variants of GNN architectures for effective crystal representation learning. Recently, graph transformer-he based architecture Matformer (Yan) al. 2022) is proposed to learn the periodic graph representation of the material, which marginally improves the performance, however, is much faster than the prior SOTA model. Moreover, scarcity of labeled data makes these models difficult to train for all the properties, and recently, some key studies (Das et al. 2022 2023b) have shown promising results to mitigate this issue using transfer learning, pre-training, and knowledge distillation respectively.
B.2 DIFFUSION MODELS
The fundamental idea of the diffusion model, as initially proposed by (Sohi-Dicksteed et al., 2015). is to gradually corrupt data with diffusion noise and learn a neural model to recover back data from noise. Idea of diffusion further developed in two broad categories 1) Score Matching Network (Song & Ermon 2019 2020) and 2) Denoising Diffusion Probabilistic Models (DDPM) (Ho) adl,2000) recent times diffusion models have emerged a powerful new family of deep generative models, achieving remarkable performance records across numerous applications such as image synthesis (Dhariwal Nichol
[021]] Ramesh et al. [002 Rombach et al. (0022), molecular
conformer generation (Shi et al.
2021 Xu et al. (022). molecular graph generation (Liu et al.)
2021),
protein folding (Luo ct al.
2022
Wu ct al. 2021 etc.
B.3
CONDITIONAL DIFFUSION MODELS
The initial DDPM model (Ho et al. 2020) demonstrated unconditional diffusion models for image generation, where the output cannot be directed towards a desired characteristic or property. In guided
16

---

Published as a conference paper at ICLR 2025
DitCCSP
TGDMat
Tasks
Only CSP Task
Both CSP and Gen Tasks
Diffusion Atom Type
Discrete Diffusion (D3PM)
Model Category
Unconditional: unable to specify the criteria required the user
Conditional; able specify the criteria required by the user (in Text Format)
Text Guided Ditfusion
No
Yes
Table 5: Key Differences between TGDMat from DiffCSP
diffusion models, the sampling process can be steered by prompt, which can be textual description of the desired output, reference image, or any other type of media.
In the field of image generation by diffusion models, Ramesh et al (Ramesh et al., 2022 came up with a text-guided diffusion model called Dall-E2 which showed how textual prompt can be used to steer the sampling process. While training the model, both the image and its textual description are encoded and mapped together, and the encoding of the prompt is used to generate the image during sampling. Another way of guiding the diffusion process using a separate classifier model was shown by (Dhariwal & Nichol 202I). They trained a classifier on the noised images and used the gradient of the classifier to guide the sampling process. In the classifier-fee setting. (Ho Salimans 2022) trained two diffusion models, one guided and one unguided, and combined the resulting score estimated during sampling to get the desired outcome. OpenAl's CLIP (Radford et al. 2021) further improved the relevance of the generated image to the given prompt by scoring the correctness of the generated image given the textual prompt.
Similar efforts have been made in the field of molecular generative models. The shortcoming of SMILES-based autoregressive models were addressed by TGM-DLM (Gong al. 2024) by utilizing diffusion models. This necessitates a two step process, text-guided generation phase, where the SMILES representation is generated from Gaussian noise with the help of textual description, and correction phase, where necessary rectification are made for the correctness of SMILES string format. This is one of the drawbacks of the SMILES string format, which was addressed by 3M-Diffusion 2024). where they have generated molecular graphs from given textual description. (Zhu et al.
B.4 CRYSTAL MATERIAL GENERATION
In the past, there were limited efforts in creating novel periodic materials, with researchers concentrating on generating the atomic composition of periodic materials while largely neglecting the 3D structure. With the advancement of generative models, the majority of the research focuses on using popular generative models like VAEs or GANs to generate 3D periodic structures of materials, however, they either represent materials as three-dimensional voxel images (Court et al. 2020 Hoffmann et al.) 2019; Long et al., 2021 Noh et al.) 2019) and generate images to depict material structures (atom types, coordinates, and lattices), or they directly encode material structures as embedding vectors (Kim et al.) 2020; Ren et al.) 2020; Zhao et al. 2021I. However, these models neither incorporate stability in the generated structure nor are invariant to any Euclidean and periodic transformations. In recent times equivariant diffusion models (Xie et al. 1021
Luo et al. 2023b Jiao et al., 2023 Yang al. 2023 Tiao al.) 2024 Miller al.) 2024 have become the leading method for generating stable crystal materials, thanks to their capability to utilize the physical symmetries of periodic material structures. In specific, state-of-the-art models
like CDVAE (Xie et al. 202T) and SyMat (Luo et al. 2023b) integrate a variational autoencoder (VAE) and powerful ccore-based decder network, work directly with the atomic coordinates of the structures and uses an equivariant graph neural network to ensure euclidean and periodic invariance. However, both CDVAE and SyMat first predict the lattice parameters and atomic composition using the VAE model and subsequently update the coordinates using score based diffusion model. Moreover, given atomic composition, DiffCSP (Jiao et al., 2023) jointly optimizes the atom coordinates and lattice using diffusiin framework to predict the crystal structure with high precision.
Relations with Prior Methods. Among the existing models, DiffCSP TTao et al. 2023) comes close to our methodology, however our work differs from it in multiple ways. DiffCSP primarily focuses only on the Crystal Structure Prediction (CSP) task and they didn't explore
17

---

Published as a conference paper at ICLR 2025
the Crystal Generation task, whereas TGDMat focuses both tasks. Moreover, unlike DiffCSP, TGDMat can leverage the informative textual descriptions during the reverse diffusion process and can jointly learn lattices, atom types, and fractional coordinates from randomly sampled noise. This makes TGDMat more flexible and robust in Crystal Generation and Structure Prediction tasks.
C INVARIANCES IN CRYSTAL STRUCTURE
The basic idea of using generative models for crystal generation is learn the underlying data distribution of material structure p(M). Since crystal materials satisfy physical symmetry properties (Dresselhaus al., 2007 Zee 2016), one of the major challenges here is the learned distribution must satisfy periodic E(3) invariance i.e. invariance permutation, translation, rotation, and periodic transformations.
Permutation Invariance: If we permute the indices of constituent atoms it will not change the material. Formally, given any material M = (,),, using any permutation matrix P if we permute and X (AA) and P(X), then new material Mp remains unchanged. Hence the underlying distribution is also the same i.e ((M) = p(Mp).
P(X), L) will
Translation Invariance: we translate the atom coordinates by a random vector will not change the structure of the material. Formally, given any material M (A, X. L.), if we translate X by an arbitrary translation vector u E R3, new generated material Mr = (A,X u1" L) will be the same as M. Hence p(M) = p(Mr) must satisfy.
Rotational Invariance: If we rotate the atom coordinates and lattice matrix, the material remains unchanged. Formally, using any orthogonal rotational matrix Q E R3x3 (satisfying I), if we rotate X and L of any material M and generate new MR (A, QX, QL).
Q Q'Q then actually different representations of the same material. Hence p(M) = p(Mæ) must satisfy.
Periodic Invariance Finally, since the atoms in the unit cell can periodically repeat itself infinite times along the lattice vector, there can many choices of unit cells and coordinate matrices representing the same material. Formally, given coordinates X, after applying periodic transformation using random matrix K E (3 new coordinates X' = X + KL are periodically equivalent. Hence M (A and M (A,X', L) are same material and p(M)= = p(M') must hold.
D TEXTUAL DATASET
Leveraging textual information to guide the reverse diffusion process remains unexplored in the material design community. To the best of our knowledge, there is currently dataset available that includes textual descriptions of the materials present in standard benchmark databases (Section (5.1) used for material generation. In specific, propose two methods for generating textual descriptions of materials. Hence, we first curate the textual dataset containing textual descriptions of these materials to train our model.
Long Detailed Textual Description: First, we utilize freely available utility tool known as Robocrystallographer (Ganose & Jain 2019) to generate detailed textual descriptions about the periodic structure of crystal materials encoded in Crystallographic Information Files (CIF Files). Robocrystallographer breaks down crystal structures into two main components: local compositional details such as atomic coordination, geometry, polyhedral connectivity, and tilt angles, as well as global structural aspects like crystal formula, mineral type, space group information, symmetry, and dimensionality. This information is presented three formats: JSON for machine processing, humann-readabee text for easy comprehension akin to descriptions provided by humans, and machine learning format for specialized analysis. We choose the readalee text format to compile textual datasets, which closely resemble descriptions given of the crystal structure by humans.
Short Custom Prompis: Secondly, we utilized shorter and less detailed prompts that are more easily interpretable by users. We extend the prompt template proposed by (Gruver et al. 2024h, which encodes minimal information about the material like its chemical formula. constituent elements, crystal system it belongs to, and its space group number. Further, we specify a few chemical properties, and instead of mentioning their actual values, provide generie information
18

---

Published as a conference paper at ICLR 2025
like negative/positive formation energy, zero/nonzero band gaps, etc. We used the Pymatgen tool (Ong et al., 2013) to extract this information from the Crystallographic Information Files (CIF Files) and curate the textual prompts.
An illustrative example of both these textual descriptions and the unit cell structure is pro- vided in Figure
BsPd2 Cublc structured and crystallizes in the cubic Fd-3m space group. Ba(1) banded in 12- c--oodddaaee geometry to tweive
description material. The chemical formula BaPdZ. The elements are Ba, Pd. The energy per negative. The band gap is zero. The energy above the convex hull is zero. space group Generatee the crystal system material
equivolent Pd(1) atoms. All Ba(1)-Pd(1) bund lengths are 3.37 equivalent Bait) and six equivalent Pd(t) atoms form mixture face, corner-shoring cubactahedra. Pdr PdF()-) bond lengths are 2.88 formation energy per is -0.578, gap 0.0, energy hull is O.O. Generate the material
Pd(1) banded
and
Detailed Description by Robocrystallographer
Short Prompt by Users
Unit Cell Structure
Figure Detailed textual description generated by Roboerystallographer, chort/less-detallee prompts by experts, and crystal unit cell structure of BaPd, from Material Projects dataset. Text generated by Robocrystallographer contains both local chemical compositional information related atom/bonds (like site coordination, geometry, polyhedral connectivity, and tilt angles) and global structural knowledge (like mineral type, space group information, symmetry, and dimensionalityy. The shorter prompt encodes minimal information about the material like its chemical formula, constituent elements, crystal system, and few chemical properties.
E JOINT EQUIVARIANT DIFFUSION ON M
Given an input crystal material Mo (Au, Xo, Lo). we define a forward diffusion process through Markov chain over steps to defuse A. X, independently as follows:
(A,, X, - L,-1
= (A, (x
t = 1,2,
(8)
E.1 DIFFUSION ON LATTICE (L)
Lattice Matrix [4, 43)" 3x33 global feature of the material which determines the shape and symmetry of the unit cell structure. Since L. is in continuous space, we leverage the idea of the Denoising Diffusion Probabilistic Model (DDPM) for diffusion on L. In specific, given input lattice matrix Lo p(L). the forward diffusion process iteratively diffuses it over T timesteps to noisy lattice matrix Lr through a transition probability LoL at each 4th step, which can be derived as follows:
(L: Lo) N L1 VorLo, (1 -a))I
where, a = Tik at, = = - 3r and {3e E (0, controls the variance of diffusion step following certain variance scheduler. By reparameterization, we can rewrite equation as:
L. VarLa + vI à,
(10)
where, e' is noise, sampled from . (0 I). added with original input sample Lo at tth step to generate L. After T such diffusion steps, noisy lattice matrix Lr generated from prior noise distribution NNO I). In the reverse denoising process, given noisy LT ~ (0, I) we reconstruct true lattice
structure Lo thorough iterative denoising step via learning reverse conditional distribution, which we
formulate as follows:
p(L-1IM Cp) - (M, Cp). (1 02-1)
(11)
(1 ā.)
where (M, - (L. يقتد e" (M,, Cp. t)). Intuitively, è' is the denoising term that Tーm
needs to subtracted from L, to generate L, and textual representation Cp will steer this reverse diffusion process. We text-guided denoising network & (A,, Xt. Lt, L, Cp) to model the noise
19

---

Published as a conference paper at ICLR 2025
term e' t). Following the simplified training objective proposed by (Ho al. 2020
train the aforementioned denoising network using (2 loss between et and
Llattice (u(,))((
EL112
(12)
E.2 DIFFUSION ON ATOM TYPES (A)
2021) consider Atom Type Matrix A as the logits/rrobability distribution for k classes E R x (continuous variable in real space) and apply DDPM to learn the distribution. However for discrete data these models are inappropriate and produce suboptimal results (Austin et al.] 2021 Campbell et al.) [2022] Hoogeboom et al., 2021). Hence we consider A as N classes and leverage discrete denoising diffusion probabilistic model (D3PM) (Austin et al.) 202T) for diffusion on A. In specific, denoting row vector a as a one-hot representation of an atom a, we can write transition probability for forward process as:
Prior studies (Jiao al. 2023 Xie et al.
discrete variables belonging
q Cat(ar;P al-1Q.)
(13)
where (a:pp: a categorical distribution over the one-hot row vector a with probabilities given by the row vector p and Q, the Markov transition matrix at time step t defined [2ik.. = g(a;
i a j). Different choices of Q, and corresponding stationary distributions are proposed by (Austin et 202T) which provides flexibility to control the data corruption and denoising process.
We adopted the absorbing state diffusion process, introducing new absorbing state [MASK] in Q.
At each time step t, we can formally define the transition matrix as:
1,
if j = [MASK].
3,,
if i = j + [MASK] [MASK].
(14)
ifi=j
Intuitively, at each time step t, atom either stays in its type state with probability 1 Br or moves to [MASK] state with probability 3, and once it moves to [MASK] state, it stays in that state. Hence, the stationary distribution of this diffusion process has all the mass on the [MASK] state. During reverse denoising process, given textual representation Cp. we first sample noisy ar and obtain do thorough iterative denoising step via learning reverse conditional transition:
pa at. Cp) xc Σ a(a )o (an a,, Cp)
(15)
do We use the text-guided denoising network (Ar. X,, L2, t, Cp) to model this backward denoising process, which is trained using the following loss function as proposed by (Austin et al. 2021)
Ltype Lv B + ALCE
(16)
where Lvn is the variational lower bound loss defined as follows:
T'
Lv = (y((n)
Dkilg(arlao)lllar + > By(arlao) [DKi ilaggaaa-aaaaaa las an) (oo)llee (po(aa-1
LT
Le-1
E Egea lao [log po (ao (a))
La
(17)
and LCE is the cross-entropy loss defined as follows:
T
LCe Eg(ao)
Eg(as (no) |log pe (aola
(18)
and is hyperparameter.
E.3 DIFFUSION ON ATOM COORDINATES (X)
[x1,12 EN. IT E N33 denotes atomic coordinate positions, where I; E R corresponds to coordinates of th atom in the unit cell. We can diffuse the atom coordinates in two ways: either by diffusing cartesian coordinates or fractional coordinates. Prior works like CDVAE (Xie et al. 2021) and SyMat (Luo et al.) (0236) diffuse cartesian coordinates whereas DiffCSP (Jiao et al. 2023) diffuse fractional coordinates. In our setup, as we are jointly learning atom coordinates and lattice matrix simultaneously. we follow the line of work by DiffCSP and diffuse fractional coordinates. Atomic fractional coordinates in crystal material lives in quotient space Nx3 NZNx3 induced by the crystal periodicity. Since the Gaussian distribution used in DDPM unable to model the cyclical and bounded domain of X, it is not suitable to apply DDPM to model X. Hence each step of forward diffusion, we add noise sample from Wrapped Normal
Coordinate Matrix
20

---

Published as a conference paper at ICLR 2025
(WN) distribution De Bortoli et al. 2022) to X and during backward diffusion leverage Score Matching Diffusion Networks (Song & Ermon | 2020) to model underlying transition probability q(x, Xo) Nw((X, Xo. I) In specific, at each tth step of diffusion, we derive X, as X, = Fo (Xo + oFE*) where, ex is a noise, sampled from No I), a is the noise scale following exponential scheduler and (.) is truncation function. Given fractional coordinate matrix X,
truncation function (X) = (X- - [X]) returns the fractional part of each element of X. As argued in (Jiao al., 2023 J. ((X,|XX)
periodic translation equivariant, and approaches uniform distribution (10 1) for sufficiently large values of dr. Hence during the backward denoising process, we first sample Xr ((, 1) and iteratively denoise via score network for T steps to recover back the true fractional coordinates Xo. We use the text-guided denoising network (A,, X,, L,, I, Cp) to model the backward diffusion process, which is trained using the following score-matching objective function:
Looord = Ex lo,o lo (-1((1
e² (M, Cp. 1
(19)
||,, xo 207
where Vx, (ogg(X((Xo) x E exp(- bution and ex (Mes Cp. 1) denoising term.
is the score function of transitional distri-
E.4 TEXT GUIDED DENOISING NETWORK
In this subsection, we will illustrate the detailed architecture of our proposed Text Guided Denoising Network &o (Ar,, Xr, Lr, L, Cp). which we used to denoise A. X and L. As mentioned in [ the learned distribution of material structure pMM) must satisfy periodic E(3) invariance. Hence we leverage an veriodic-E(3)) E(3)-Equivariian Graph Neural Network (GNN) integrated with a pre-trained textual encoder to model the denoising process. In particular, as a text encoder, we adopt a pre-trained MatSciBERT (Gupta et al. 2022 model, which is domain-specific language model for materials science, followed by projection layer. MatSciBERT is effectively a pre-trained SeiBERT model on a scientific text corpus of 3.I7B words, which is further trained on a huge text corpus of materials science containing around 285 M words. We feed textual description of material T and extract embedding of [CLS] token hCLS as a representation of the whole text. Further. we pass hCLS through a projection layer to generate the contextual textual embedding for the material Cp E Rd, which we pass to the equivariant GNN model to guide the denoising process. Practically. as the backbone network for the backward diffusion process, we extend CSPNer architecture (Jiao et al.) (023). originally developed for crystal structure prediction (CSP) task. CSPNet is built upon EGNN (Satorras et al.. 2021), satisfying periodic E(3) invariance condition periodic crystal structure. At the hddh layer message passing, the Equivariant Graph Convolutional Layer (EGCL) takes as input the set of atom embeddings * [ h,, h l atom coordinates x² = . ..., xx and Lattice Matrix L and outputs transformation on h² ++1 Formally, we can define the th layer message passing operation as follows:
maj Pm (h h L L, (x 1}
(20)
h k++1 = h + + ( m;}
(21)
where m; = EE= mij- Pm. Ph are multi-layer perceptrons and vrr is a Fourier Transformation
function applied on relative difference between fractional coordinates x x. Fourier Transformation used since is invariant to periodic translation and extracts various frequencies of all relative fractional distances that are helpful for crystal structure modeling. We fuse textual representation Cp into input atom feature h, as
h { farom (a,) || fpoe C,
(22)
where is the timestamp of the diffusion model, Spos() (.) is sinusoidal positional encoding (Ho et al. 2020 Vaswani et al.) (207).. fatom (.) learned atomic embedding function and || is concatenation operation. Input atom features h² and coordinates R are fed through layers of EGCL to produce 24, p(A.-) M,) and ex as follows:
= N
Σ I:
N
(23)
p(A,-1 1 MM.) = pa ):
ex px (11
21

---

Published as a conference paper at ICLR 2025
where PL.PA, PX multi-layer perceptrons the final layer embeddings. Intuitively, we feed global structural knowledge about the crystal structure into the network by injecting contextual representation Cp into input atom features. This added signal will participate through message- passing operations in Eq. 20Jand guides in denoising atom types, coordinates, and lattice parameters such that it can capture the global erystal geometry and aligned with the input stable structure specified by textual description.
Algorithm 1 Training Algorithm
1: Input: Atom type Matrix Ao (One hot Vector Representation), Coordinate Matrix X Lattice matrix Lo, Markov Transition Matrix 12 Textual Representation Cp. Number of diffusion step T and hyperparameters As, Λx. XL.
2: repeat
3: Sample U0, T)
4:
Sample Noise cx v I(0,))
5:
L. varLo + VI dre
6:
X, (xp + O(c²)
7:
A, Cat(Ai = 11-
8:
ELEEN,A A Do X,, Log t, Cp)
9:
Llastice = l 212
10:
Lcoord Vx, lo, - Ex112
Ltype LvB + XLCE
12:
Minimize L Alattice + As Ltypee + Ax Lcoord and update parameters of tb
until converged
Algorithm 2 Sampling Algorithm
1: Sample Lr (0, I), ~ (0)
2: Randomly sample each atom type between 0 to 99 (Max possible atom type) and form Ar
3: Cp f Textual Representation
4: for ( Too do
5:
* ,,,,x, ee NOO, T)/ Sample
6:
A, t (A,, X,, Lt, Cp)
7:
L + var
_ (L vi-an
a c + v
Be 1-ac-1
1-a;
8:
A Softmax Á + a)
9:
X, X 1 + w(, + (a? - 1
oe-1
o2-1 02-1vo-- o
X Po X,_ L Cp)
11:
Th - step_size * de-1
12:
X, t- wi (X, niX 2
end for
E.5 TRAINING AND SAMPLING
TGDMat is trained using the following combined loss:
X1,Lattice + A + Ax Leoord
(24)
where Llattices Leype and Looord are lattice l2 loss (Eq. 12 type cross-entropy loss (Eq. 16 and coordinate score matching loss (Eq. 19) respectively and ÄL. AA, Ax are hyperparameters control the relative weightage between these different loss components. During training, freeze the MatSciBERT parameters and do not tune further. During sampling, we use the Predictor-Corrector sampling mechanism to sample Ag. X₂ and Lg. Next explain algorithms for training and sampling.
22

---

Published as a conference paper at ICLR 2025
Shat
Geneisic
(a) Random Material Generation (Gen) Task
(b) Crystal Structure Prediction (CSP) Task
Figure 5
F EXPERIMENTS
F.1 EXPERIMENTAL SETUP
Benchmark Tasks. We evaluate our proposed model TGDMat on two different categories of tasks for material generation, Random Material Generation (Gen) and Crystal Structure Prediction (CSP). In Gen task, the goal of the generative model is to generate novel stable materials (atom types, fractional coordinates, and lattice structure). In CSP task, atom types of the materials are given and the goal is to predict/match the crystal structure (atom coordinates and lattice). In TGDMat model, by design choice, we use the textual description crystal materials during each step of the reverse diffusion process to enhance the generation capability in both tasks. A pictorial illustration of both tasks is provided at
Dataset. Following Xie et al Xie et al. 2021) we evaluate our model on three baseline datasets: Perov-5, Carbon-24 and MP-20. Perov-5 (Castelli al.] 20T2a b)) dataset consists of 18,928 perovskite materials, each with 5 atoms in a cell. They generally can be denoted by
ABX3 indicating the three different types of atoms usually observed such materials. Carbon-24
(Pickard.] 2020) dataset has 10,153 materials with 6 to 24 atoms of carbon in the crystal lattice. Finally, MP-20 (Jain al. 2013b) dataset has 45,231 materials curated from the Materials Project library (Jain et al. 3a), where each material has at most 20 atoms in the lattice. Crystals from Perov-5 dataset share same structure but differ composition, whereas Crystals from Carbon-24 share the same composition but differ in structure. Crystals from MP-20 differs in both structure and composition. We curated textual data for these datasets with a textual description of each material. Specifically, we generate both long detailed textual descriptions and shorter prompts using approaches mentioned in Appendix[D [
The structures in all three datasets derived from quantum mechanical simulations and are all at local energy minima. Most materials in Perov-5 and Carbon-24 are hypothetical, whereas MP-20 represents realistic dataset that includes many experimentally known inorganic materials, each with a maximum of 20 atoms in the unit cell, most of which are globally stable. A model that performs well on MP-20 could potentially generate novel materials that can be synthesized experimentally. While training TGDMat, we split the datasets into the train, test, and validation sets following the convention of 60:20:20 as done by Xie (Xie et 2021).
Parpme-arraeters Details. our TGDMat model, we adopted 4 layers CSPNet as message passing layer with hidden dimension set as 512. Further, we use pre-trained MatSciBERT (Gupta et al. 2022 followed by two-layer projection layer (projection dimension 64) as the text encoder module. We keep the dimension of time embedding at each diffusion timestep 64. We train it for 500 epochs using the same optimizer, and learning rate scheduler as DiffCSP and keep the batch size as 512. We perform all the experiments in the Tesla P100-PCIE-16GBB GPU server.
23

---

Published as a conference paper at ICLR 2025
F.2 EVALUATION METRICS
Random Material Generation (Gen) Task. Following CDVAE (Xie et al. 2021 evaluate the performance of TGDMat and baseline models on generating novel material structure using seven metrics under three broad categories: Validity, Coverage, and Property Statistics. Under Validity, following the prior line of work (Court et al. 2020; Xie et al. (202]), we measure structural and compositional validity, representing the percentages of generated crystals with valid periodic structures and atom types, respectively. A structure is valid as long as the shortest distance between any pair of atoms is larger than 0.5 Å whereas the composition is valid if the overall charge is neutral as computed by SMACT (Davies et al.) 2019). In Coverage, we consider two coverage metrics, COV-R (Recall) and COV-P (Precision). COV-R measures the percentage the test set materials being correctly predicted, whereas COV-P measures the percentage of generated materials that cover at least one the test set materials. (More detailed discussions can be found in (Xie et al., (221)) and (Ganea et al. )02I)). Finally, we evaluate the similarity between the generated materials and those in the test set using various Property Statistics, where we compute the earth mover's distance (EMD) between the distributions in element number (# Elem), density (p. unit g/cm3), and formation energy (E, unit eV/atom) predicted by a GNN model.
Crystal Structure Prediction (CSP) Task. We evaluate the performance of TGDMat and baseline models on stable structure prediction using standard metrics proposed by the prior works (Jiao et al. 2023: Xie et al. 202ID, by matching the generated structure and the input ground truth structure the test set. Specific, for each material structure in the test set, we generate samples given the textual description and then identify the matching if at least one of the samples matches the ground truth structure. We calculate the Match Rate and RMSE metrics using the Structure:Matcher class in Pymatgen, which identifies the best match between two structures while accounting for all material invariances. Match rate indicates the percentage of the matched structures over the test set satisfying thresholds stol=0.5, angle.tol=10, Itol=0.3. RMSE is computed between the ground truth and the best-matching candidate, normalized by R/V/N where V is the volume of the lattice, and averaged over the matched structures. For baselines and TGDMat, we evaluate using k = and k 20.
F.3 COMPLETE AND DETAILED RESULTS
In this subsection, we provide full comprehensive results on both Gen and CSP tasks across three benchmark datasets and evaluate the performance of all the baseline models, their text- rext-guided variants (both short and long), and our proposed TGDMat(Long) & (SDMMt(Short). We report the CSP and Gen task results in Table and respectively.
Following the Insights or Observations:
For both tasks, across all the datasets, text guidance outperforms the vanilla diffusion models in almost all metrics.
Our experiments suggest that using shorter prompts text-guided models outperforms the vanilla bascline models. However, performance is even superior when using text- text-guided diffusion using longer prompts.
For the CSP task, using text guidance during the reverse denoising process, with just one generated sample per test material, text-guided variants outperform respective vanilla models, thereby reducing computational overhead.
Our proposed TGDMat (Long) stands out as the leading model when compared to all baseline models and their text-guided variants across three benchmark datasets. In specific, for Gen Task, TGDMat (Long) outperforms the closest bascline DiffCSS+ (Long) because we leveraged discrete diffusion atom types, which is more powerful in learning discrete variables like atom types.
Finally, results indicate that utilizing shorter prompts TGDMat (Short) results in slight de- crease in overall performance compared to the longer variant TGDMat (Long). Nonetheless, the performance remains superior comparable to baseline models (vannila and text-guided variants).
24

---

Published as a conference paper at ICLR 2025
Perov-5
Carbon-24
MP-20
Method
# samples
Match
RMSE
Match
RMSE
Match
RMSE
45.31
0.1138
17.09
0.2969
33.9
0.1045
CDVAE
20
88.51
0.0464
88.37
0.2286
66.95
0. 0016
1
48.97
0.1063
22.65
0.264
40.33
0.1037
CsVAE+(short)
20
89.54
0.0423
89.61
0,2188
70.22
0.0876
1
49.25
0.1055
23.73
0.259
41.8
0.1021
CDVAE+(long)
20
89.73
0.0417
89.77
0.2053
72.56
0.084
1
47.32
0.1074
20.81
0.2655
33.92
0.1039
SyMat
20
90.25
0.0316
89.29
0.2184
71.03
0.0945
1
49.39
0,0985
23.71
0.2567
40.84
0.1027
SyMat+(short)
20
92.1
0.0255
90.86
0.2069
71.31
0.0875
50.88
0.0963
28.18
0.251
43.17
0.1016
(lMgt+(long)
20
92.3
0.0201
91.65
0.187
72.96
0.082
1
52.02
0.076
17.54
0.2759
51.49
0.0631
DiffCSP
20
98.6
0.0128
88.47
0.2192
77.93
0.0492
1
56.54
0.0583
24.13
0.2424
52.22
0.0597
DifiCSP+(short)
20
98.25
0.0137
88.28
0.2252
80.97
0.0443
90.46
0,0203
44.63
0,2266
55.15
0.0572
DifiCSP+(long)
20
98.59
0.0072
95.27
0.1534
82.02
0.0391
1
56.54
0.0583
24.13
0,2424
52.22
0.0597
TGDMat (short
20
98.25
0.0137
88.28
0,2252
80.97
0.0443
1
90.46
0.0203
44.63
0.2266
55.15
0.0572
TGDMat (long)
20
98.59
0.0072
95.27
0.1534
82.02
0.0391
Table Summary the Complete and Detailed Results on the CSP Task.
25

---

Published as a conference paper at ICLR 2025
Validity
Coverage
Property
Dataset
Method
Comp
Struct
Cov-R
Cov-P
# element
Density
form_energy
CDVAE
98.29
100
99.25
98.39
0.0731
0.1462
0,0291
CDVAE+ (Short)
98.17
100
99.4
99.01
0.0706
0.1395
0.0246
CDVAE+ (Long)
98.45
100
99.53
99.09
0.0609
0.1276
0.0223
SyMat
96.83
100
99. 16
98.29
0.0193
0.1991
0.2827
Perov 5
SyMat+ (Short)
96.94
100
99.22
98.4
0.0192
0.1827
0.2633
SyMat+ (Long)
97.88
100
99.71
98.79
0.0172
0.1755
0.2566
DiffCSP
98.15
100
99.28
98.08
0.0132
0.1281
0.0267
DifICSP+ (Short)
98.21
100
99.61
98.39
0.0123
0.1193
0.0266
DiftCSP+ (Long)
98.44
100
99.85
98.53
0.0119
0.1071
0.0241
TGDMat(Short)
98.28
100
99.71
99.24
0.0108
0.0947
0.0237
TGDMat(Long)
98.63
100
99.87
99.52
0.009
0.0497
0.0187
CDVAE
100
99.35
82.66
0.1539
0.2889
CDVAE+ (Short)
100
99.34
82.96
0.1398
0.2804
CDVAE+ (Long)
100
99.82
84.76
0.1377
0.266
SyMat
100
99.42
97.17
0.1234
3.9628
Carbon 24
SyMat+ (Short)
100
99.52
97.2
0.1206
3.7422
SyMat+ (Long)
100
99.9
97.63
0.1171
3.862
DifCCSP
99.9
99.49
97.27
0.0861
0.0876
DifICSP+ (Short)
100
99.61
97.29
0.0811
0.087
DifICSP+ (Long)
100
99.93
97.33
0.0763
0.0853
TGDrtt(Short)
100
99.81
91.77
0.0681
0.0865
TGDMat(Long)
100
99.91
92.43
0.0436
0.0632
CDVAE
86.3
100
99.15
99,49
1.4921
0.7085
0.3039
CDVAE+ (Short)
87.05
100
99.36
99.6
0.993
0.642
0.297
CDVAE+ (Long)
87.42
100
99.57
99.81
0.972
0.6388
0.2977
SyMat
87.96
99.9
98.3
99.37
0.5236
0.4012
0.3877
MP 20
SyMat+ (Short)
88.08
99.9
98.59
99.47
0.5031
0.3917
0.3622
SyMat+ (Long)
88.47
99.9
99.01
99.95
0.4865
0.3879
0.3489
DiffCSP
83.25
100
99.41
99.76
0.3411
0.3802
0.1497
DifiCSP+ (Short)
84.57
100
99.52
99.85
0.331
0.38
0.1379
DiffCSP+ (Long)
85.07
100
99.81
99.89
0.3122
0.3799
0.1355
TGDMat(Short)
86.6
100
99.79
99.88
0.3337
0.3296
0.1189
TGDMat(LLng)
92.97
100
99.89
99.95
0.289
0.3082
0.1154
Table 7: Summary of the Complete and Detailed Results on the Gen Task.
26

---

Published as a conference paper at ICLR 2025
F.4 CORRECTNESS OF GENERATED MATERIALS
Setup. In this section, we investigate whether the generated material matches different features specified by the textual prompts. TGDMat has the capability process textual prompts given by the user, enabling it to manage global attributes about crystal materials such as Formula, Space group, Crystal System, and different property values like formation energy, band-gap, etc. To ensure the fidelity of our model's outputs concerning these specified global attributes from the text prompt, We randomly generated 1000 materials (sampled from all three Datasets) based on their respective textual descriptions(both Long and Short) and assessed the percentage of generated materials that matched the global features outlined in the text prompt. In specific, we matched the Formula, Space group, and Crystal System, and Dimensions of generated materials with the textual descriptions. Moreover, we examined whether properties such as formation energy and bandgap matched the specified criteria as per the text prompt (positive/negative, zero/nonzero).
Results and Discussions. We report the results in Table g In general, using longer text, considering Perov-5 and Carbon-24 datasets, the generated material meets the specified criteria effectively. However, when dealing with the MP-20 dataset, which is more intricate due to its complex structure and composition, performance tends to decline. Additionally, when using shorter prompts, overall performance suffers across all datasets compared to longer text inputs. This is because the longer text, provided by the roboerystallographer, offers comprehensive range of information, both global and local, thereby enhancing the generation capabilities of TGDMat.
Method
Global Features in Text Prompt
% of Matched Materials
Perov-5
Carbon-24
MP-20
Formula
97.50
98.20
70.54
Space Group
87.00
80.79
67.88
TGDga((Long)
Crystal System
92.60
91.55
73.54
Formation Energy
95.49
92.88
Band Gap
98.61
96.73
Formula
90.70
92.56
65.22
Space Group
86.51
80,50
58.77
TGDMat(Long)
Crystal System
83.19 90.33
81.64
72.77
Formation Energy
91.00
Band Gap
95.90
93.33
Table 8: Summary of results on % of generated materials matching different global features specified by the textual prompts.
F.5 PERFORMANCE ON MORE SHORTER PROMPTS
In this section, we explore the generalizability and robustness of our model by examining potential variability in text description lengths. The goal of this paper is, given the text prompt. to generate specific material, not any generic or class of materials. Hence some minimum essential information about the crystal, like formula, space group, crystal system, property value, must be given input to the pre-trained model. However, to investigate the robustness of our proposed TGDMat model with more custom and shorter prompts, we did an experiment where we evaluated TGDMat (trained with full text) with even shorter custom prompts with very little information as follows:
Specifying only Formula: "The chemical formula is GaSiSO2. The elements are Ga, Si, S, O. Generate the material."
Specifying only Space Group Info:
"The spacegroup number is 1. Generate the material."
Specitying only Property Info: "The formation energy per atom is positive, Generate the material."
We report the results in table 9 We observe that though TGDMat can handle more custom prompts, but it affects the quality of generated materials, Hence we conclude some minimum essential information about the crystal must be given as input to TGDMat to generate high quality crystal materials.
27

---

Published as a conference paper at ICLR 2025
Perov-5
Carbon-24
MP-20
Text Encoder
Co
Struct (%6)
Comp(%))
Struct (98)
Comp(56)
Ouly Formula
97.06
99.19
98.76
86.16
96.01
Ouly Space Group
85.91
98.97
95.39
84,22
96.88
Ouly Pruperty
86.53
Full Text
98.28
100
100
86.60
100
Table 9: Summary of results on generated materials using more custom/shorter Prompt.
Table 10: Ablation study results on different choices of Text Encoders.
Text Encoder
Perov-5
Carbon 24
MP-20
MR
RMSE
MR
RMSE
RMSE 4
BERT MaISciBERT
96.64
0.0109
72.21
0,2679
79.53 82.02
0.057 0.039
98.63
0,0072
95.27
0.1534
Comp 1
Struct
Comp
Struct
Comp1
Struct
BERT MatSciBERT
98.63
100
100
92.97
100
F.6 ABLATION STUDY CHOICE OF TEXT ENCODER
Further, we investigate the expressiveness of textual representation during the reverse diffusion process. In particular, we are interested in understanding whether there are any benefits gaining from using a domain-specific pre-trained text encoder MatSciBERT. We conduct an ablation study where substitute MatSeiBERT with pre-trained BERT (Devlin al. 2018) model (which is domain agnostic) as text encoder in TGDMat and evaluate the performance on both tasks. The results presented in Table 10 demonstratee that MaTSciBERT surpasses BERT (Devlin et al. 2018) in performance for both tasks. This highlights the richer expressiveness of contextual representation achieved through the use of Nomain-sppccife pre-trained language model.
F.7 MORE VISUALIZATION ON PEROV-5, CARBON-24 AND MP-20
28

---

Published as a conference paper at ICLR 2025
Detailed Desemiption
Short Prompe
Ground truth]
Generated Samples
orthrrhombic group. Y is bonded in distorted square co-planar ge-
YCaS02 crystallizes
space
ometry two equivalent S, two equivalent O, and two equivalent O atoms. Hoth Y-S hand lengths
2.74
Both Y-O bond keglls are 2.24 There is one shorler (2,09 A) and one longer (2:39 À) Y-O bond longth, Co is bonded in distorted sec-saw-like like geometry
Below is description a halk material chemical formula YCoS02, The
equivalent
bond lengths are 2.31 A. Both Co- bond lengths are 2,40 A, S honded in a d-coondinate geometry to two equivalent atoms. Both hond lengths 2,30 There are
equivalent atoms.
Co-S
clements are Y. Co, O. The formation energy per atom positive. The space- group number is 24. The crystul system onhorhommic. Generate the material:
equivalent
equivalent
two incquivalent O sites. In the lins O site, is beaded in rectangular saw-likwe geometry rwo equivalent Co atoms. in second site, O is bonded a distorted square co- colppaanz geome- two equivalent Y and two equivalent S stoms.
equiva-
ScMoN20 (Cubic) Perovskite-oe derived structured and
crystallioes in the tetragonal Pimm space group. Sc
is honded to four equivalent N and two equivalent O
ScN402 octahedra shore with six equivalent SaN402 octahedra
faces
eight oquivalent MoN804 cuboctabedra. The corner-
sharing octahedral tilt angles range from 0-17. All
Sc-N lengths
A. (2.00 and one longer (2.01 A) Se-O bond length
Below description of bulk rial. chemical formula ScMaN20 The sloments are St, Mo. N. O The for- mation energy per
Mo bonded to dight equivalent N and four equivalent
O atonns form MuN804 cuboccaahodra comers with twelive equivalen McN804 cuboctahodaa
share
positive. The spacegroup number 98. crystal system is tetragonal. Generate the mate- rial:
faces with six equivalent McNSC4 cuboctaledra, and
faces with eight equivalent ScN402 octahedra. There four shoeter (2.83 A) N bond lengths. All Mo-O boad lengths are 2,83 N is bonded in liocar geometry two oquivalent Sc and four equivalent
longer (2.84
bonded in lincar
geometry to two equivalent So and four equivalent Mo atoms. The formation energy atom is 1.893
SaNO2Ga alpha Rhenium reriive-derived structured and crystallizes The structre consists of one Go cluster insid ScNO2 framework. In the Ga cluster, Ga boaded coordinate geometry
othorthombic
space group
atoms. work, Sc is bonded to two equivalent N, two equivalent Below is a description of a bulk mate- O, and two equivalent O atoms frrm comer-shrring rial. The chemical formula ScGaNN2
frame-
ScN204 octabedra. The corrersharrnn actahedral till The elements are Sc,
O. The for
angles range from D-1", Both Se-N bond lengths are mation energy per atom positive, The 2,09 Å. Both Sc-O bond lengths are 2.09 A. Both Sc
Sc-spacerouup number is 24, The crystul lincad system orthorhombie. Generate the
O bond longthe are 2,09 A. N is bonded atoms. There two material:
geometry two equivalent
inequivalent O sites, In the first O site, O is honded in linear goometry to two equivalent Se atoms. the secoed
bonded in linear genmetry oquivalent So stoms. The formation energy per stom is 1,796..
OsAuO3 is (Cubic) Perovakite structured and crystal-
space group. bonded to six aquivakent O skonts to form Os06 octabodra that share coreer with six equivalent Ox06 octabedia and with cight oquivalent AuO12 cuboctahedra. corner sharing octuhedra are not tilted. Os- bond
description bulk material The chemical formula is OsAuO3, The lengths are 1.97 A. Au bonded to twelvv equivalent elements are Os. Au, O. The formation O atoms to form discorted AuQ12 cuboctahedra that energy per
1.4248. The space- share comers with twelve equivalent As012 pubocta-group number is 220, The crystal system hedra, faces with six equivalent An012 cuboctabedra.) is cubic. Generate the material. and faces with eight equivalent Os06 octahedra. All lengths are 2.79 A. geometry to two equivalent Os and four equivalent An atoms. The foraation energy per atom is 1.4248.
in linear
Table 11: Visualization of the generated structures given textual description for Perov-5 dataset
29

---

Published as a conference paper at ICLR 2025
Derailed Description
Shart Prompe
Groond truth]
Genented Samples
C crystallizes in the trielinic Pl space group. There
twenty insquivalent C sites. In the first C site Below a description hulk material C(l)
C. The elements lent (19) atoms form comnersharing CC4 tetrahedra are C. The energy per atom is negative .wo equivalent C(1) atous form misture of dis- spucegroup number is The crystal torted corner and edge sharing CC4 trigonal pyramids. system triclinic. Generate material The energy per atom is 1336.
C(18), one C(5), and equiva
chemical
C crystallizes in the orthcchomiic Cmem space group There
incquivalent
lirst C
C()) honded one C(2) three equivalent C())
Below description a bulk material
atoms to forn mixture of corner and edge-sharing
CC4 gonal pyramiés. The C(1)-C(2) bond length
The chemical formula is C. The elements
1.19
two sheeter
one longer
energy per The spacegroup number The crys-
nogative
(1.56 A) (CI) band length. In the second site.
((2) bonded one C(T) and three equivalent (12) lul system orthochombic. Gencrate the
form cornen- shuring CC4 tetmheemm There material.
two shorter (1.5 1.54 A] and one longer (1.5 Áj ((2)-C12)
bond length. The energy atom is 154.2425.
C crystallizes the trielinic space group. There dix inequivalent sites. is bonded to one C(3), one CI5), and two equivalent
first site, C(D) Below a description a bulk material
The chemical formula is C. The elements energy per C(4), and two equivalent C(3) zsoms to form distorted squcegropp number is corner sharing CC4 tetrahedrn The energy atom is system is trielini.. Generate the material 1338.138.
C(d) to form commersharing CC4 letrahedra.
bonded
C(2), one
negative
crystal
Cis Theorctical Carbon Structureliike structure and crystallizes in the trielinic P-1 group. There and nine inequivalent tites. In the first site. C(I)
bonded C(S), one C(6), C(7), and atom to form a mixrure of corner and edge-sharing
Helow description hulk material The chemical formula is C. The elements
CC4 tetraledrss The C(I)C(() bond length 1.51
are C. The energy stom is 154.2197.
The spacegroup number is 1. The crystal system triclinic. Generate mate- rial.
The C(T)C((6 bond longth is 1.54 A.... In the ninth C site. C19) is
length 1,56 Thee (11)-C21)
bonded to one C(4), one C(6), one C(T), and one C(8)
form mixture comerr and edge-sharing CC4
tetrahedm. energy per stom 1197.1197.
Table 12: Visualization of the generated structures given textual description for Carbon-24 dataset
30

---

Published as a conference paper at ICLR 2025
Detailed Description
Short Promps
Ground truth]
Generated Samples
Ea2PCI is Casweliiltvrite-iikk like structured and crystal
liees the letragoral 14/mmmm space group. There
deseripiion hulk material The chemical formala is Ea2PCI. The el
two inqquivalent Hu sites.
Hu site, Hu(1)
bonded two oquivalent P(3) and four equivalont Cl(l)
form EuP2CI octahedra that share with equixalent JP2CIA octahedmu, adges
ements are Fa, P. and C1 The formation energy per atom -1.7615.
band gap 0.O. The energy above the con- yex hull is 0.0. The spacegroup number is 122. The crystal system tetragonal per alom is -1 -1.7615. The band gap is acro. The energy Generate material. above the corvex bull is zero.
four equivalent Eut [ IP2C44 octaledmm and edges with
eight equivalent BaC3)M4C12 octahedra. The comer- not tilted. formation energy
sharing octaledra
MgNdHg2 is Heuster structured and crystallizes
cubic. Fm-3m space group. Mg(l) honded
Below is description a bulk material. The chemical formula is NdMggHg2. The
body-centered cubic geomeity to eight equivalent
Hg(l) atoms. All Mg(1)-Hg( bond lengths 3.18
No(() bonded bodyesentered try to eight equivalent Hg( stoms. All Nd(I>-Hg())
goome-
elements Nd, Mg. Hg. The for- mation energy per atom 40,4708.
hond lengths are 3.18 A. Hg(1) is honded in hody.
band gap is 0.O. The energy above the hull is 0.O. The spacegroup
cubic geometry four equivalent Nd(1) atoms. The formation energy per
equivalent Mg(1)
crystal system cubic
Generate the moterial.
atom 0.4708. The band pap is 0.0. The energy above the convex bull 0.0. The spacegroup number 224.
MgNdTI crystallizes
bexagonal P.62m space
group. Mg( 1) is honded in 4-coorinate geometry to
two equivalent TI() and two equivalent TH(2) atoms. leggits
M-(I)-TI(
3.01 A My(l)-
[Bllow is description a bulk material S-coordinate geometry to one TI(2) and four equivalent The chemical formula is NdMgTL The atoms. The Nd(1)-T1(2) lengths are length 3.31 clements
TH(2) bond lengihs are 3.03 A. Nd(1) is bonded in
The two energy per atom 355.,
Mg.
(d(1)-TI(1)
inoquivalent Tl site. In the lirst II site, TI(2) is bonded pap is 0.0. The energy above the com- in distorted
six equivalent My(1) and hull is 0.0. The spacegroup number equivalent N)(1) atoms. In second sit 188. The crystal systemi bexagonal. TI(I) is bonded in 9-coordinace geometry to three Generate the material. equivalent Mg(1) and six equivalen Nd(1) atoms. The formation energy per is -0.355. The hand gap is O.D. The energy above the convex hull is 0.0. The spacegroup number is 188.
LaNi2Ge2 crystalliees in the tetragonal 14mmm space) group. La( bonded eight equnvaent Ni(1) and eight equivalient Ge(1) nioms (ellow is a description of a bulk male- All La( Laii)-Nic)) 1) bond longths 3,25 A. All La(T-rial. The chemical formula is LaiNiGei2 elements La, 4-coordinate geometry to four equivalens La(l) and four formation energy per atom is -0.69) equivalent Ge(1) atoms. All Nir ()-Ge() bond lengths The band gap is 0.0. The energy above 2.39 A. Ge(l) bonded 9-courdinate geometry convex hull 0.0. The spacegroup to four equivalent lal1) equivalent Ni(1), Go(l) stom. The Ge()))Ge() bond length is 2.66 letragonal. Generate the material.. The formation energy atom is 41.691. The hand gap
geometry
Ge(1) bond lengths are 3,26 Ni(1)
in
number
crystal system
is energy above convex O.O.
Table 13: Visualization of the generated structures given textual description for MP-20 dataset
31


Reviews:
[{"review_id": "C5d9FeQJvV", "reviewer": "Reviewer_cwRr", "summary": "This paper presents TGDMat, a novel text-guided diffusion model for generating periodic crystal materials. TDGMat jointly models the generation of atom types, coordinates, and lattices of materials using separate diffusion processes. Specifically, it uses a Denoising Diffusion Probabilistic Model for lattice modeling, and uses a discrete diffusion model D3PM for modeling atom types. As for atom coordinate modeling, the authors mainly follow DiffCSP and use a score matching objective. Note that, they apply diffusion on fractional coordinates, instead of Cartisian coordinates, which cannot reflect the periodicity of crystal materials. To ensure geometric symmetry, TGDMat uses the CSPNet proposed by DiffCSP, which ensures periodic E(3) invariance for periodic crystals.\n\nFor the text-guided component, material descriptions are generated using Robocrystallographer software. Text embeddings, produced by a language model, are then concatenated with node embeddings to guide material generation.", "questions": "1. How large is the proposed model and the compared baselines for material generation?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The overall performance is good, as shown in Table 3. TGDMat shows significant improvement on the random material generation task compared to previous methods. Additionally, the authors demonstrate the importance of incorporating text-guidance by showing the improved performance of baselines in Table 1 and Table 2.\n2. The method is overall sound and well-engineered. The authors have employed state-of-the-art diffusion methods for each modality associated in the whole material generation process. The source code is attached as a supplementary material. \n3. The textual annotation for the material dataset is a nice additional contribution, and is potentially impactful. It can stimulate future research for joint text and material modeling.", "weaknesses": "1. It seems that a large proportion of the methodology is borrowed from the previous work DiffCSP. This includes the diffusion process for atom coordinates, the diffusion for lattice, and the GNN backbone. \n2. The authors are suggested to use the \\citep command instead of \\cite for citations to improve the readability. \n3. The proposed method has achieved significant performance on the employed evaluation metrics, like validity and coverage. Does this mean the model can be readily employed for practical material discovery in industry? If yes, can you include further discussion on this application? If not, what is a barrier? Are there any other evaluation metrics that should be measured, like the novelty of the generated material compared to the training set, before application?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "96CnUfwrtK", "reviewer": "Reviewer_D75u", "summary": "The paper introduces a text-guided diffusion model for generating 3D periodic materials. The work leverages a periodic-E(3)-equivariant graph neural network (GNN) to jointly generate atom types, fractional coordinates, and lattice structures, while integrating textual descriptions at each denoising step.", "questions": "1. Could the authors provide more details on how the contextual representation of long, detailed text data is integrated into the denoising network? What specific techniques or architectures are used to achieve this?\n2. Could the authors include an ablation study comparing long and short text prompts on the baselines in Table 1 or 2? How would the results differ, and what insights could be gained from such a study?\n3. Can the authors provide more detailed visualizations of the generated materials and explicitly discuss how well these results align with the given text prompts? Are there any metrics or qualitative assessments to evaluate this alignment?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The model effectively learns the joint distribution of atom coordinates, types, and lattice structure through an end-to-end diffusion network, which is a significant improvement over existing models that handle these aspects separately.\n2. Incorporating textual descriptions as a condition during the denoising process enhances the model's ability to generate materials that meet specific user-defined criteria, making it more versatile and user-friendly.", "weaknesses": "1. The paper does not provide sufficient detail on how the contextual representation of long, detailed text data is fused into the denoising network to generate text-guided variants of baseline models. \n2. The visualization of de novo generated materials lacks a thorough discussion on whether the results align with the given prompts. Additionally, the alignment of generated materials with general textual conditions is not adequately discussed, which is crucial for validating the model's performance.\n3. There is a need for a more comprehensive ablation study comparing the effects of long, detailed descriptions versus short prompts on the model's guidance performance. This would help understand the robustness and versatility of the model under different input conditions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6RWnOJaqKL", "reviewer": "Reviewer_ijYE", "summary": "The paper focuses on a text-guided diffusion model for periodic material generation. It first designs a diffusion model that can jointly model atom types, coordinates, and lattice structures for periodic materials, then proposes incorporating material structures and properties through text to enhance generation performance. In the experiments, it demonstrates improvements by incorporating text into existing diffusion models, such as SyMat and DiffCSP, followed by showing the superiority of the proposed TGDMat over existing baselines.", "questions": "- In line 48, to address issues in local message passing, how about using Transformers?\n- I am not sure if I missed anything, Figure 2 shows ‘SE(3)-Equivariant GNN model’.. Also, are the diffusion and reverse processes in Figure 2 are correct?\n- The paper conducts joint diffusion on lattices, atom types, and coordinates, is there any ablation study for the separately and jointly learning of crystal geometry?\n- The CDVAE employs SE(3) equivariant GNNs adapted with periodicity to ensure the invariance of materials. Why not use it and what’s the difference between SE(3) and periodic E(3)-equivariant denoising model backbone?\n- Noticing there would be several generated samples in Table 4, how would one select the best sample if the ground truth was not available?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- This paper proposes the joint learning fashion for three types of crystal structural information. Text guided denoising network can be the right way to the solution of the problem.\n- The paper is well-organized, self-contained with a comprehensive and detailed reference section.\n- The design of textual description in the reverse diffusion process is convincing according to the supplementary material.\n- The paper provides the computational advantage of integrating text knowledge during reverse diffusion compared to other baseline models.", "weaknesses": "- Motivation should be clarified: (a) Providing guidance in diffusion models for improved generation is important, but it is unclear why text must be used to incorporate such guidance. The guidance information in line 70 could also be modeled as feature vectors, reducing the need for text embedding models, which may not always provide accurate text-based embeddings. Additionally, text-based models may struggle to capture subtle numerical differences. Is there any prior work on using feature vectors for guidance? What are the advantages of using text rather than directly using numerical values or feature vectors? (b) The idea of studying joint diffusion over all three components of crystal structures: the lattices, atom types, and coordinates, by employing a periodic-E(3)-equivariant denoising model, kind of the same as the DiffCSP. Though exploring both crystal structure prediction (CSP) and random generation task (Gen), the text guided denoising network seems to be straightforward by further considering the CSP model architecture. It is not clear how much of this part of work is based on previous research and what is new. \n\n- Method design should have rationales: The authors applied the diffusion process independently to three variables: lattice (line 210), atom types (line 248), and atom coordinates (line 272). However, in the reverse process, the denoising models use all variables with conditions for noise estimation (line 289). Given that lattice noise is independent of atom types/coordinates and vice versa, is there a reason for not simply using the lattice variable (plus $C_p$) for lattice denoising?\n\n- Experiments should be improved: The paper lacks an evaluation of whether the generated materials meet the text descriptions. Coverage may reflect this to some extent, but it is too general and lacks details on specific properties mentioned in the descriptions. Property statistics are at the distribution level between two sets and lack point-to-point comparisons. Also, the paper lacks the ablation study for the denoising model to demonstrate the individual importance of each component (or the authors could highlight the part if I missed it).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Q1Lhd9UISI", "reviewer": "Reviewer_mJeo", "summary": "This paper presents a text-guided diffusion model for periodic material generation. Compared to the baseline methods, the proposed text-guided approach achieves more valid and stable performance on benchmark datasets.", "questions": "See Weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Integrating text information to guide crystal structure generation is a novel practice in the field of material structure generation.\n2. From the experimental results, the method proposed in this paper achieves better performance compared to baseline methods. Additionally, the proposed method also demonstrates improved generation efficiency.", "weaknesses": "1. Although combining text information in the field of material generation is an innovative approach, the text-guided diffusion model is not a new model framework. Therefore, the contribution of this paper is limited from the perspective of architectural innovation.\n2. TGDMat(Short) does not always outperform baseline methods, and the improvement of TGDMat(Long) over baseline methods is also marginal. This indicates that the model's results are heavily dependent on how the text information is constructed, and richer textual information can lead to slightly better outcomes than the baseline.\n3. As metrics commonly used to evaluate generative performance, the authors should consider including diversity and novelty as additional metrics for comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)