SlowGuess/NeurIPS_2023_OCR huggingface dataset:
For P-cG-SchNet, we apply the SchNet [29] with 9 layers, 128 hidden states as the backbone model. The model is trained for 500 epochs on each dataset with an Adam optimizer with initial learning rate  $1 \times 10^{-4}$  and a Plateau scheduler with a decaying factor 0.5 and a patience of 10 epochs. We select the element proportion and the number of atoms in a unit cell as conditions for the CSP task. For CDVAE, we apply the DimeNet++ [63] with 4 layers, 256 hidden states as the encoder and the GemNet-T [64] with 3 layers, 128 hidden states as the decoder. We further apply a Transformer [51] model with 2 layers, 128 hidden states as the additional prior encoder as proposed in Appendix B.2. The model is trained for 3500, 4000, 1000, 1000 epochs for Perov-5, Carbon-24, MP-20 and MPTS-52 respectively with an Adam optimizer with initial learning rate  $1 \times 10^{-3}$  and a Plateau scheduler with a decaying factor 0.6 and a patience of 30 epochs. For our DiffCSP, we utilize the setting of 4 layer, 256 hidden states for Perov-5 and 6 layer, 512 hidden states for other datasets. The dimension of the Fourier embedding is set to  $k = 256$ . We apply the cosine scheduler with  $s = 0.008$  to control the variance of the DDPM process on  $L_{t}$ , and an exponential scheduler with  $\sigma_{1} = 0.005$ ,  $\sigma_{T} = 0.5$  to control the noise scale of the score matching process on  $F_{t}$ . The diffusion step is set to  $T = 1000$ . Our model is trained for 3500, 4000, 1000, 1000 epochs for Perov-5, Carbon-24, MP-20 and MPTS-52 with the same optimizer and learning rate scheduler as CDVAE. For the step size  $\gamma$  in Langevin dynamics for the structure prediction task, we apply  $\gamma = 5 \times 10^{-7}$  for Perov-5,  $1 \times 10^{-5}$  for MP-20 and MPTS-52, and for Carbon-24, we apply  $\gamma = 5 \times 10^{-6}$  to predict one sample and  $\gamma = 5 \times 10^{-7}$  for multiple samples. For the ab initio generation and optimization task on Perov-5, Carbon-24 and MP-20, we apply  $\gamma = 1 \times 10^{-6}$ ,  $1 \times 10^{-5}$ ,  $5 \times 10^{-6}$ , respectively. All models are trained on GeForce RTX 3090 GPU.
4https://github.com/txie-93/cdvae/tree/main/data   
5https://github.com/sparks-baird/mp-time-split   
6http://www.comates.group/links?software  $\equiv$  gn_oa   
7https://github.com/atomistic-machine-learning/cG-SchNet   
8https://github.com/txie-93/cdvae   
9https://github.com/hyperopt/hyperopt   
10https://github.com/guofei9987/scikit-opt
C Exploring the Effects of Sampling and Candidate Ranking
C.1 Impact of Sampling Numbers
Figure 6 illustrates the impact of sampling numbers on the match rate. The match rate of all methods increases when sampling more candidates, and DiffCSP outperforms the baselines methods under the arbitrary number of samples.
Figure 6: Comparison on different number of samples.
C.2 Diversity
We further evaluate the diversity by yielding the CrystalNN [65] fingerprint of each generated structure, calculating the L2-distances among all pairs in the 20 samples of each composition, collating the mean and max value of the distances, and finally averaging the results from all testing candidi
dates. We list the diversity of CDVAE and DiffCSP on Perov-5 and MP-20 in Table 5.
Table 5: Comparison on the diversity.
Perov-5	MP-20
Mean	Max	Mean	Max
CDVAE	0.3249	0.7316	0.3129	0.6979
DiffCSP	0.3860	0.8911	0.2030	0.5292
C.3 Ranking among Multiple Candidates
In § 5.1, we match each candidate with the ground truth structure to pick up the best sample. However, in real CSP scenarios, the ground truth structure is not available, necessitating a confidence model to rank the generated candidates. To address this, we develop three types of confidence models to score each sample for ranking from different perspectives: Energy Predictor (EP). Since lower formation energy typically leads to more stable structures, we directly train a predictor using energy labels and apply the negative of the predicted
Table 6: Results on different confidence models. Oracle means applying the negative RMSD against the ground truth as the ranking score.
Match rate (%) ↑	RMSE ↓
EP	51.96	0.0589
MD (d=0.1)	60.30	0.0357
MD (d=0.3)	60.13	0.0382
MD (d=0.5)	59.20	0.0469
CS	58.81	0.0443
Oracle	77.93	0.0492
energy as the confidence score. Match Discriminator (MD). Inspired by Diffdock [12], we first generate five samples for each composition in the training/validation set using DiffCSP and calculate their RMSDs with the ground truth. We then train a binary classifier to predict whether the sample matches the ground truth with an RMSD below a threshold  $d$ . The predicted probability serves as the score. Contrastive Scorer (CS). Drawing inspiration from CLIP [66], we train a contrastive model between a 1D and 3D model to align the corresponding compositions and structures. The inner product of the 1D and 3D models is used as the score. We select the Top-1 result among the 20 candidates ranked by each confidence model on the MP-20 dataset, as shown in Table 6. The results indicate that MD and CS perform relatively better, but there remains a gap between the heuristic ranking models and the oracle ranker. Designing powerful ranking models is an essential problem, which we leave for future research.
D Impact of Noise Schedulers
We explore the noise schedulers from three perspectives and list the results in Table 7 and 8. 1. For lattices, we originally use the cosine scheduler with  $= 0.008$ , and we change it into the linear and sigmoid schedulers with  $\beta_{1} = 0.0001, \beta_{T} = 0.02$ . We find that the linear scheduler yields comparable results, while the sigmoid scheduler hinders the performance. 2. For fractional coordinates, we use the exponential scheduler with  $\sigma_{min} = 0.005, \sigma_{max} = 0.5$ , and we change the value of  $\sigma_{max}$  into 0.1 and 1.0. Results show that the small- $\sigma_{max}$  variant performs obviously worse, as only sufficiently large  $\sigma_{max}$  could approximate the prior uniform distribution. We visualize the PDF curves in Figure 7 for better understanding. 3. For atom types, we conduct similar experiments as lattices, and the results indicate that the original cosine scheduler performs better. In conclusion, we suggest applying the proposed noise schedulers.
Table 7: CSP results on different schedulers. DiffCSP utilizes cosine scheduler on  $\mathbf{L}$  and  ${\sigma }_{max} = {0.5}$  on  $\mathbf{F}$  .
Match rate (%) ↑	RMSE ↓
DiffCSP	51.49	0.0631
Schedulers of L
Linear	50.06	0.0590
Sigmoid	45.24	0.0664
Schedulers of F
σmax=0.1	32.56	0.0913
σmax=1.0	47.89	0.0675
Table 8: Ab initio generation results on different type schedulers. DiffCSP utilizes cosine scheduler on  $A$  .
Validity	Coverage
Struct.(%)	Comp.(%)	Recall	Precision
DiffCSP	100.00	83.25	99.71	99.76
Linear	99.70	79.78	98.29	99.48
Sigmoid	99.88	81.59	99.33	99.55
E Learning Curves of Different Variants
We plot the curves of training and validation loss of different variants proposed in § 5.3 in Figure 8 with the following observations. 1. The multi-graph methods struggle with higher training and validation loss, as the edges constructed under different disturbed lattices vary significantly, complicating the training procedure. 2. The Fourier transformation, expanding the relative coordinates and maintaining the periodic translation invariance, helps the model converge faster at the beginning of the training procedure. 3. The variant utilizing the fully connected graph without the Fourier transformation (named "DiffCSP w/o FT" in Figure 8) encounters obvious overfitting as the periodic translation invariance is violated, highlighting the necessity to leverage the desired invariance into the model.
F Comparison with DFT-based Methods
F.1 Implementation Details
We utilize USPEX [59], a DFT-based software equipped with the evolutionary algorithm to search for stable structures. We use 20 populations in each generation, and end up with 20 generations for each compound. We set  $60\%$  of the lowest-enthalpy structures allowed to produce the next generation through heredity  $(50\%)$ , lattice mutation  $(10\%)$ , and atomic permutation  $(20\%)$ . Moreover, two
Figure 7: PDF curves of the wrapped normal distribution with periodic as 1 and different noise scales. It can be found that  $\sigma = 0.5$  is practically large enough to approximate the prior uniform distribution.
Figure 8: Learning curves of different variants proposed in § 5.3. MG and FT denote multi-graph edge construction and Fourier transformation, respectively.
lowest-enthalpy structures are allowed to survive into the next generation. The structural relaxations are calculated by the frozen-core all-electron projector augmented wave (PAW) method [67] as implemented in the Vienna ab initio simulation package (VASP) [68]. Each sample's calculation is performed using one node with 48 cores (Intel(R) Xeon(R) CPU E5-2692 v2 @ 2.20GHz), while the populations within the same generation are concurrently computed across 20 nodes in parallel. The exchange-correlation energy is treated within the generalized gradient approximation (GGA), using the Perdew-Burke-Ernzerhof (PBE) function [69].
F.2 Results
We select 10 binary and 5 ternary compounds in MP-20 testing set and compare our model with USPEX. For our method, we sample 20 candidates for each compound following the setting in Table 1. For USPEX, we apply 20 generations, 20 populations for each compound, and select the best sample in each generation, leading to 20 candidates as well. We list the minimum RMSD of each compound in Table 9, and additionally summarize the match rate over the 15 compounds, the averaged RMSD over the matched structures, and the averaged inference time to generate 20 candidates for each compound in Table 10. The results show that DiffCSP correctly predicts more structures with higher match rate and significantly lower time cost.
Table 9: The minimum RMSD of 20 candidates of USPEX and DiffCSP, "N/A" means none of the candidates match with the ground truth.
Binary	Co2Sb2	Sr2O4	AlAg4	YMg3	Cr4Si4
USPEX	0.0008	0.0121	N/A	0.0000	N/A
DiffCSP	0.0005	0.0133	0.0229	0.0003	0.0066
Binary	Sn4Pd4	Ag6O2	Co4B2	Ba2Cd6	Bi2F8
USPEX	0.0184	0.0079	0.0052	N/A	N/A
DiffCSP	0.0264	N/A	N/A	0.0028	N/A
Ternary	KZnF3	Cr3CuO8	Bi4S4Cl4	Si2(CN2)4	Hg2S2O8
USPEX	0.0123	N/A	N/A	N/A	0.0705
DiffCSP	0.0006	0.0482	0.0203	N/A	0.0473
Table 10: Overall results over the 15 selected compounds.
Match Rate (%)↑	Avg. RMSD↓	Avg. Time↓
USPEX	53.33	0.0159	12.5h
DiffCSP	73.33	0.0172	10s
G Extension to More General Tasks
G.1 Overview
Our method mainly focuses on CSP, aiming at predicting the stable structures from the fixed composition  $\mathbf{A}$ . We first enable the generation on atom types and extend to ab initio generation task in § G.2, and then adopt the energy guidance for property optimization in § G.3. Figure 9 illustrated the differences and connections of CSP, ab initio generation, and property optimization. Besides, CDVAE [9] proposes a reconstruction task specific for VAE-based models, which first encodes the ground truth structure, and require the model to recover the input structure. As our method follows a diffusion-based framework instead of VAE, it is unsuitable to conduct the reconstruction task of our method.
G.2 Ab Initio Generation
Apart from  $L$  and  $F$ , the ab initio generation task additionally requires the generation on  $\mathbf{A}$ . As the atom types can be viewed as either  $N$  discrete variables in  $h$  classes, or the one-hot representation  $\mathbf{A} \in \mathbb{R}^{h \times N}$  in continuous space. We apply two lines of diffusion processes for the type generation, which are dually shown as follows.
Multinomial Diffusion for Discrete Generation Regarding the atom types as discrete features, we apply the multinomial diffusion with the following forward diffusion process [70, 42, 71],
$$
q \left(\boldsymbol {A} _ {t} \mid \boldsymbol {A} _ {0}\right) = \mathcal {C} \left(\boldsymbol {A} _ {t} \mid \bar {\alpha} _ {t} \boldsymbol {A} _ {0} + \frac {(1 - \bar {\alpha} _ {t})}{h} \mathbf {1} _ {h} \mathbf {1} _ {N} ^ {\top}\right), \tag {18}
$$
(a) Crystal Structure Prediction (CSP)
(b) Ab Initio Generation
(c) Property Optimization
(d) Reconstruction
Figure 9: Different tasks for crystal generation. Our approach mainly focuses on the CSP task (a), and is capable to extend into the ab initio generation task (b) by further generating the composition, and the property optimization task (c) via introducing the guidance on the target property. The reconstruction task (d) in Xie et al. [9] is specific for VAE, which is unnecessary for our diffusion-based method.
where  $\mathbf{1}_h\in \mathbb{R}^{h\times 1}$ ,  $\mathbf{1}_N\in \mathbb{R}^{N\times 1}$  are vectors with all elements setting to one,  $A_0$  is the one-hot representation of the origin composition, and the function  $\mathcal{C}(\cdot)$  samples the multinomial distribution with the conditional probability and returns the one-hot representation of  $\mathbf{A}_t$ .
The corresponding backward generation process is defined as
$$
p \left(\boldsymbol {A} _ {t - 1} \mid \mathcal {M} _ {t}\right) = \mathcal {C} \left(\boldsymbol {A} _ {t - 1} \mid \tilde {\theta} _ {t} / \sum_ {k = 1} ^ {h} \tilde {\theta} _ {t, k}\right), \tag {19}
$$
$$
\tilde {\theta} _ {t} = \left(\alpha_ {t} \mathbf {A} _ {t} + \frac {(1 - \alpha_ {t})}{h} \mathbf {1} _ {h} \mathbf {1} _ {N} ^ {\top}\right) \odot \left(\bar {\alpha} _ {t} \hat {\epsilon} _ {\mathbf {A}} \left(\mathcal {M} _ {t}, t\right) + \frac {(1 - \bar {\alpha} _ {t})}{h} \mathbf {1} _ {h} \mathbf {1} _ {N} ^ {\top}\right), \tag {20}
$$
and  $\hat{\epsilon}_{\mathbf{A}} \in \mathbb{R}^{h \times N}$  is predicted by the denoising model. We further find that specific for  $t = 1$ ,  $\mathbf{A}_0 = \operatorname{argmax}(\hat{\epsilon}_{\mathbf{A}}(\mathcal{M}_1, 1))$  works better.
The training objective for multinomial diffusion is
$$
\mathcal {L} _ {\boldsymbol {A}, \text {d i s c r e t e}} = \mathbb {E} _ {\boldsymbol {A} _ {t} \sim q (\boldsymbol {A} _ {t} | \boldsymbol {A} _ {0}), t \sim \mathcal {U} (1, T)} \left[ \mathrm {K L} \left(q \left(\boldsymbol {A} _ {t - 1} | \boldsymbol {A} _ {t}\right) \mid p \left(\boldsymbol {A} _ {t - 1} | \boldsymbol {A} _ {t}\right)\right) \right]. \tag {21}
$$
One-hot Diffusion for Continuous Generation Another approach is to simply consider the composition  $\mathbf{A}$  as a continuous variable in real space  $\mathbb{R}^{h\times N}$ , which enables the application of standard DDPM-based method [41]. Similar to Eq. (2)-(4), the forward diffusion process is defined as
$$
q \left(\boldsymbol {A} _ {t} \mid \boldsymbol {A} _ {0}\right) = \mathcal {N} \left(\boldsymbol {L} _ {t} \mid \sqrt {\bar {\alpha} _ {t}} \boldsymbol {A} _ {0}, (1 - \bar {\alpha} _ {t}) \boldsymbol {I}\right). \tag {22}
$$
And the backward generation process is defined as
$$
p \left(\boldsymbol {A} _ {t - 1} \mid \mathcal {M} _ {t}\right) = \mathcal {N} \left(\boldsymbol {A} _ {t - 1} \mid \mu_ {\boldsymbol {A}} \left(\mathcal {M} _ {t}\right), \sigma_ {\boldsymbol {A}} ^ {2} \left(\mathcal {M} _ {t}\right) \boldsymbol {I}\right), \tag {23}
$$
where  $\mu_{\mathbf{A}}(\mathcal{M}_t) = \frac{1}{\sqrt{\alpha_t}}\Big(A_t - \frac{\beta_t}{\sqrt{1 - \bar{\alpha}_t}}\hat{\epsilon}_{\mathbf{A}}(\mathcal{M}_t,t)\Big),\sigma_{\mathbf{A}}^2 (\mathcal{M}_t) = \beta_t\frac{1 - \bar{\alpha}_{t - 1}}{1 - \bar{\alpha}_t}$ . The denoising term  $\hat{\epsilon}_{\mathbf{A}}(\mathcal{M}_t,t)\in \mathbb{R}^{h\times N}$  is predicted by the model.
The training objective for one-hot diffusion is
$$
\mathcal {L} _ {\boldsymbol {A}, \text {c o n t i n u o u s}} = \mathbb {E} _ {\boldsymbol {\epsilon} _ {\boldsymbol {A}} \sim \mathcal {N} (0, I), t \sim \mathcal {U} (1, T)} [ \| \boldsymbol {\epsilon} _ {\boldsymbol {A}} - \hat {\boldsymbol {\epsilon}} _ {\boldsymbol {A}} (\mathcal {M} _ {t}, t) \| _ {2} ^ {2} ]. \tag {24}
$$
The entire objective for training the joint diffusion model on  $L, F, A$  is combined as
$$
\mathcal {L} _ {\mathcal {M}} = \lambda_ {L} \mathcal {L} _ {L} + \lambda_ {F} \mathcal {L} _ {F} + \lambda_ {A} \mathcal {L} _ {A}. \tag {25}
$$
We select  $\lambda_{L} = \lambda_{F} = 1$ ,  $\lambda_{A} = 0$  for the CSP task as  $A$  is fixed during the generation process, and  $\lambda_{L} = \lambda_{F} = 1$ ,  $\lambda_{A} = 20$  for the ab initio generation task to balance the scale of each loss component. Specifically, we do not optimize  $\mathcal{L}_A$  on the Carbon-24 dataset, as all atoms in this dataset are carbon.
Sample Structures with Arbitrary Numbers of Atoms As the number of atoms in a unit cell (i.e.  $N$ ) is unchanged during the generation process, we first sample  $N$  according to the distribution of  $N$  in the training set, which is similar to Hoogeboom et al. [41]. The sampled distribution  $p(\mathcal{M})$  can be more concisely described as  $p(\mathcal{M}, N) = p(N)p(\mathcal{M}|N)$ . The former term  $p(N)$  is sampled from pre-computed data distribution, and the latter conditional distribution  $p(\mathcal{M}|N)$  is modeled by DiffCSP.
**Evaluation Metrics** The results are evaluated from three perspectives. **Validity:** We consider both the structural validity and the compositional validity. The structural valid rate is calculated as the percentage of the generated structures with all pairwise distances larger than  $0.5\mathring{\mathrm{A}}$ , and the generated composition is valid if the entire charge is neutral as determined by SMACT [72]. **Coverage:** It measures the structural and compositional similarity between the testing set  $\mathcal{S}_t$  and the generated samples  $\mathcal{S}_g$ . Specifically, letting  $d_S(\mathcal{M}_1,\mathcal{M}_2)$ ,  $d_C(\mathcal{M}_1,\mathcal{M}_2)$  denote the  $L2$  distances of the CrystalNN structural fingerprints [65] and the normalized Magpie compositional fingerprints [73], the COVerage Recall (COV-R) is determined as COV-R =  $\frac{1}{|\mathcal{S}_t|} |\{\mathcal{M}_i|\mathcal{M}_i\in \mathcal{S}_t,\exists \mathcal{M}_j\in \mathcal{S}_g,d_S(\mathcal{M}_i,\mathcal{M}_j)< \delta_S,d_C(\mathcal{M}_i,\mathcal{M}_j)< \delta_C\}|$  where  $\delta_S,\delta_C$  are pre-defined thresholds. The COVerage Precision (COV-P) is defined similarly by swapping  $\mathcal{S}_g,\mathcal{S}_t$ . **Property statistics:** We calculate three kinds of Wasserstein distances between the generated and testing structures, in terms of density, formation energy, and the number of elements [9], denoted as  $d_{\rho}$ ,  $d_E$  and  $d_{\mathrm{elem}}$ , individually. The validity and coverage metrics are calculated on 10,000 generated samples, and the property metrics are evaluated on a subset with 1,000 samples passing the validity test.
Results We denote the abovementioned discrete and continuous generation methods as DiffCSP-D, DiffCSP-C, respectively. The results of the two variants and the strongest baseline CDVAE on MP-20 are provided in Table 11. We observe that DiffCSP-D yields slightly lower validity and coverage rates than DiffCSP-C. Moreover, DiffCSP-D tends to generate structures with more types of elements, which is far from the data distribution. Hence, we select DiffCSP-C for the other experiments in Table 4 (abbreviated as DiffCSP). We further find that DiffCSP-C supports the property optimization task in the next section. Besides, both variants have lower composition validity than CDVAE, implying that more powerful methods for composition generation are required. As our paper mainly focuses on the CSP task, we leave this for future studies.
G.3 Property Optimization
On top of DiffCSP-C, we further equip our method with energy guidance [74, 75] for property optimization. Specifically we train a time-dependent property predictor  $E(\pmb{L}_t, \pmb{F}_t, \pmb{A}_t, t)$  with the same message passing blocks as Eq. (6)-(8). And the final prediction is acquired by the final layer as
$$
E = \varphi_ {E} \left(\frac {1}{N} \sum_ {i = 1} ^ {N} h _ {i} ^ {(S)}\right). \tag {26}
$$
And the gradients w.r.t.  $L, F, A$  additionally guide the generation process. As the inner product term  $L^{\top} L$  is O(3) invariant, and the Fourier transformation term  $\phi_{\mathrm{FT}}(f_j - f_i)$  is periodic translation invariant, the predicted energy  $E$  is periodic E(3) invariant. That is,
$$
E \left(\boldsymbol {Q} \boldsymbol {L} _ {t}, w \left(\boldsymbol {F} _ {t} + \boldsymbol {t 1} ^ {\top}, \boldsymbol {A} _ {t}, t\right) = E \left(\boldsymbol {L} _ {t}, \boldsymbol {F} _ {t}, \boldsymbol {A} _ {t}, t\right). \right. \tag {27}
$$
Table 11: Results on MP-20 ab initio generation task.
Data	Method	Validity (%) ↑	Coverage (%) ↑	Property ↓
Struc.	Comp.	COV-R	COV-P	dρ	dE	delem
MP-20	CDVAE [9]	100.0	86.70	99.15	99.49	0.6875	0.2778	1.432
DiffCSP-D	99.70	83.11	99.68	99.53	0.1730	0.1366	0.9959
DiffCSP-C	100.0	83.25	99.71	99.76	0.3502	0.1247	0.3398
Algorithm 3 Energy-Guided Sampling Procedure of DiffCSP(-C)
1: Input: denoising model  $\phi$ , energy predictor  $E$ , step size of Langevin dynamics  $\gamma$ , guidance magnitude  $s$ , input structure before optimization  $\mathcal{M} = (L, F, A)$ , number of sampling steps  $T'$ , maximum number of sampling steps  $T$ .
2: if  $\mathrm{T}' = \mathrm{T}$  then
3: Sample  $L_{T^{\prime}}\sim \mathcal{N}(\mathbf{0},\mathbf{I}),F_{T^{\prime}}\sim \mathcal{U}(0,1),A_{T^{\prime}}\sim \mathcal{N}(\mathbf{0},\mathbf{I})$
4: else
5: Sample  $L_{T^{\prime}}\sim q(L_{T^{\prime}}|L),F_{T^{\prime}}\sim q(F_{T^{\prime}}|F)$ $\pmb{A}_{T^{\prime}}\sim q(\pmb{A}_{T^{\prime}}|\pmb{A})$  according to Eq. (2),(5) and (22).
6: end if
7: for  $t \gets T', \dots, 1$  do
8: Sample  $\epsilon_L, \epsilon_F, \epsilon_A, \epsilon_F' \sim \mathcal{N}(\mathbf{0}, I)$
9:  $\hat{\epsilon}_L, \hat{\epsilon}_F, \hat{\epsilon}_A \gets \phi(\pmb{L}_t, \pmb{F}_t, \pmb{A}_t, t)$ .
10: Acquire  $\nabla_{\pmb{L}}E,\nabla_{\pmb{F}}E,\nabla_{\pmb{A}}E$  from  $E(L_{t},F_{t},A_{t},t)$
11:  $L_{t - 1}\leftarrow \frac{1}{\sqrt{\alpha_t}} (L_t - \frac{\beta_t}{\sqrt{1 - \bar{\alpha}_t}}\hat{\epsilon}_L) - s\beta_t\cdot \frac{1 - \bar{\alpha}_{t - 1}}{1 - \bar{\alpha}_t}\nabla_L E + \sqrt{\beta_t\cdot\frac{1 - \bar{\alpha}_{t - 1}}{1 - \bar{\alpha}_t}}\epsilon_L.$
12:  $\pmb{A}_{t - 1}\gets \frac{1}{\sqrt{\alpha_t}} (\pmb {A}_t - \frac{\beta_t}{\sqrt{1 - \bar{\alpha}_t}}\hat{\pmb{\epsilon}}_{\pmb{A}}) - s\beta_t\cdot \frac{1 - \bar{\alpha}_{t - 1}}{1 - \bar{\alpha}_t}\nabla_{\pmb{A}}E + \sqrt{\beta_t\cdot\frac{1 - \bar{\alpha}_{t - 1}}{1 - \bar{\alpha}_t}}\pmb {\epsilon}_{\pmb{A}}.$
13:  $\pmb{F}_{t - \frac{1}{2}}\gets w(\pmb{F}_t + (\sigma_t^2 -\sigma_{t - 1}^2)\hat{\epsilon}_\pmb{F} - s\frac{\sigma_{t - 1}^2(\sigma_t^2 - \sigma_{t - 1}^2)}{\sigma_t^2}\nabla_\pmb{F}E + \frac{\sigma_{t - 1}\sqrt{\sigma_t^2 - \sigma_{t - 1}^2}}{\sigma_t}\pmb {\epsilon}_\pmb{F})$
14:  $\mathbf{\Phi}_{-},\hat{\pmb{\epsilon}}_{F},_{-}\leftarrow \phi (\pmb{L}_{t - 1},\pmb{F}_{t - \frac{1}{2}},\pmb{A}_{t - 1},t - 1).$
15:  $d_{t}\gets \gamma \sigma_{t - 1} / \sigma_{1}$
16:  $\pmb{F}_{t - 1} \gets w(\pmb{F}_{t - \frac{1}{2}} + d_t\hat{\epsilon}_\pmb{F} + \sqrt{2d_t}\epsilon_\pmb{F})$ .
17: end for
18: Return  $L_0, F_0$ ,  $\operatorname{argmax}(A_0)$ .
Table 12: Results on property optimization task. The results of baselines are from Xie et al. [9].
Method	Perov-5	Carbon-24	MP-20
SR5	SR10	SR15	SR5	SR10	SR15	SR5	SR10	SR15
FTCP	0.06	0.11	0.16	0.00	0.00	0.00	0.02	0.04	0.05
Cond-DFC-VAE	0.55	0.64	0.69	-	-	-	-	-	-
CDVAE	0.52	0.65	0.79	0.00	0.06	0.06	0.78	0.86	0.90
DiffCSP	1.00	1.00	1.00	0.50	0.69	0.69	0.82	0.98	1.00
Taking gradient to both sides w.r.t.  $L, F, A$ , respectively, we have
$$
\boldsymbol {Q} ^ {\top} \nabla_ {\boldsymbol {L} _ {t} ^ {\prime}} E \left(\boldsymbol {L} _ {t} ^ {\prime}, w \left(\boldsymbol {F} _ {t} + \boldsymbol {t 1} ^ {\top}, \boldsymbol {A} _ {t}, t\right) \mid_ {\boldsymbol {L} _ {t} ^ {\prime} = \boldsymbol {Q} \boldsymbol {L} _ {t}} = \nabla_ {\boldsymbol {L} _ {t}} E \left(\boldsymbol {L} _ {t}, \boldsymbol {F} _ {t}, \boldsymbol {A} _ {t}, t\right), \right. \tag {28}
$$