Unleashing the power of novel conditional generative approaches for new materials discovery

Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 18:11:00

Abstract: For a very long time, computational approaches to the design of new materials have relied on an iterative process of finding a candidate material and modeling its properties. AI has played a crucial role in this regard, helping to accelerate the discovery and optimization of crystal properties and structures through advanced computational methodologies and data-driven approaches. To address the problem of new materials design and fasten the process of new materials search, we have applied latest generative approaches to the problem of crystal structure design, trying to solve the inverse problem: by given properties generate a structure that satisfies them without utilizing supercomputer powers. In our work we propose two approaches: 1) conditional structure modification: optimization of the stability of an arbitrary atomic configuration, using the energy difference between the most energetically favorable structure and all its less stable polymorphs and 2) conditional structure generation. We used a representation for materials that includes the following information: lattice, atom coordinates, atom types, chemical features, space group and formation energy of the structure. The loss function was optimized to take into account the periodic boundary conditions of crystal structures. We have applied Diffusion models approach, Flow matching, usual Auto-Encoder (AE) and compared the results of the models and approaches. As a metric for the study, physical pymatgen matcher was employed: we compare target structure with generated one using default tolerances. So far, our modifier and generator produce structures with needed properties with accuracy 41% and 82% respectively. To prove the offered methodology efficiency, inference have been carried out, resulting in several potentially new structures with formation energy below the AFLOWLib-derived convex hulls.

Main paper: 
 introduction: 
 The search for novel materials with specified properties has been a cornerstone of scientific exploration for decades. From the discovery of semiconductors revolutionizing electronics to the development of superalloys enhancing aerospace technologies, the synthesis of new materials has continually propelled technological advancements.However, traditional methods for material discovery often employ exhaustive trial and error experimental approaches. In turn, computational efforts, relying on density functional theory (DFT) [1] approaches, usually require huge amounts of computing power. In this regard, automatic descriptor generators [2] , GNNs [3] [4] and transferable GNN models [5] fueled combination of these methods and machine learning (ML) approaches. In particular, the utilization of generative machine learning models, such as Variational Autoencoder [6] and GANs [7] , presents a paradigm shift in how crystal structures are generated and optimized.By harnessing the power of data-driven approaches, we can navigate the vast landscape of possible crystal structures with unprecedented efficiency and precision.Recent advancements in the field of materials discovery have yielded promising results through various innovative approaches. For instance, FTCP [6] utilizes Autoencoders for uncovering new materials, while CubicGAN [7] leverages GANs for the discovery of cubic crystal materials. Additionally, Physics Guided Crystal Generative Model (PGCGM) [8] has introduced a method for generating crystal structures based on specific space groups encoding. DP-CDVAE [9] is a model, that combines VAE and diffusion approaches. MatterGen [10] employed equivariant GNNs as score matching function in diffusion processes for crystal structure generation.One of the most discussed frameworks is GNoME [11] that has made most recent and large advancements in the field of the new materials discovery employs a sophisticated pipeline to discover new materials, particularly focusing on inorganic crystals. This allows for the discovery of innovative materials beyond known structures.After generating candidate structures through both pipelines, GNoME evaluates their stability by predicting their formation energies. Based on the comparison of the obtained formation energy with those of the known competing phases (i.e. stability assessment), the model selects the most promising candidates for further evaluation using known theoretical frameworks.The question of the completeness of chemical space arises due to two main concerns with GNoME-derived stable structures. Firstly, they mostly contain three or more unique elements, while ternary and quaternary structures are less explored than binary compounds. Secondly, the comparison of GNoME-discovered structures to the Materials Project, which has 154,718 materials, is flawed since larger databases like AFLOW, NOMAD, and the Open Quantum Materials Database contain millions of entries. This raises questions about the novelty of the discovered materials.In this study, we present an end-to-end framework for the generation of crystal structures with specified properties using advanced generative AI techniques. The basis architecture of the models is Autoencoder, enabling encoding and decoding structural representations. Then, the most commonly used generative approaches in image generation were utilized to model probability distribution transformations, and to capture complex underlying structure-property relationships within our dataset: Flow Matching [12] , Denoising Diffusion Probabilistic Models(DDPM) [13] , and Denoising Diffusion Implicit Models(DDIM) [14] . Through the integration of these techniques, we aim to transcend conventional limitations in materials discovery, paving the way for accelerated predictions of materials with desired properties.To employ model architectures often used for image/video generation, a matrix representation of crystal structures was developed, containing crucial information such as chemical composition, atomic coordinates, symmetries (space group), and formation energies. Within the approach proposed, it has become important to develop a novel metric for assessing the similarity between generated structures and target configurations. This metric obviates the need for computationally expensive DFT calculations, allowing for rapid validation and refinement of generated structures. Furthermore, we introduce a loss function that accounts for the periodic boundary conditions inherent in crystal lattices, ensuring the fidelity of the generated structures.Our study explores two distinct approaches for crystal structure prediction: 1) conditional structure modification and 2) conditional structure generation. The former involves optimizing the stability of existing structures by generating more stable polymorphs, while the latter entails the generation of entirely new structures based on user-defined criteria. Through rigorous analysis, we demonstrate the efficacy of our approach in discovering novel materials with desired properties.Importantly, to validate the utility of our framework, we conducted a series of generation experiments using the Vienna Ab initio simulation package(VASP) [15] as a tool for inference validation. Remarkably, our method facilitated the discovery of 6 structures below the corresponding convex hull. This significant outcome underscores the remarkable potential of our framework in uncovering thermodynamically stable materials, thereby offering promising avenues for advanced materials discovery and design.2 Data, DatasetQuestion: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes] ,Justification: The main claims made in the abstract and introduction do accurately reflect the paper's contributions and scope. Every aspects mentioned in the abstract and introduction are further revealed in the main paper.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper.• The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations.A No or NA answer to this question will not be perceived well by the reviewers.• The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.• It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
data overview: 
 In this study, the AFLOW database [16] was utilized as a source of data on the structures and properties of materials. AFLOW is an extensive and comprehensive database that consolidates a vast array of materials-related information, offering an expansive repository for crystallographic data, computed properties, and various other materials-science-related datasets. AFLOW database contains more than 3.5 million structures.From the extensive collection housed within AFLOW, the focus was narrowed to select only polymorphs, because models are trained to distinguish composition-property and structureproperty relations with numerous structures of the same chemical composition. Specifically, the selection process targeted polymorphic structures with 4 to 60 atoms within their unit cells. This criterion aimed to encompass a diverse yet manageable subset of structures, balancing complexity with computational feasibility. By filtering polymorphs based on their atom count, the dataset was balanced.Moreover, in order to decrease the complexity of the data, we have removed all structures containing elements and space groups found in less than 1% of all structures. The entire dataset consisted of more than 85000 polymorph groups including more than 2.1 million structures. The minimum size of group of polymorphs was 7 samples and the maximum one was 71 samples. The total number of space groups was 19 and the total number of chemical species over the dataset was 55. Each structure S in the dataset is described by the following features:• Fractional coordinates of atoms in the lattice basis X coord (has 60 rows with 3 coordinates x, y, z each) and X lattice (matrix 3 by 3 constructed of 3 base vectors).Overall matrix X of structure is constructed asEQUATION• Chemical elements which are presented as a one-hot matrix elements ij of size 64 × 103 (including padding), where ones are positioned at the indices corresponding to the position of a certain chemical element in the periodic table.elements ij = 1 if i-th atom's element number from the periodic table = j 0 otherwise• Elemental property matrix elementalP roperties containing 22 chemical features encoding chemical elements obtained from [8] . The properties of each element were calculated using Mendeleev package [17] .• Space group spg of a structure. We use the space group encoding method presented in [8] , when each space group is represented by a 192 × 4 × 4 matrix, which corresponds to 192 possible symmetry operations.• Structure formation energy E• Nsites -number of atoms in a crystal lattice.The crystal pair sampling strategy involves handling a potential data leakage: possible inclusion of structures from the same polymorph group but with different energies into training and validation subsets. To mitigate this issue, the polymorph group formulas were initially divided into distinct training and validation sets, ensuring a relatively balanced distribution of chemical elements across these subsets. Subsequently, the pairs were categorized into two groups: those with low-energy (lowest energy in polymorph group) targets designated as lowestEnergyP airs = (S i , S 0 )∀i ∈ [1, ..., structuresN um] and those with non-low-energy targets, all structures except the most optimal one, formed as nonLowestEnergyP airs = (S i , S j )| i > j > 0. The validation set was constructed as a subset of lowestEnergyP airs.The training set was dynamically constructed every epoch from lowestEnergyP airs and nonlowestEnergyP airs, preserving equal numbers of pairs sampled and maintaining a limited count per polymorph group. This strategy ensured a robust separation between training and validation sets, thus preventing data leakage and improving model performance.Each pair sample {S init , S target } ∈ pairDataset consisted of the information about each structure (hereinafter, we will call them initial and target structures). The following data was used:• Coordinates and lattice information of initial and target structures X init , X target• Difference in formation energies between initial and target structuresE dif f = E target -E init• Space group of target structure spg target• Elements matrix elemetsM atrix, elemental property matrix elementalP roperties and number of sites numSites, which are the same for initial and target structure because of identical chemical composition.The modification task involved transforming the input structure X init into the target structure X target .In its tern the generation task receives normal or uniform (depends on a model) noise as input from which the structure is generated, which is akin to the image generation processes in computer vision tasks.For the generation task, an additional dataset was constructed. Data for the generation task is slightly simpler, while it considers only {S target }. Therefore, the models can be trained on all structures available, rather than pairs. The following data is used:• Coordinates and lattice information of target structure X target• Formation energy of target structure E target• Space group of target structure spg target• Elements matrix elemetsM atrix, elemental property matrix elementalP roperties and number of sites numSites of target structure.3 Loss and metricsIf the target coordinate of the atom is located on lattice edge (two coordinates are equal to 1 or 0 and one is not). For example, a lattice edge atom at point {0, 1, 0.3} has identical atoms at points {0, 0, 0.3}, {1, 0, 0.3}, {1, 1, 0.3}. As we can see, in this example z-coordinate is fixed but x and y are exchangeable.Therefore, if the target point is represented as {x, y, z}, we can use the following formula:L edge (preds i , target i ) = min e∈edgeP oints ||preds i -e||,where edgeP oints is a set of 4 possible positions according to PBC.• Case of fixed point x: edgeP oints = {{x, 0, 0}, {x, 0, 1}, {x, 1, 0}, {x, 1, 1}}• Case of fixed point y: edgeP oints = {{0, y, 0}, {0, y, 1}, {1, y, 0}, {1, y, 1}}• Case of fixed point z: edgeP oints = {{0, 0, z}, {0, 1, z}, {1, 0, z}, {1, 1, z}} 3. Sides evaluation: If the target coordinate of the atom is located on lattice side (one coordinate is equal to 1 or 0 and two are not). For example, lattice side atom at point {0, 0.5, 0.3} has identical atom at point {1, 0.5, 0.3}. In this example y and z coordinates are fixed but x is exchangeable. Therefore, if the target point is represented as {x, y, z}, we can use the following formula:L size (preds i , target i ) = min s∈sideP oints ||preds i -s||,where sideP oints is a set of 2 possible positions according to PBC.• Case of exchangeable point x: sideP oints = {{0, y, z}, {1, y, z}}• Case of exchangeable point y: sideP oints = {{x, 0, z}, {x, 1, z}}• Case of exchangeable point z: sideP oints = {{x, y, 0}, {x, y, 1}} Since the min(x 1 , x 2 , ..., x n ) function is undifferentiable at multiple points (x i = x j ∀i ̸ = j ), it makes a loss function to have a more complicated surface. Therefore, we used a norm function with order k → -∞ which is differentiable at all points as a replacement.min dif f (x 1 , x 2 , ..., x n ) = ( n i=1 |x i | k ) 1 k k -→ -∞Therefore, overall PBC-aware loss for a structure is represented as:L P BC (preds, target) = n i=1 I(target i is vertex point)L vertex (preds i , target i ) +I(target i is edge point)L edge (preds i , target i ) +I(target i is side point)L side (preds i , target i ) +I(target i is usual point)L 2 (preds i , target i )As the count of atoms varies across different structures, the L P BC metric tends to yield higher values for structures featuring a larger number of atoms. Thus, it is important to normalize the loss function with the number of atoms in the structure if it would be used in batches with structures with different number of atoms. Therefore, a PBC-aware loss for a batch of structures is formulated as: L batchP BC (• The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean.• It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.• For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).• Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.• The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.• The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
atomic coordinates: 
 The atomic coordinates are represented as a 60 × 3 matrix, where each row corresponds to the coordinates of an atom. The L1 loss was utilized during the training of a model for predicting atomic coordinates.L 1 (preds, target) i = ||preds i -target i || 1 = 3 j=1 |preds ij -target ij |, where target and pred are target and predicted atomic coordinate matrices.The lattice itself is represented as a 3x3 matrix, where each row signifies a directing basis vector. In this case, we have also used the L1 norm as a loss function.This section presents an enhanced loss function, designed for the regression model (see Section 5.1), that addresses this challenge by integrating periodic boundary conditions into the loss calculation, outperforming the conventional L1 loss function. In the field of ML applied to atomic structures, even slight displacement of atomic coordinates is crucial and employing appropriate loss functions that consider the periodic nature of atomic structures increases the flexibility of model predictions.In The empirical validation of this enhanced loss function showcases its superiority(Figure4) in capturing discrepancies within atomic structures, thus indicating its potential as a robust tool for improving the accuracy of ML models in materials science applications.As a metric, we have chosen an analogue of accuracy: the generated structures are compared to the target structures using a specialized matcher, yielding the proportion of structures that successfully pass the matching process. For metric calculation, we employed the Pymatgen StructureMatcher with the default set of parameters (ltol = 0.2, stol = 0.3, angle_tol = 5).Although this approach is less accurate than structure relaxation using ab initio calculations and comparing the structure formation energy with the energy above the hull, it enables model validation to be performed orders of magnitude faster than the traditional method. 
model: 
 For experiments, a 1d UNet model (see Figure2 (b)) architecture similar to the 2d UNet model described in [18] was utilized along with 2D and 1D convolutional neural networks (CNNs)for the space group and element matrix embeddings, respectively. Based on this model, 3different training processes have been developed: ordinary regression model, ConditionalFlow Matching (CFM) [19] model, and diffusion model.The model was conditioned (see Figure2 (a)) on the following data: time condition (t), the same as in [18] , element condition (el), formation energy difference condition (E dif f ), and desirable space group (spg). el emb , spg emb and E dif f are concatenated into one embedding respectively, then a linear interpolation x t is calculated as x t = tx 1 + (1 -t)x 0 (exponential moving average between distributions x 0 and x 1 ; t is sampled from a uniform distribution U(0, 1)), and afterwards pass the x t and t as inputs to our model f θ , forcing the model to predict a velocity from the distribution x 0 to x 1 . Therefore, the loss for CFM model is the following:L CF M = E t,x1,x0 [||f θ (x t , t) -(x 1 -x 0 )|| 2 ] = E t,x1,x0 [||f θ (tx 1 + (1 -t)x 0 , t) - (x 1 -x 0 )|| 2 ]For the modification approach, x 0 and x 1 are both sampled from our dataset distribution according to the sampling strategy for modification mentioned in 2.2. Also, the model isconditioned to el, spg 1 , E dif f , besides t (see Algorithm 4)For the generation approach, we tested two noise distributions for the x 0 : normal distribution N (0, 1) and uniform noise distribution U(0, 1), which resulted in significantly better performance. The intuition for using uniform distribution instead of normal one was inspired by the diagram of x, y, z coordinate distribution (Figure 3 ). The model is also conditioned to el, spg 1 , E, and t (see Algorithm 5)During the sampling stage, we generate X 1 structure by the given X 0 by solving the following ordinary differential equation (ODE): dx t = f θ (x t , t, el, spg 1 , E)dt, beginning with x 0 . In order to solve the ODE, the Euler method was employed:x t+h = x t + hf θ (x t , t, el, spg 1 , E) (Algorithm 6)The PBC loss function operates through several steps:1. Vertices evaluation: If the target coordinate of the atom is lattice vertex (all 3 coordinates x, y, zare equal to 1 or 0), then loss between prediction point preds i and target point target i is being calculated using following formula:L vertex (preds i , target i ) = min v∈vertices ||preds i -v||,where vertices is a set of 8 possible positions according to PBC ({0, 0, 0}, {0, 0, 1}, ..., {1, 1, 1}). 
diffusion models: 
 In our work, we observe diffusion models. Diffusion models generate samples from a target distribution x 1 , starting from a source distribution x 0 ∼ N (0, I).During training, these models are trained to reverse a Markovian forward process, which adds noise x 0 to the data step by step. Meaning, diffusion models are trained to predict the noise added to the data samples x 1 . In order to train a model in this setup, the following lossfunction is used, L simple = E t,x1,x0 [||x 0 -f θ ( √ ᾱt x 1 + √ 1 -ᾱt x 0 , t)|| 2 ]where ᾱt = t s=1 α s and α t = 1 -β t (β t is the variance by which added noise is being scheduled on each step t).Our modification approach is based on Palette, which enables sample-to-sample generation (from noise ϵ ∼ N (0, 1)) using x 0 structure coordinates and lattice, el, spg 1 , E dif f and t as conditions for generation of x 1 using the DDPM algorithm. Sampling stage is performed by a backward diffusion process with linear scheduler (see Algorithms 7, 8) .For the generation approach ((Algorithm 9), x 0 is sampled from a normal distribution and el, spg 1 , E, t are fed into the model as conditions. During our experiments, we tested 2 approaches: DDPM(Algorithm 10) classic approach and DDIM(Algorithm 11) which results in usage of smaller number of sampling steps in order to speed up the generation process. Moreover, DDIM enables the process of generating samples from random noise to be deterministic. 
experiment results: 
 All the models presented in tables (Table 1 and Table 2 ) have been trained with the same hyperparameters and architectures. The metric used is described in Section 3.4. We also provide all experiment details in A.3.  
inference: 
 In order to demonstrate a potential of the proposed approaches, we have chosen a chemical composition, containing numerous variations and phases of structures composed of [W, B, Ta] with well-explored convex hull. Structures that lie on the convex hull are considered to be thermodynamically stable, and the ones above it are either metastable or unstable.The proposed testing procedure involves generating test conditions for structures, passing them to the trained generative models, pre-optimizing the generated structures to accelerate the following ab initio calculations, and final relaxation and formation energy calculating using VASP. Although in this work two approaches were proposed: Generation and Modification, the following pipeline has only been applied to generation models, due to the fact, that modification approach is based on structure-polymorphs, which leads to the necessity to have at least one structure with needed composition, which is not always so. That fact makes generation models much more flexible in generation structures not only with needed properties, but also with needed composition. Another advantage of the generation models is value of metric that is two times bigger than in modification tasks. The inference algorithm is as follows:1. Test Condition Formation:• The chosen chemical formulas were utilized for feature extraction of el. Three chemical compositions have been used: 1) Ta 1 W 1 B 6 , 2) Ta 1 W 2 B 5 and 3)Ta 2 W 1 B 5 .• We have taken spg presented in the dataset as an additional condition, obtaining 19 space groups.• Finally, a set of target formation energies E has been formed. We have carried out three experiments: 1) starting from the energy of the convex hull and decreasing with a step of 0.01 eV/atom, 2) starting from the energy of the convex hull and decreasing with a step of 0.1 eV/atom, and 3) starting from the energy 1 eV/atom less than the energy of the convex hull and decreasing with a step of 0.01 eV/atom. In total, 21 energy values were used for every inference run.• Final inference conditions were obtained by making all possible combinations of spg and E for a certain composition el 2. Model Inference: The conditions from the step 1 have been put to one of the trained models, resulting in the generation of structures. Two models have been employed:Diffusion approach and Flow matching 3. Pre-Optimization: Following the generation of all structures, each structure has been pre-optimized using the PyMatGen structure relaxation method. The method used m3gnet [22] model with default parameters. PyMatGen pre-optimization contributed to overall speedup of further VASP relaxation.4. Structure relaxation: Pre-optimized structures were relaxed using VASP (the recommended pseudopotentials, plane wave energy cutoff of 500 eV, Ediif and Ediffg convergence criteria of 10 -5 and -10 -2 were used).To summarize, 6 experiments have been carried out for two different models and for three formation energy conditionings. Every experiment includes 3*380 structures, per 380 structures for every single chemical composition. The results of experiments can be seen in Table 3 As can be seen, 4 structures were obtained with formation energies significantly lower than those obtained from the AFLOW-derived convex hull. Thus, it can be concluded that this observation indicates the potential stability of the generated structures rather than differences in the computational methods used in this work and during AFLOW generation.Another four structures also have energies below the convex hull, but in the vicinity of it.Thus, their potential stability should be interpreted with caution. 
data availability: 
 The raw crystal dataset is downloaded from https://aflowlib.org  
conclusion: 
 In this article, we have offered two approaches to generate crystal structures: conditional generation and conditional modification. The first approach is significantly more flexible as it does not require structure-polymorphs, enabling the generation of structures without restrictions on chemical composition, which can be crucial in certain scenarios. Another advantage of the first approach is the simplicity of data preprocessing; it only requires the chemical composition, space group, atom coordinates, and formation energies.Our methodology has experimentally proven its effectiveness, resulting in four confident potentially new crystal structures with the following energies above the hull: {-1.409, -5.497, -5.426, and -4.852} meV/atom, and four uncertain candidates with energies of {-0.483, -0.466, -0.387, and -0.042} meV/atom. We have demonstrated that conditional generation approaches, commonly used in image generation, are also fruitful in the design of new materials.Although the proposed methodology demonstrates its efficiency in generating potentially new crystal structures, it has certain limitations. Firstly, the data is represented in a matrix form, which does not account for all possible symmetries of the crystal structures. Secondly, the structures in the dataset range from 4 to 60 atoms per unit cell, with most structures containing fewer than 8 atoms per unit cell. However, to perform well on structures with a large number of atoms per unit cell, the models just should be pretrained on a dataset that includes larger structures.Furthermore, despite the limited number of experiments(6) and structures generated (7182), we succeeded in identifying hypothetically new structures. We hope that our article will help to reveal the potential of generative AI in design of new materials with targeted thermodynamic properties and inspire other researchers to be part of this innovative journey in materials design. We believe that rapid and efficient generation of novel materials can lead to breakthroughs in various fields such as electronics, pharmaceuticals, and energy storage.This can accelerate technological advancements and make cutting-edge technologies more accessible and affordable.Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes]Justification: The broad societal impacts are discussed int the conclusion section.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out.For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation.On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.• The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.• If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: The paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.• Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.• We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: The models utilized in this research are appropriately cited within the text, including references to the AFLOW database. We have no commercial interests related to the use of these models and the crystal structure database. The code used in this study was entirely developed by our team.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL.• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.• We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS 
 
 Appendix: A.1 Distribution of atomic coordinates x 0 ∼ q(x 0 ); x 1 ∼ q(x 1 ); el ∼ q(el); spg 1 ∼ q(spg 1 ); E ∼ q(E)3: L ← ||x 1 -f θ (x 0 , t, el, spg 1 , E)|| 4:θ ← U pdate(θ, ∇ θ L(θ)) 5: until converge Algorithm 2 Inferencing Regression Modification Model 1: x 0 ∼ q(x 0 ); el ∼ q(el); spg 1 ∼ q(spg 1 ); E ∼ q(E) 2: x 1 = f θ (x 0 , t, el, spg 1 , E)3: return x 1 Algorithm 3 CFM Training 1: repeat 2:x 0 ∼ q(x 0 ); x 1 ∼ q(x 1 )3:t ∼ U(0, 1) 4: x 0 ∼ q(x 0 ); x 1 ∼ q(x 1 ); el ∼ q(el); spg 1 ∼ q(spg 1 ); E ∼ q(E)x t = tx 1 + (1 -t)x 0 5: L CF M ← ||f θ (x t , t) -(x 1 -x 0 )|| 6: θ ← U pdate(θ, ∇ θ L CF M (θ)3:t ∼ U(0, 1) 4: x 0 ∼ N (0, 1) or x 0 ∼ U(0, 1) 3:x t = tx 1 + (1 -t)x 0 5: L CF M ← ||f θ (x t , t, el, spg 1 , E) -(x 1 -x 0 )|| 6: θ ← U pdate(θ, ∇ θ L CF M (θ)) 7: until convergex 1 ∼ q(x 1 ); el ∼ q(el); spg 1 ∼ q(spg 1 ); E ∼ q(E) 4:t ∼ U(0, 1) 5:x t = tx 1 + (1 -t)x 0 6: L CF M ← ||f θ (x t , t, el, spg 1 , E) -(x 1 -x 0 )|| 7:θ ← U pdate(θ, ∇ θ L CF M (θ)) 8: until converge Algorithm 6 Sampling with CFM for Modification or Generation1: h = 1 T 2:x 0 ∼ q(x 0 ) or x 0 ∼ N (0, 1) or x 0 ∼ U(0, 1) 3: el ∼ q(el); spg 1 ∼ q(spg 1 ); E ∼ q(E) 4: for dot = 1, . . . , T do 5: x 0 ∼ q(x 0 ); x 1 ∼ q(x 1 ); el ∼ q(el); spg 1 ∼ q(spg 1 ); E ∼ q(E) x 0 ∼ q(x 0 ); x 1 ∼ q(x 1 ); el ∼ q(el); spg 1 ∼ q(spg 1 ); E ∼ q(E)x t+1 = x t + hf θ (x t ,3: t ∼ U({1, . . . , T }) 4: ϵ ∼ N (0, I) 5: L D ← ||ϵ -f θ ( √ ᾱt x 1 + √ 1 -ᾱt ϵ, x 0 , t, el, spg 1 , E)|| 6: θ ← U pdate(θ, ∇ θ L D (θ)) 7: until convergez ∼ N (0, I) if t > 1 else z = 0 5: x 1 ∼ q(x 1 ); el ∼ q(el); spg 1 ∼ q(spg 1 ); E ∼ q(E) x 1 ∼ q(x 1 ); el ∼ q(el); spg 1 ∼ q(spg 1 ); E ∼ q(E) x 0 ∼ q(x 0 ); x 1 ∼ q(x 1 ); el ∼ q(el); spg 1 ∼ q(spg 1 ); E ∼ q(E)x t-1 = 1 √ αt (x t -1-αt √ 1-ᾱt f θ (x t , x 0 , t, el, spg 1 , E)) + √ 1 -3: t ∼ U({1, . . . , T }) 4: ϵ ∼ N (0, I) 5: L D ← ||ϵ -f θ ( √ ᾱt x 1 + √ 1 -ᾱt ϵ, t, el, spg 1 , E)|| 6: θ ← U pdate(θ, ∇ θ L D (θ)4: z ∼ N (0, I) if t > 1 else z = 0 5: x t-1 = 1 √ αt (x t -1-αt √ 1-ᾱt f θ (x t , t, el, spg 1 , E)) + √ 1 -α tz ∼ N (0, I) if t > 1 else z = 0 5:x θ = f θ (x t , t, el, spg 1 , E) 6: x t-1 = √ α t-1 ( xt- √ 1-αtx θ √ αt ) + 1 -α t-1 -σ 2 t x θ + σ t z 7:8: end for 9: return x 1All the experiments use the same hyperparameters for the model:• num_res_blocks = 7• attention_resolution = (1, 2, 4, 8)• model_channels = 128In all the experiments models are trained with the same training parameters:• optimizer = Adam -betas = (0.9, 0.999) -eps = 1e-08 -weight_decay = 0• batch_size = 256• epochs = 400• learning_rate = 1e-4• lr_warmup_steps = 500• random_state = 42An important note, that all our experiments have been conducted in mixed precision in fp16.Generation task: Diffusion Model (DDPM):• num_train_timesteps = 1000 (diffusion process discretization)• beta_start = 0.0001• beta_end = 0.02• num_inference_steps = 100• beta_schedule = "squaredcos_cap_v2" (cosine)Diffusion Model (DDIM):• num_train_timesteps = 1000 (diffusion process discretization)• beta_start = 0.0001• beta_end = 0.02• num_inference_steps = 100• beta_schedule = "squaredcos_cap_v2" (cosine)Flow Matching x 0 ∼ N (0, 1):• num_inference_steps = 100Flow Matching x 0 ∼ U(0, 1):• num_inference_steps = 100Modification task:Regression UNet:• num_inference_steps = 1Diffusion Model:• num_train_timesteps = 1000 (diffusion process discretization)• beta_start = 0.0001• beta_end = 0.02• num_inference_steps = 100• beta_schedule = "squaredcos_cap_v2" (cosine)Flow Matching:• num_inference_steps = 100 A.For our computational needs in model training and inference, we deployed a total of three GPU servers with the following configurations:Server 1:• GPU: NVIDIA A100/80G• CPU: 8vCPU of Intel(R) Xeon(R) Gold 6248R @ 3.00 GHz• RAM: 64GbServer 2:• GPU: NVIDIA V100 (32GB)• CPU: 8vCPU of Intel(R) Xeon(R) Gold 6278C @ 2.60 GHz• RAM: 64Gb Server 3:• GPU: NVIDIA V100 (32GB)• CPU: 8vCPU of Intel(R) Xeon(R) Gold 6278C @ 2.60 GHz• RAM: 64GbEvery model training time consumed up to 2 weeks employing computing power of one GPU server.For the ab-initio calculations implemented in VASP, we deployed a total of 5 identical CPU servers with the following configurations:• CPU: 64vCPU of Intel(R) Xeon(R) Gold 6278C CPU @ 2.60GHz• RAM: 256GbStructure relaxation with VASP for all six experiments mentioned in Table 3 took more that 180 thousand CPU hours. Computing power of all CPU servers was employed.The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.Please read the checklist guidelines carefully for information on how to answer these questions.For each question in the checklist:• You should answer [Yes] , [No] , or [NA] .• [NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.• Please provide a short (1-2 sentence) justification right after your answer (even for NA).The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] " provided a proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] " or "[NA] " is not grounds for rejection.While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.IMPORTANT, please:• Delete this instruction block, but keep the section heading "NeurIPS paper checklist",• Keep the checklist subsection headings, questions/answers and guidelines below.• Do not modify the questions and only use the provided macros for your answers. 


Decision: This paper conducts an empirical investigation into the performance of various generative approaches for materials discovery. While the objective is valuable and relevant to the ML community, there are several concerns that remain unresolved even after the authors' rebuttals, as highlighted by the reviewers:

- Properties of the proposed data structure considered to be crucial in materials discovery, such as symmetry, invariance, or equivalence, are not adequately analyzed. This weakens the understanding of the approach’s applicability to the field.
- There are several issues in experiments, including the selection of datasets and the lack of appropriate baselines. These issues need to be addressed to ensure a comprehensive empirical evaluation. 
- The overall presentation requires significant improvement. Mathematical descriptions of the ML methods are often inaccurate, and the figures provided are not sufficiently informative.

In conclusion, the current quality of the manuscript is not sufficient for publication, and I must therefore recommend rejection. I hope the authors find the feedback provided by the reviewers useful for future revisions of the paper.

Additionally, as noted by several reviewers, the GitHub link in Section 9 could break anonymity. While this issue was not taken into account in the technical assessment of the paper, it could have led to a desk rejection. Please be careful of such issues in future submissions.

Reviews: review: paper_summary: paper_summary: The paper deals with an important application of generative models for science: generation of crystalline structures. However, I have some serious concerns. First, scope. While comparing different methods for the same objective is informative, I am not so sure what is the purpose here. Having so many different methods certainly dilutes the main message in a short paper format like neurips. Second, approach. From what I understand, there are questionable design problems with the technical approach. Third, results. The authors spend most of their space explaining various methods such that there is little room left for explaning the impact of their results, or comparing their results with existing approaches. Finally, presentation. Figure 1 is kind of trivial or at least very simple and I am not sure it is worth a separate figure. The overall typesetting looks not too professional.
strength_weakness: strength_weakness: I have a good number of questions on the technical approaches. More specifically, the model takes a specially formulated data structure that does not seem to be obviously invariant or equivariant under permutation, translation, rotation, which is concerning. For example, change the selection or ordering of the unit cell vectors and everything will change in an uncontrolled way.

The conditioning approach seems to be to provide desired properties as inputs to the generative approaches. I am not sure this always make sense. For example, if one desires a certain space group, there is no enforcing compliance with the space group. One can always easily check it. It is perhaps more suitable to enforce space group compliance using a guidance-based conditioning approach.
questions: questions: See above.
limitations: limitations: There is insufficient discussion about the limitations given my concerns shown above.
paper_summary: paper_summary: The paper addresses the inverse problem of generating crystal structures based on given properties, thereby avoiding the need for extensive computational resources typically required in traditional methods. The authors utilized the AFLOW materials database, selecting unstable and stable series of structures for two specific tasks: modifying structures to achieve stability and conditional structure generation.
strength_weakness: strength_weakness: 1. From the perspective of model application, although the authors used the AFLOW database for their study, they did not compare the data range and coverage with other significant databases like the Materials Project. This omission leaves a gap in understanding how the generative models perform across different datasets and whether the results are consistent and generalizable. Comparing the performance of the same generative models on different databases could provide valuable insights into the robustness and applicability of their approach.

2. There is a partial break of anonymity in the GitHub link on Page 9 in this paper.
questions: questions: 1. In Section 2.2, the authors process the dataset for the structure modification task by training unstable structures towards stable ones. However, unlike typical trajectory datasets for structure optimization, the authors select the most stable polymorph of the same chemical formula as the target and use the remaining polymorphs as initial structures to complete the structure modification task. This dataset approach leans more towards converting between different polymorphs rather than optimizing the stability of an arbitrary atomic configuration as stated by the authors in the abstract. The authors could consider explain the relationship between using this part of the dataset and the stated objective in detail.

2. In terms of the generation task, the authors generate structures by specifying formation energies. However, in Section 6, the authors only list a metric related to formation energy without providing a detailed discussion on other important aspects, such as the effectiveness of other condition controls, the validity of the generated structures, and whether duplicate structures are generated. These issues seem to remain unanswered.
limitations: limitations: The authors proposed two major directions: conditional generation and conditional modification. There is room for improvement in both the experimental results and the data used for conditional modification. For example, they could consider recognizing unit cells with translational and rotational transformations and introducing more ways to assess generation results.
paper_summary: paper_summary: The authors studied the use of diffusion and flow matching approaches for the generation of crystalline materials. The authors trained UNet models on polymorphs in the AFLOW database (which has a series of DFT-computed properties for these materials) using either simple R3 regression, diffusion/flow matching. The authors then presented inference results on similarity to training structures (Section 3.4, which shows these methods can reproduce training structures to different extent), and showed that a subset of the modified generated crystals (with Ta, W, B) can have a small non-zero formation energy.
strength_weakness: strength_weakness: *Originality*. The manuscript lacks originality. The diffusion/flow matching techniques are well-established in inorganic crystal structures (e.g. CDVAE cited here, DiffCSP/FlowMM that's not here). Sure, using a network architecture not designed for materials/crystals and using no invariances/equivalences is new, but it deviates from standard practices in the field without sufficient justification. I believe the implementations shown in the paper are a great exercise for practitioners interested in the field, but unfortunately, I do not see it as a NeurIPS paper.

*Quality*. The manuscript is _very_ bare-boned, making a comprehensive technical critique challenging without appearing disproportionately critical. 
- On the ML side, there are numerous large fallacies/mistakes (e.g. no consideration of bonds between atoms at all, Sec. 3.3 there is no description of the PBC loss, the generation does not consider the unit cell, no generation with atom types, and there is no investigation of any experiments observed e.g. why is uniform noise better for CFM, the result in Table 1 appears to evaluate overfitting rather than novel generation, the list can go on). 
- On the chemistry/validation side, there are again numerous problems (why would formation energy be given during the generation process, what functional did you use in DFT, there are no comparisons against existing structures and hence cannot be claimed as novel, etc.) 
- There is no comparison against _any_ known methodologies. 
- The results overall, are very weak both in ML and in chemistry (e.g. Table 3 shows most if not all materials generated have extremely large positive formation energies despite the simple elemental composition; the remaining few negative ones are at the brink of instability, in any case they likely would not be synthesizable).  

*Clarity*. The manuscript suffers from poor presentation, starting with a promotional-style title that lacks scientific descriptiveness. I unfortunately do not understand the novelty of the paper in comparison to existing methods. The paper consistently fails to provide essential explanations across both machine learning and chemical methodologies. 
- On the ML side, there are numerous things poorly presented (e.g. Figure 1 is just periodic translation invariance and in a typical manuscript would be summarized in one sentence). 
- On the chemistry side, things are greatly exaggerated (e.g. computationally making a few materials with negative formation energy can be done by undergraduate students and certainly does not warrant descriptions such as 'This significant outcome underscores the remarkable potential of our framework in uncovering thermodynamically stable materials)
questions: questions: Unfortunately, without significant innovations and revisions, I do not believe I can convinced this paper would be accepted in NeurIPS.
limitations: limitations: Partially.
paper_summary: paper_summary: This paper presents a framework for crystal structure generation, focusing on polymorphs. The framework utilizes matrix representation of crystals and various generative models used in vision tasks, with specially designed similarity metrics and loss function. It is tested on (1) modification of given structures and (2) generation from scratch within the dataset, as well as finding new structures in the Ta–W–B system.
strength_weakness: strength_weakness: - In the demonstrated use cases, the generation is conditioned on elements, space group, etc., but not materials properties of interest. These show limited usefulness in materials discovery and design.
- The matrix representation does not take physical constraints into account, e.g., space group determines symmetries in the lattice parameters. Besides, related previous works, e.g., [UniMat](https://openreview.net/forum?id=wm4WlHoXpC), should be discussed.
- The clarity and rigorousness need to be improved (see Questions). The mathematical notations are not unified, e.g., $x$ vs $X$.
questions: questions: - Line 53, “both pipelines” are not introduced. The previous paragraph is also not unclear, please check the grammar.
- In Line 71, do “conventional limitations” mean those of conventional approaches in materials science, or previous ML/generative methods? Not just how the proposed method differs from previous ones, but also what challenge it overcomes, should be specified.
- In the descriptions of the matrix representation, the role of elemental properties is unclear. My understanding is that they are not part of the data structure to be generated. Where and how are they used? A probably related question is, what is $el_{emb}$, and how is it obtained?
- Line 207, what is the role of time condition t, and why does it matter for materials discovery?
- The modification task shows significantly lower metrics than the generation task. What does this imply? Does this indicate generation is preferred over modification? If not, what are the scenarios where modification can be useful, and how to mitigate the performance?
limitations: limitations: Discussed in the Conclusion.
Besides, Sec. 9 contains a GitHub link that could break anonymity.

Comments: comment: Thank you for your detailed response and the clarifications provided. I appreciate the effort you've put into addressing the concerns raised. However, after careful consideration, I will maintain my initial assessment. I believe the points discussed are valuable for future work and manuscript revisions. I genuinely appreciate the hard work and dedication of the authors.
I very appreciate the authors for the detailed response. I am willing to raise my score to 3 and revise my soundness/contribution scores. While the rebuttal indeed clarifies a few things, the key ML problems (technical novelty, lack of baselines, lack of comparable tasks to other papers) and chemistry problems persist.

Re chemistry problems, I just want to clarify one point regarding DFT calculation: There is still no description of functional and k-point mesh used in VASP, VASP is a package that accommodates many good and bad functionals. In fact it is a very delicate package where small changes to parameters can very much change the results. In this sense, formation energy is very easy to manipulate in general and hence not a useful objective in computational crystal design (and even less so experimentally)
Thanks for the explanation. I'll keep my score.
The authors' response has addressed my clarifying questions. However, some issues are not addressed or are inherent limitations of the proposed method.
1. Physical constraints and symmetry. Including space groups as conditions does not ensure the generated matrix follows the required symmetry.
2. Time conditioning. My main concern is how $t$ relates to materials science applications. To use this generative model for materials discovery or design, how should one set $t$?

With the confusing expressions properly addressed, I could raise the Presentation score to 3, however, the lack of physical constraints limits the Contribution and Soundness.