MatExpert: Decomposing Materials Discovery By Mimicking Human Experts
Authors: Qianggang Ding
Santiago Miret
Bang Liu
Venue: ICLR 2025 Poster (2025)
Keywords: Material Discovery
Large Language Models (LLMs)
Material Generation
Crystal Structure Generation
Contrastive Learning
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We present MatExpert, a novel framework leveraging Large Language Models (LLMs) and contrastive learning to streamline material discovery like human expert workflows.

Abstract: Material discovery is a critical research area with profound implications for various industries. In this work, we introduce MatExpert, a novel framework that leverages Large Language Models (LLMs) and contrastive learning to accelerate the discovery and design of new solid-state materials. Inspired by the workflow of human materials design experts, our approach integrates three key stages: retrieval, transition, and generation. First, in the retrieval stage, MatExpert identifies an existing material that closely matches the desired criteria. Second, in the transition stage, MatExpert outlines the necessary modifications to transform this material formulation to meet specific requirements outlined by the initial user query. Third, in the generation state, MatExpert performs detailed computations and structural generation to create a new material based on the provided information. Our experimental results demonstrate that MatExpert outperforms state-of-the-art methods in material generation tasks, achieving superior performance across various metrics including validity, distribution, and stability. As such, MatExpert represents a meaningful advancement in computational material discovery using langauge-based generative models.

Published as a conference paper at ICLR 2025
MATEXPERT: DECOMPOSING MATERIALS DISCOVERY BY MIMICKING HUMAN EXPERTS
Qianggang 2 Santiago Miret', Bang Liu DIRO & Institut Courtois, Université de Montréal 2 Mila Quebec AI Institute Intel Labs
fqianggang. ding, bang. iuj wwwnttreal.... santiago. miret& (intel. com
ABSTRACT
Material discovery is critical research area with profound implications for var- tous industries. In this work, we introduce MatExpert, a novel framework that leverages Large Language Models (LLMs) and contrastive learning to accelerate the discovery and design of new solid-state materials. Inspired by the workflow of human materials design experts, our approach integrates three key stages: re- trieval, transition, and generation. First, in the retrieval stage, MatExpert iden- tifies an existing material that closely matches the desired criteria. Second, the transition stage, MatExpert outlines the necessary modifications to transform this material formulation to meet specific requirements outlined by the initial user query. Third, in the generation state, MatExpert performs detailed compu- tations and structural generation to create new materials based on the provided information. Our experimental results demonstrate that MatExpert outperforms state-of-the-art methods in material generation tasks, achieving superior perfor- mance across various metries including validity, distribution, and stability. As such, MatExpert represents a meaningful advancement in computational mate- rial discovery using langauge-based generative models. Codes are available at: https::/ com -aw.o//Bnnnaa--dde------------ MineMMMilll Mila Mat Expert.
INTRODUCTION
The discovery and design of new materials are central challenges in modern materials science, driven by the need for materials with tailored properties for applications energy, electronics, and catal- ysis. Traditional methods for material discovery, such as hightthroughuu experiments and density functional theory (DFT) simulations, are computationally expensive and often require significant domain expertise achieve accurate predictions (Miret al., 2024). Recent advancements in artifi- cial intelligence (AI), particularly large language models (L..Ms), have opened new possibilities for automating and accelerating the materials design process (Miret & Krishnan 2024 Jablonka al., 2024 Song et al. 2023a
Zhang et al. 2024 Ramos et al. 2024).
LLMs such as GPT-4 OpenAl 2023) have demonstrated remarkable success in natural language processing tasks and have shown potential for application in scientific problems beyond language, including chemistry and materials science FllamSShepherd & Aspuru-Guzik (2023); Gruver et al.) 2024); Schilling- Wilhelmi
-chilling-Wilhelmi et al. (2024); Mirza et al. [ et al. (2023). For example, I.MMs have been used to generate molecular structures Gruver et al. (2024) and predict material properties from textual descriptions Alammaraa et al. (2024). In the context of material generation, models like Crystal Diffusion Variational Autoencoders (CDVAE) Xie et al. (2022b), as well as fine-tuned LLMs such as LLaMA-2 Gruver et al.] (2024), CrystaLLM Antunes et al. (2023), and LM-AC/LM-CH, have made significant progress in generating crystal structures directly from Crys- tallographic Information Files (CIFs) Antunes et al. (2023). Together, these models aim accelerate materials design by predicting stable structures and optimizing key properties, such as energy above hull for stability Riebesell et al. (2023).
"Corresoonding Author. Canada CIFAR AT Chair.

---

Published as a conference paper at ICLR 2025
Despite the significant progress made in applying Al models to materials science, current methods for material generation still face critical limitations
Mimta Miret & Krishnan (2024); Alampara et al. (2024); Mirza et al. (2024). Most approaches generate material structures in single step, often resulting in static outputs that lack the flexibility for iterative refinement and optimization based on intermediate feedback. Moreover, these models struggle to integrate multimodal data-such as textual descrip- tions of desired properties and structural information like atomic coordinates from Crystallographic Information Files (CIFs)-in meaningful and cohesive way. Consequently, they fall short of repli- cating the complex, reasoning -rriinn processes employed by human experts, limiting their ability to generate materials that meet specific target properties.
In contrast to AI models, human experts follow a methodical, multi-step process when designing new materials. They begin by identifying an existing material with properties closely matching the desired specifications. From there, they apply series of logical modifications- atomic arrangements or chemical compositions sive refinement do they finalize the structural details, ensuring that the material is both feasible and aligned with the target properties.
as altering iteratively refine the material. Only after exten-
To overcome the limitations of current models, we propose MatExpert, a framework that mimics this expert-driven workflow by breaking down the material generation process into three stages: retrieval, transition, and generation. Using a text-structure retrieval approach combined with chain-of- chain-of-thought reasoning Wei et al. 2002).. MatExpert can iteratively refine materials hased on intermediate feed- back, leading more accurate and interpretable material designs that meet desired specifications.
To comprehensively assess the performance of MatExpert, we assembled a large-scale dataset from the NOMAD database Scheidgen al. (2023), which contains total of 2, 886, 120 materials. This extensive evaluation on a large-scale testbed demonstrates the robustness and generalizability of MatExpert in addressing real-world materials discovery challenges.
Concretely, our paper makes the following contributions:
We propose MatExpert, a novel framework that mimics the expert-driven workflow for material discovery by decomposing the generation process into three stages: retrieval, tran- sition, and generation. By leveraging a text-structure retrieval mechanism and a chain-of- thought reasoning approach, MatExpert enables the design of accurate and interpretable materials.
2. We curated a large-scale dataset from the NOMAD Scheidgen et al. (2023) database, which serves comprehensive testbed for evaluating MatExpert across diverse material com- positions. We will publicly release the datasets and source codes upon publication.
3. We conducted extensive experiments on both the Material Project et al. (2013) and the curated NOMAD datasets. Our results demonstrate that MatExpert significantly outper- forms state-of tht-hhhrrt art bascline methods in material generation, showcasing its effectiveness and generalizability.
RELATED WORK
The use of AI and large language models (LLMs) in materials sci- ence has gained traction as a means to accelerate material discovery, which traditionally relies computationally intensive methods like DFT simulations Duval et al. (2023); Govindarajan et al. (0024).
Caneraga
Compontiion
CTAM
Miret& & Krishnan (2024) provide an extensive review of the limi- tations of current L.Mss in materials science, particularly their in- ability to handle multi-modal data and iteratively refine structures
Danaity Distavope
Atoon Oistanoe
based on feedback.
Early work, such as the Crystal Diffusion Variational Autoencoder (CDVAE) Xie et al. (2022b), generated crystal structures by incor- porating physical stability constraints to produce valid periodic ma- terials. Recent work has expanded on CDVAE's approach by ploring alternative representations for diffusion-basee models Jiao et al. (2024); Yang al, (2024); /Zeni al. (2023), as well as flow-
Figure 1: MatExpert achieves remarkable performance on all metrics, especially metrics of distances. See Table for details.
2

---

Published as a conference paper at ICLR 2025
Property Description
Bntso ies description balk material:
The formation energy stomis 1.8375.
The band pap is 8.2133.
Property Embeddings
The energy sbove the conves hullia 0.062.
T5 Encoder
Minimize
Structure Description
Distance
NooMMa structured Cosemlisdeeriee-eriiee
Na,MnCoNiO,
thres ineuutrainnt Na altes. firt aits, Nact) bonded to one OE2, one O30) Taoo equilalent 06%, and two equicilent....
Structure
TS Encoder
Embeddings
We utilize contrastive learning framework to train encoders for reference material retrieval. For given sample material (e.g., Na;MnCCNiiO, we extract both its property description and structural description using MyMggg Ong al. (2013) and Robocrysttallogrphee Ganose & Jain (2019), respectively. The model employs two 15-based encoders Raffel et al. (2020). which are trained minimize the distance between these two representations.
Figure 2:
matching models Miller et al. (2024). Further advancements in-
clude the fine-tuning of LLMs for materials generation. Gruver et al. (0244) fine-tuned LLaMA-2, improving metastable material generation rates, while CrystaLLMM Antunes et al. (2023) employs an autoregressive model trained on CIF files generate inorganic crystal structures, leveraging Monte Carlo Tree Search for refinement. Flam-Shepherd & Aspuru-Guzik (2023) proposed LM-AC and LM-CH, which explore generating 3D molecular structures directly from CIF, XYZ. and PDB for- mats, pushing the boundaries of LLM upplications in material science. In addition to generative models for materials discovery, as well the development domain-spectio
LLMs for diverse lan- guage tasks materials science (Song et al., 2223b a; Xie et al.) [023 Zhang et al. 2024). LLMs are also being used automate the extraction of structured data from scientific literature, streamlining the creation of datasets for machine learning Hira et al. (2024); Mishra et al. (2024): Schilling-
Wilhelmi et al. (2024).
In summary, while LLMs show great promise, challenges remain in integrating multi-modal data and refining material designs iteratively. Most prior work has focused on unconditioned material generation, but current methods lack the ability to iteratively refine and reason like human experts. Our MarExpert framework addresses this shortcoming by mimicking an expert-driven workflow, employing multi-stage iterative refinement through retrieval, transition, and generation processes, and achieves remarkable performance compared to state-of-the- methods, particularly in metrics of distances (see Figure D
METHODOL.OGY
Our MatExpert framework simplifies the complex process of material discovery by emulating hu- man experts through three key stages: retrieval, transition, and generation. First, we retrieve a reference material from database that closely aligns with the desired properties. Next, using fine-tuned large language model (LLM) with Low-Rank Adaptation (LoRA) Hu et al. (2022), we generate insights how modify the reference material to meet the target specifications. Finally, we generate the new material's 's structure based on the reference and transition steps. The framework employs a T5-based Raffel al. (2020) model for the retrieval stage and fine-tuned LLM for the transition and generation stages. A comprehensive illustration of the MatExpert framework pipeline shown in Figure
3.1
FIRST STAGE: RETRIEVAL.
The retrieval stage is the foundational step of our framework, aimed at identifying the material in the database that best matches the desired property description provided by the user.
To bridge the gap between the representation of the desired properties and the relevant material, we apply contrastive learning. Contrastive learning Chen et al. (2020) is a self-supervised method that learns representations by bringing positive pairs closer together while pushing negative pairs farther apart in the embedding space. Contrastive learning was first applied in the domain Radford et al.) 200T): (Singh al. (2022), recent works have successfully applied similar methods to domains to
3

---

Published as a conference paper at ICLR 2025
Below descripttion buik materfal: formation
1,438. The specegroup
1,1081.
hull
(Stage I: Retrisval) The most similar material related to the description in the databass in Ni,, Min.MiCoNio,
Ns3MnCoNiOGit is Capwelisliverte derived structured and crystallizes in monoclinie space group, three insquivalent sites, the first site, Nu(1) is bonded one O(3), one O(é), two equiveient 0111, and two equivalent OSS) atoms form distorted NaOS octahedre share coreers with three equivalent Mot1)00 actahedru, comers three...
should modity NacMnCoNiOS to develop new material with desired properties?
(Stage 2: Transition) transit from NaaMrCoNN06 new material, process would involve altering the cation compoattion the crystul structure while maintaining the metal and oxygen framework. Specificalit, sodium [Nu) stoma NidMrCoNiOS, which exist in configuration forming various bonds angles in monaclinic Cm space group,
Based Information, vectors and then the element type and coordinates
new material, could you generate description lengths and angles stom
lattics?
(Stugl 3: Guneration) [ALX representationn 5.1 S.n so 120 28 www.m/iiiid..nn.. wwtp://w.........oo...ooooooo... 8.2n 0. 0.21taMrivs0.422 0.0000 www....0....1ne.. www.://nnneedddddee 26 MiMMiun do 0.7thnDusd. www.00000.....00w...ooo..omoo 0.28 www.://www....000000o.00..oooo 0.8dhvnknn..000.0 0.93taDind. 1.0.80 0.80 w.w.sssinn... 0.44 wat.ninnn... 0.48
Ground Truth:
Ground Truth Material
GPT44
Guraratton
Pathwaa
Superclsed
Fouatura
LLamna LLM
Fiveture
Supervised
Material Database Siricture
Cenerata
Gerverate ate
PyMargmn
Generated Material CI representation
Retrieved Material
Transition Puthway
Generated
Properties
Figure 3: Pipeline of MatExpert: Given description of the desired material, MatExpert first trieves the most similar material from the database (e.g., NayMnCoNiOg)) Next, the LLM provides transition pathways modify the retrieved material into the desired material (e.g., replacing Na with Li). Finally, the LLM generates the detailed structural information of the desired material (LigMnCoNiOss). See Figure for full case of conditional material generation.
materials design, such molecular Liu et 2022) and protein modeling Xu et al. (2023) In the context of material retrieval, we use this method to embed material properties and structures in shared space, enabling the efficient retrieval of materials that closely match specific queries.
To implement this, we employ a contrastive learning framework (see Figure 2}. T5-based encoders to process and embed both property and structure descriptions. The property descriptions include key characteristics such as composition, space group number, formation energy, band gap, and energy above hull. Structural descriptions are extracted using RoboCrystallographer, which provides detailed linguistic descriptions of the structure for each material.
using two parallel
The primary objective is to ensure that the embedding of the property description (q:) is closely aligned with the embedding of the corresponding structure description (m;). while distinctly sep- arating from embeddings of other materials (mæ). To quantitatively enforce this alignment, contrastive loss function is defined as:
log exp(sim(q,, m:)/T) Σ exp(sim(q; (a)ssimmmmm))/) m²)
Here, (im m,) denotes the cosine similarity between the property and structure embeddings, is a temperature parameter that controls the sharpness of the distribution, and N is the size of negative sampling. The output of this stage the best matching material, which serves as candidate for further refinement.
3.2 SECOND STAGE: TRANSITION
The transition stage focuses developing detailed and viable method for modifying the retrieved material to meet the desired properties. This stage bridges the gap between the existing material and the target material by outlining specific structural compositional changes required to achieve the desired specifications.
To build model capable of generating transition pathways that describe how transform the exist- ing reference material into the desired target material, two key steps are necessary: (1) constructing training dataset consisting of <source material, transition on pathways, target
4

---

Published as a conference paper at ICLR 2025
material> triples, and (2) fine-tuning model that produce transition pathways given am source material as a reference.
In this stage, utilize two LLMs in sequential process. First, during the training phase, ploy GPT-4 to generate ground-trune truth transition pathways, which provide comprehensive. step-by- step instructions for transforming the retrieved material to meet the target properties. Subsequently, we fine-tune a second, smaller LLM using these ground-truth pathways. In practical applications, including our experiments, this fine-tuned LLM is deployed generate the potential transition path- ways, offering more efficient solution while maintaining accuracy.
During the training phase, utilize GPT-4 to generate detailed transition pathways by providing it with carefully designed prompts. These prompts are constructed to ensure that GPT-4 receives all necessary information about both the source material and the target material, enabling it to produce accurate and comprehensive modifi- cation pathways. An example template of such a prompt is:
Ground-Truth Pathway Generation with GPT-4:
Cormula_sourees Based on the descriptions and properties of the two materials below, can you summarize the main reasons for the differences properties when transitioning from <formula_source> to <formula.target>? The description of <formula.source>; <description.souree source. The description of <formula.targeta <descripttio.ttrrget>
I have two materials:
and <formula.targes
The ground-trute -ouuth transition pathways generated by GPT-4 form the basis for the supervised fine-tuning of a second LLM. This model is trained to learn and replicate the detailed modification steps provided by GPT-4, allowing it to generate potential transition pathways in practical applications. An example prompt template used for fine-tuning is:
Fine-Tuning the Second LLM:
am looking to design a new material with the following properties: <property.list>, The closest existing material I found in the database is <formula.source>, which has similar properties. Below the structure description <formula.souree>, In <description.souret |n How should I modify <formula source> develop new material with the desired properties?
3.3 THIRD STAGE: GENERATION
The generation stage is focused on producing the CIF (Crystallographic Information File) represen- tation of the predicted material. Specifically, we first generate detailed description of the lattice vectors and atomic coordinates, which we refer to as the ALX representation. This ALX representa- tion is then automatically converted into CIF format using the pymatgen library, which serves as the final output of our framework.
The generation process builds upon the conversation initiated during the earlier stages of the framework. The output from the transition stage serves as input for the generation stage. An additional prompt is introduced to extend the conversation, guiding the model to generate the ALX representation based on the transition pathway provided the previous stage. The prompt for the generation stage is:
Fine- Tine-Tuning:
Based on the information, for the new material, could you generate description of the lengths and angles of the lattice vectors and then the element type and coordinates for each atom within the lattice?
ALX Representation: After fine-tuning, the LLM generates the ALX representation for the mate-
rial based on the transition pathways (as shown in the final response in Figure This representation includes:
The lengths and angles of the lattice vectors that define the unit cell of the material.
Lattice Vectors:
5

---

Published as a conference paper at ICLR 2025
Atomic Coordinates: The precise atomic types and their coordinates within the lattice, ensuring that the generated structure aligns with the desired material properties.
Final Output in CIF Format:
Once the ALX representation is generated, it is converted into CIF format using the pymat library. The CIF file contains the complete structural information of the predicted material and serves as the final output of the generation stage. This file can be utilized for further computational studies, experimental validation, or integration into material databases.
4 DATA COLLECTION & PRE-PROCESSING
To rigorously evaluate the performance of MatExpert, we assembled comprehensive dataset from the NOMAD database (ScaasigeSeeeennnnnna et al. (2023), utilizing its API. This extensive dataset serves as the large-scale testbed for assessing the capabilities of our framework.
Data Collection from NOMAD NOMAD Scheidgen et (2023) a vast repository of material data. For our study, we refined the dataset by excluding materials with inert elements to focus on chemically active compounds. We also restricted entries to those simulated using VASP Kresse & Hafnr (1993) for high-quality computational data and excluded structures with over 30 atoms to enhance computational efficiency.
By applying these filtering criteria, we obtained a dataset of 2, 886, 120 materials. This large-scale dataset provides a rich foundation for training and evaluating our models, enabling us to explore a wide range of material compositions and properties.
Pre-processing Each filtered material converted into CIF format, ensuring the structural in- tegrity of the original data was preserved. CIF files are widely used materials science to represent crystal structures and compatible with various computational tools, facilitating seamless integra- tion into our framework.
In addition to structural data, we enriched our dataset with linguistic descriptions of each material, generated using ToboCrystallographer [annose & Jain (2019), a tool that automatically produces human-readable summaries of crystallographic information. This step allows for the integration of language-baad baauag processing within our framework, leveraging the strengths of large language models. We also extracted key material properties such formation energy, total energy, and elemental composition-usse the M3GNet Chen & Ong (2022) and PyMatgen Ong et al. (2013) libraries in Python. These properties provide essential inputs for condition-bar based material design and evaluation within MatExpert.
By leveraging this meticulously curated dataset, we ensure that MarExpert is evaluated robust and diverse set of large-scale materials, facilitating comprehensive assessments of its performance and effectiveness.
5 EXPERIMENTS
5.1 DATASETS AND BASELINES
Material Project: We leverage datasets from the Materials Project database similar to prior stud- ies Xie et al. (2022b); Gruver et al. (2024). The MP-20 dataset (Xie et al.) (002b) for unconditional generation contains 45,231 stable materials, filtered to exclude structures with over 30 atoms per unit cell for computational efficiency. The datasets are exclusively used for the unconditional generation task.
NOMAD: We also collected a large-scale dataset from the NOMAD database, consisting of 1958,355 materials. The dataset filtered to exclude structures with inert atoms and more than 30 atoms per unit cell, and only includes structures computed using VASP in the database. The datasets are exclusively used for the conditional generation task.
To evaluate the effectiveness of MatExpert, we compare its performance against the following state- of-the-art baselines in material generation.

---

Published as a conference paper at ICLR 2025
Table 1: MatExpert shows general outperformance compared to baseline methods for unconditional
generation on Materials Project based on various metrics like validity checks, coverage. property
distribution, and stability. The best results are in bold, while the second-best results are underlined.
Method
Validity Check
Coverage
Distribution
Metastable M3GNett
Stable
Struct
Compt
Reef
Prect
wdist(p)) wdist(Nai))
DFTT
CDVAE
1.00
0.867
0.991
0.995
0.68
1.43
28.8% n/a n/a
5.4%
LM-CH
0.848
0.835
0.992
0.978
0.86
0.13
LM-AC
0.958
0.889
0.996
0,985
0.69
0.09
n/a
n/a
Crystal-LLM with
LLaMA-2
7B (T 1.0)
0.918
0.879
0,969
0.960
3.85
0.96
35.1%
6.7%
7B (T = 0.7)
0.964
0.933
0.911
0.949
3.61
1.06
35.0%
6.2%
70B (T = 1.0)
0.965
0.863
0.968
0,983
1.72
0.55
35.4%
10.0%
70B (T 0.7)
0.996
0.954
0.858
0.989
0.81
0.44
49.8%
10.6%
MatExpert
LLaMA-2
7B (T = 1.0)
0.920
0,908
0.984
0.994
0.49
0.11
37.5%
7.8%
7B (T = 0.7)
0.946
0,943
0.952
0,986
0.51
0.13
36.2%
8.0%
70B (T 1.0)
0,968
0.878
0.982
0.986
0.23
0.06
50.2%
11.3%
70B (T 0.7)
0.998
0.959
0.969
0.993
0.23
0.07
50.9%
1 11.8%
MatExpert with
LLaMA-3
8B (r 1.0)
0.933
0.910
0.992
0.995
0.36
0.07
39.1%
8.1%
8B (T = 0.7)
0.975
0,959
0.988
0.990
0.38
0.10
39.5%
8,4%
70B (T 1.0)
0.970
0,880
0.980
0.988
0.21
0.05
50.5%
11.0%
70B (T 0.7)
0.998
0.961
0.986
0,991
0.18
0.04
51.0%
12.0%
Table 2: Comparison results of conditional generation on NOMAD datasets with MatExpert outper- forming baseline methods. The best results are bold.
Method
Validity Check
Coverage
Distribution wdist(p))
Struct
Compt
Reet
Preet
wdist(N.)1
Crystal-LLM with LLaMA-2
70B (r 1.0)
0.983
0.892
0.979
0.984
1.63
0.53
70B (T 0.7)
0.997
0.971
0.873
0.988
0.77
0.42
MatExpert with
LLaMA-2
70B (r = 1.0) 0.991
0.913
0.986
0.992
0.18
0.08
70B (r 0.7)
0.998
0.981
0.982
0.994
0.21
0.08
MatExpert with
LLaMA-3
70B (r 1.0)
0.996
0.922
0.991
0.997
0.15
0.08
70B (T 0.7) 0.999
0.986
0.989
0.985
0.14
0.09
Crystal-LLM Gruver et al. (2024): A state-of-the-art approach fine-tuning large language models (LLaMA-2 Touvron al. (2023) ) for generating stable inorganic materials. It excels in generating valid crystal structures and serves as primary baseline due to its pioneering use of LLMs for material generation.
CDVA Xie et al. (202bb): A Crystal Diffusion Variational Autoencoder tailored for periodic ma- terial generation. It combines generative models within continuous VAE latent space, widely used for generating stable materials and considered a strong benchmark.
LM-CH and LM-AC -lam-SSepheerd & Aspuru-Guzik (2023): Transformer- models for 3D crystal structure generation. LM-CH uses character-level tokenization, while LM-AC uses atom and coordinate- tokens.
For a fair comparison, MatExpert and baselines are trained on the same datasets for each experiment.
5.2 UNCONDITIONAL GENERATION
In the unconditional generation task, we aim to assess the ability of MatExpert to produce novel and stable material structures without any specific property constraints. This stage serves as a benchmark for evaluating the intrinsic generative capabilities of the model, focusing the validity, diversity, and stability of the generated materials. For unconditional generation, we randomly select material from the database during the first stage of MatExpert.

---

Published as a conference paper at ICLR 2025
The comparison results shown in Table | As evident from the table, the MatExpert family with LLaMA-3 (Dubey et al., 2024) outperforms all other methods across all metrics except metrics of coverage. Traditional models such as CDVAE, LM-CH, and LM-AC excel in coverage and structural validity. When compared to the state-of-the-ar LLLM-based model Crystal-LI.M, our MatExpert method consistently outperforms Crystal-LLLM across all setrings, including the same model size and temperature as well as DFT-based stability. Notably, MatExpert achieves remarkable results in distribution metries, benefiting from the transition from existing material in the database. While Crystal-LLMM may struggle with the hallucinations of LLM, thus generating terials.
ma-
5.3 CONDITIONAL GENERATION
In the conditional generation task, we evaluate MatEx- pert's ability to generate material structures that meet spe- cific property constraints using the large-scale NOMAD dataset. This task crucial for practical applications, where researchers often require materials with targeted properties, such as specific band gaps, space groups, and chemical compositions, etc. As shown in Table 2 MatEx- pert significantly outperforms the baseline Crystal-LLLM for all metries. MatExpert also excels in property dis- tribution, achieving much lower Wasserstein distances, which reflects its ability to generate materials with prop- erty distributions closely aligned with the training data. Notably, the stability metric like the energy above the hull is treated as an input property in this task and is not separately evaluated as performance metric.
100%
Condition
Satisfaction
80%
60%
61%
76%
57%
69%
52%
Crysss-LMM
66%
MelExpert
5640%
40%
31%
20%
0%
110
5.29
Space group
(oomala)
Pormuta]
Figure 4: Conditional satisfaction rates of common property constraints. Ma- tExpert consistently outperforms base- line methods.
To rigorously evaluate the effectiveness of MatExpert satisfying user-defined property constraints, also conducted additional experiments aimed at quantifying the percentage of generated materi- als that meet specific property criteria set by users.
We selected set of common property constraints, including:
Formula: We parse the composition from the generated CIF. We show the results in three
levels by atom numbers: < atoms, < 10 atoms, and < 15 atoms.
Energy above hull: We use M3GNet & Ong (2022) to estimate the energy above hull.
We bin stability as Ehull < 0.1 for metastable and Phull > 0.1 for unstable.
Space Group Number: We use PyMatgen's SpacegroupAnalyzer with a precision of 0.2 angstroms. Ong et al. (2013)
For each constraint, we generated 10,000 materials using MatExpert and the baselines. We then calculated the percentage of materials that met the specified property criteria. The results of the con- dition satisfaction experiments are summarized in Figure MatExpert consistently outperformed the baselines across all property constraints, demonstrating its superior capability to generate mate- rials that meet user-defined specifications.
ANALYSIS
6.1 ANALYSIS OF DIVERSITY AND NOVELTY
To evaluate the MatExpert framework, we use metrics that assess the diversity and novelty of gener- ated materials. Following Xie al. (2022b), diversity is calculated as the pairwise distance between samples using a featurization of structure and composition. Novelty is determined by comparing the distance to the nearest element of the training set for each sample, with a sample considered novel if this distance exceeds threshold. We also assess the overall novelty, defined having either new structure or composition. To further understand the gaps between MatExpert and baselines, we illustrate the score of each metric normalized to testing samples in Figure As we can see, Mat- Expert consistently achieves high-level scores in both structure and composition diversity compared to CDVAE and various Crystal-LLLM configurations. This indicates MatExpert's superior ability
8

---

Published as a conference paper at ICLR 2025
Tost
0.5
0
Structure Diversity Composition Diversity
Structure Novelty
Composition Novelty
Either Novelty
ODVAE
Crysstal-LLM (r-1.0)
CrysttllLLM (r=0.7)
Crysla-LLM 138 (r=1.0)
Crystal-LLM 130 (m=0.7)
Cryslai-LM 70B (r1100
Crysta-LLM 70B (r-0.7)
MatExperr 8B (t=1.0)
MalExpert 88 (r=0.7)
MolE spert 708
MatExpert 708 (r=0.7)
Figure 5: Scores of diversty and novelty normalized testing samples. MatExpert consistently achieves remarkable scores on all metrics. All socres are calculated on the generated materials passed the validity.
to explore a broad chemical space. terms of novelty, MatExpert demonstrates a strong capacity to generate structures and compositions not found in existing databases, evidenced by its high novelty scores. Notably, Crystal-LLM faces challenges in balancing model size and novelty, with larger models exhibiting lower novelty. In contrast, MatExpert consistently maintains high novelty regardless of model size.
6.2 ABLATION STUDIES
To understand the contribution of different compo- nents of our framework, we conduct ablation stud- ies by systematically removing modifying key el- ements of MatExpert and observing the impact on performance.
Table Choices of encoders in retrieval stage. We use top-1 and top-5 accuracy, along with the average rank of the ground- truth materials metrics.
Impact of Transition (CoTT) Stage The transition stage, which involves generating modification path- ways using Chain of Thought (CoT) reasoning Wei
Accuracy
Encoder
Avg Rank
Top-1 (%)
Top-5 (%)
Bert
59.96
84.51
2.36
al. (2022), plays a crucial role the overall per- formance of MatExpert. To evaluate the impact of this component, we conducted experiments where the CoT reasoning was removed entirely. The results in highhigt its efffcctiveness of CoT reasoning.
(ours)
71.35
93.54
1.59
Impact of Retrieval Stage We assess the impact of the retrieval stage with the contrastive learning framework. By removing the retrieval stage, we ob-
80%
serve noticeable decline in the quality of the
Sallsfaction
(foruue))
trieved materials (See Figure 6). This confirms that
Condition
60%
atoms (formua))
atoms (formula)
the retrieval stage is crucial for identifying high-
40%
Space group
quality reference materials, which are essential for effective subsequent modifications.
30%
Energy above hull
20%
MatÉxpert
w/o CoT
wio Retrieval Got
Choices of Encoders in Retrieval Stage In the re- trieval stage, we explored various encoder architec- tures to determine their impact on the quality of ma- terial retrieval (See T)ble(3). We compared T5-based encoders with BERT Devlin et al. (2019). Our find- ings indicate that T5-based encoders consistently outperform BERT, providing more accurate embed- dings for both property and structure descriptions, making it more effective for capturing the most similar material aligned with the desired properties in the database.
Figure 6: Impact of Transition (CoT) and Re- trieval Stages. We sequentially remove the transition (CoT) stage and the retrieval stage to demonstrate changes in condition satisfac- tion rates.
6.3 CASE STUDY
To showcase the capabilities of MatExpert, present comprehensive example of conditional generation under property constraints in Figure 7 This case illustrates the process of generating a predicted material and its adherence to the specified properties.

---

Published as a conference paper at ICLR 2025
am looking to design new materiat with the following pppertty The formation energy atorm The energy above the convex hull 0.o. The elements Nu, S, Ti, The spacegroup number B.
is -1.4661. The band gap is 8.0.
(Stage 1: Retrieval) The most similar material related to description in the database is My5oV84.
Below is the structure desoriptio of MgsoVS4.
MySeVS4 apinsl-deerved structured and crystallizes the arthorhamble imma space group. Mg''] In bended to tuo equivalent 5)1) two equivalent 5/2) stoma to form MgSS tetrahedm share corners with ais equivalant Sc|1
angles range from 55-62", Mgrt)-5(1 bond lengths are 2.46 Both Hglh)- bond lengths are 2.42 Soft) banded to two equivalent 5)1) and four equivalintt 512) atoms to form Sc56 octahedra that share corners with aix equivalent Mgt 1/54 tetrahedra, n, edges with two equivalent 5c/1/56 octahedra, and edges with four equisalent V115S actahedra, Both Sc(t)-S(1) bond lengths are (c(1)) band langthe are 2.57 V/ bonded two equivelent 512) and four equiealant 5(1) stoms form VSI octahedra that share conmera with equisolant Myl1 tetrahedrs, edges with two equivalent VI1J58 acteherira, in and edges with four equivalent 5o/1/56 octahedra, Both VIT)-SP bond lengths are 2.46 All bood lengths are 2.48 There are two Inequivalent 5 sites. In the first alte, 5(1) is bonded in rectangular see like geometry to one Mgt1) one So(t), and two equivallnt VJ1) stoms. the second S site, 512) banded in rectangular swe-xow geometry to one Mirt), two equivatent Sc and one VV1] stom.
octebadra and car cormera with aix equivelent W1556 actahedra.
corner corner-aheinng octahedral
How chould I modity MgSoVS4 to develop a new material tol with the destred property?
(Stage 2: Transition) transit Mg to new material, transformation the crystal structure and composition be necessary. initiallly the arthorhomblo space group. This involves reorganizing arrangement bonding the atoms, partioularll cations coordination environment around
space group MgSeV84 would need be altored to monoclinic
O
coodinates differently sultur, forming would replaced by substituted
octahedra with adapted the new crystol structure, Additionally bonding lengtha and angles would
edge comer- sharing prometries.
bondling
adjusted new spotial contiguraaion reconfigurato otton require precise control over stotchlometry and synthesis conditions ensure stability and integrity new matertal's crystal structure.
coordination gooecceiee required ed the monoetinic symmetry. comptex
Based the information, the new material, could you generate description the lengthe and anglen the lattice vectors and the element type and coordinates each stom within luttion?
(Stuge 3: Generation) 7.0
5.7in85 30 0.8s/nTito.00 w.w.hn www.:.........100 w.w0000000.000000100 0.37
0.85hv5ite0.2o 1.12 0.00ho5tnn.. 54 w.w.2n55n0..... 8.65
7.07.06.7
generattes usin pymatann NaTiV54
ganerated pymaagen
66
symmetry space group
symmebry space group name,
7,40020080
length
length
0.890.25000
angla sipha 03.00000260
_cell ang.angeee. algha
0.160.61
angle 65.00000000
angln 5856665209
symmetyy angle gamma
angle gamma 18.41645300
0.01
symmehry
17600.12 0.08
_chemical forouuaa sum Noa V1
volume 57684733331
votume 178.60701266
0.990.90...0
formula loop
TIVSA
loop
0.290.650.46
symmetry equie
sammetry.cuuh pos, site_d
symmetty equie
symmetry.cc www.yylly.....o pew.t.oosseeeeeee 25, 'x.yyr
ALX representation
yxy.2 loop
toop
type symbol
type symboot
Converted by PyMatgen
.atom symmetry, mutippicity
Relaxed
.a0om_ label
.300m. mottiiylllyttiittyyyyyyy
.3tmm fract y
MIGNet
.30mm
occupancy
.a0om, pccupancy
0,86000000 8,00000000 0.65000000
0.64634006
0.00634087 0.78428800
0.50300300 25000000 99030000
0.86369659
22366663
100662582
0.10300000 6.50003009
0.32002030
463325543
2.29000247
0.01020000
6.00128275
0.39226276
0.75886350
0.79154123
0.11164118 0,06250051
540000000 0,80000000 22000000
6.6813899 0.91513980
0,26929000
0,29000050 0,66300000 0,45000000
8.8886605 0.67887963 0,37772363
CIF repr regresentation conwerted from repressstenntaitonn
CIF representation of atructureeelsceed by M3GNNt
Figure 7: A comprehensive example of MatExpert for conditional generation: Enter the desired properties, and MatExpert will generate the CIF representation of the material.
7 CONCLUSION
In this work, we introduced MatExpert, novel framework leveraging LLMs and contrastive learn- ing for the material discovery process. By emulating the workflow of human materials science expert, MatExpert integrates retrieval, transition, and generation stages to design new materials. Our experiments show that MatExpert significantly outperforms afate-of-the-an methods in material generation tasks, thereby exhibiting its potential to become scalable tool for accelerating material discovery with higher-quality crystal generation.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEGMENTS
This work is supported by the Mila internal funding Program P2-V1: Industry Sponsored Academic Labs (project number: 10379), the Canada CIFAR Al Chair Program, and the Canada NSERC Discovery Grant (RGPIN-2021-03115))
REFERENCES
Nawaf Alampara, Santiago Miret, and Kevin Maik Jablonka. Mattext: Do language models need more than text & scale for materials modeling? In Al for Accelerated Materials Design-Vienna Vienna 2024, 2024.
Luis M Antunes, Keith Butler, and Ricardo Grau-Crespo. Crystal structure generation with au- toregressive large language modeling. arXiv preprint (rXiv:2307.043400 2023.
Chi Chen and Shyue Ping Ong. A universal graph deep learning interatomic potential for the periodic table. Nature Computational Science, 2(1 (11):718-728, 2022.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey E. Hinton. A simple framework for contrastive learning of visual representations. In ICML, volume 119 of Proceedings of Machine Learning Research, pp. 1597-1607. PMLR, 2020.
Callum Court, Batuhan Yildirim, Apoorv Jain, and Jacqueline M Cole. 3-d inorganic crystal structure generation and property prediction via representation learning. Journal of Chemical Information and Modeling, 60(10):45 (1:00):5188444 884535, 2020.
Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory- efficient exact attention with io-awareness. Advances Neural Information Processing Systems, 35:16444-16359, 2022.
Grégoire Delétang, Anian Ruoss, Paul-Ambroise Duquenne, Elliot Catt, Tim Genewein, Christo- pher Mattern, Jordi Grau-Moya, Li Kevin Wenliang, Matthew Aitchison, Laurent Orseau, et al. Language modeling is compression. arXiv preprint (XXX::2339.10660 2023.
Jacob Devlin, Ming- Wei Chang. Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Pupers), pp. 4171-4186, 2019.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.2 1783, 2024.
Alexandre Duval, Simon Mathis, Chaitanya K Joshi, Victor Schmidt, Santiago Miret, Fragkiskos D Malliaros, Taco Cohen, Pietro Lio, Yoshua Bengio, and Michael Bronstein. A hitchhiker's guide to geometric gnns for 3d atomic systems. arXiv preprint a7X1v22312.05511 2023.
Daniel Flam- Sham-Shepherd and Alán Aspuru-Guzik. Language models generate molecules, materi- als, and protein binding sites directly in three dimensions as xyz, cif, and pdb files. arXiv preprint 2000002000 2023.
Alex M Ganose and Anubhav Jain. Robocrystallographerr automated crystal structure text descrip- tions and analysis. MRS Communications, 933::774-81,, 2019.
Prashant Govindarajan, Santiago Miret, Jarrid Rector- -rooos, Mariano Phielipp. Janarthanan Ra- jendran, and Sarath Chandar. Learning conditional policies for crystal design using offline rein- forcement learning. Digital Discovery, 3(4):769-775, 2024.
Nate Gruver, Anuroop Sriram, Andrea Madotto, Andrew Gordon Wilson, C. Lawrence Zitnick, and Zachary Ward Ulissi. Fine-tuner language models generate stable inorganic materials as text. In Al for Accelerated Materials Design NeurIPS 2023 Workshop, 2023. URL https:
//openrevien ew. net. 2ttp://oorrmm2d00000500522552

---

Published as a conference paper at ICLR 2025
Nate Gruver, Anuroop Sriram, Andrea Madotto, Andrew Gordon Wilson, C. Lawrence Zitnick, and Zachary W. Ulissi. Fine-tuned language models generate stable inorganic materials as text. In ICLR. OpenReview.net, 2024.
Kausik Hira, Mohd Zaki. Dhruvil Sheth, NM Anoop Krishnan, et al. Reconstructing the materials tetrahedron: challenges in materials information extraction. Digital Discovery, 36)):0221-1037, 2024.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation large language models. In International Con- ference Learning Representations, 2022. URL https::/opeereviiii net/ forum?
id=nZevKeeFYES
Kevin Maik Jablonka, Philippe Schwaller, Andres Ortega- and Berend Smit. Leverag- ing large language models for predictive chemistry. Nature Machine Intelligence, 6(2):161-169, 2024.
Anubhav Jain, Shyue Ping Ong, Geoffroy Hautier, Wei Chen, William Davidson Richards, Stephen Dacek, Shreyas Cholia, Dan Gunter, David Skinner, Gerbrand Ceder, et al. Commentary: The ma- terials project: A materials genome approach to accelerating materials innovation. APL materials, 1(1), 2013.
Rui Jiao, Wenbing Huang, Peijia Lin, Jiaqi Han, Pin Chen, Yutong Lu, and Yang Liu. Crystal structure prediction by joint equivariant diffusion. Advances Neural Information Processing Systems, 36, 2024.
Georg Kresse and Jürgen Hafner. Ab initio molecular dynamics for liquid metals. Physical review B, 47(1):558, 1993.
Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Advances Neural Information Processing Systems, 35: 15:1950-1966 1965, 2022.
Benjamin Kurt Miller, Ricky T. Q. Chen, Anuroop Sriram, and Brandon M Wood. FlowMM: Gen- erating materials with riemannian flow matching. In Forty-first International Conference on Ma- chine Learning, 2024. URL /ttLtttss//ppereeveee
net www.fformmmiieem Doooummiid-WWBB7Vb2222
Santiago Miret and NM Krishnan. Are 1lms ready for real-world materials discovery? arXiv preprint arXiv:2402.05220, 2024.
Santiago Miret, NM Anoop Krishnan, Benjamin Sanchez-Lengeling, Marta Skreta, Vineeth Venu- gopal, and Jennifer N Wei. Perspective on ai for accelerated materials design the ai4mat-2023 workshop at neurips 2023. Digital Discovery, 2024.
Adrian Mirza, Nawaf Alampara, Sreekanth Kunchapu, Benedict Emoekabu, Aswanth Krishnan, Mara Wilhelmi, Macjonathan Okereke, Juliane Eberhardt, Amir Mohammad Elahi, Maxim- ilian Greiner, et al. Are large language models superhuman chemists? arXiv preprint
2rXiv:2444.01475. 2024.
Vaibhav Mishra, Somaditya Singh, Mohd Zaki, Hargun Singh Grover, Santiago Miret, Mausam and Anoop Krishnan. LLamat: Large language models for materials science. In A/ for Accelerated Materials Design Vienna 2024, 2024. URL /
Shyue Ping Ong, William Davidson Richards, Anubhav Jain, Geoffroy Hautier, Michael Kocher, Shreyas Cholia, Dan Gunter, Vincent L Chevrier, Kristin A Persson, and Gerbrand Ceder. Python materials genomics (pymatgen): A robust, open-source python library for materials analysis. Computational Materials Science, 68:314-319, 2013.
OpenAl. GPT-4 technical report. CoRR, abs/2303.08774, 2023. 2023.
12

---

Published as a conference paper at ICLR 2025
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp- 8748-8763. PMLR, 2021.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1 1-677 2020.
Mayk Caldas Ramos, Christopher Collison, and Andrew White. A review of large language models and autonomous agents in chemistry- arXiv preprint arXiv:2407.06003 2024.
Janosh Riebesell, Rhys EA Goodall, Anubhav Jain, Philipp Benner, Kristin A Persson, and Al- pha A Lee. Matbench discovery-an evaluation framework for machine learning crystal stability prediction. arXiv preprint ar arXiv:2308. /4920, 2023.
Markus Scheidgen, Lauri Himanen, Alvin Noe Ladines, David Sikter, Mohammad Nakhaee, Ádám Fekete, Theodore Chang, Amir Golparvar, José A Márquez, Sandor Brockhauser, Se- bastian BrA'/ckner, Luca M. Ghiringhelli, Felix Dietrich, Daniel Lehmberg, Thea Denell, An- drea Albino, Hampus Naesstroem, Sherjeel Shabih, Florian Dobener, Markus Kátáhbach, Rubel Mozumder, Joseph F. Rudzinski, Nathan Daelman, José M. Pizarro, Martin Kuban, Cuauhtemoc Salazar, Pavel Ondracka, Hans-Joachim Bungartz, and Claudia Draxl. Nomad: A distributed web-based platform for managing materials science research data. Journal of Open Source Soft- ware, 8398):5388, 2023. doi: 10.21 105/88ss.05388. URL https: /doi. org wtt.////.//........ joss. 05388
Mara Schilling-iiiiii Withielii Martiño Ríos-García, Sherjeel Shabih, María Victoria Gil, Santiago Miret, Christoph Koch, José A Márquez, and Kevin Maik Jablonka. From text to insight: Large language models for materials science data extraction. arXiv preprint arXiv:2407.16877 2024.
Amanpreet Singh, Ronghang Hu, Vedanuj Goswami, Guillaume Couairon, Wojciech Galuba, Mar- cus Rohrbach, and Douwe Kiela. Flava: A foundational language and vision alignment model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 15638-15650, 2022.
Yu Song, Santiago Miret, and Bang Liu. Matsci-nlp: Evaluating scientific language models on ma- terials science language tasks using text-to-schema modeling. In Proceedings of the 61 Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3621- 3639, 2023a.
Yu Song, Santiago Miret, Huan Zhang, and Bang Liu. Honeybee: Progressive instruction finetuning of large language models for materials science. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 5724-5739, 2023b.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models. CoRR, abs/2307. .09288. 2023.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi. Quoe V Le, Denny Zhou, et al. Chain-of th-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-2444 -24337, 2022.
13

---

Published as a conference paper at ICLR 2025
T Wolf. Huggingface's transformers: ofate-of-the-arr natural language processing. arXiv preprint arXiv:1910.03771, 2019.
Tian Xie, Xiang Fu, Octavian Ganea, Regina Barzilay, and Tommi Jaakkola. Crystal diffusion vari- ational autoencoder for periodic material generation. Bulletin of the American Physical Society, 67, 2022a.
Tian Xie, Xiang Fu, Octavian-Eugen Ganea, Regina Barzilay, and Tommi S. Jaakkola. Crystal diffusion variational autoencoder for periodic material generation. In ICLR. OpenReview.net, 2022b.
Tong Xie, Yuwei Wan, Wei Huang, Zhenyu Yin, Yixuan Liu, Shaozhou Wang, Qingyuan Linghu, Chunyu Kit, Clara Grazian, Wenjie Zhang, et al. Darwin series: Domain specific large language models for natural science. arXiv preprint arXiv:2308. 13565, 2023.
Minghao Xu, Xinyu Yuan, Santiago Miret, and Jian Tang. Protst: Multi-modality learning of protein sequences and biomedical texts. In International Conference on Machine Learning, pp. 38749- 38767. PMLR, 2023.
Sherry Yang, KwangHwan Cho, Amil Merchant, Pieter Abbeel, Dale Schuurmans, Igor Mordatch, and Ekin Dogus Cubuk. Scalable diffusion for materials generation. In The Twelfth International Conference on Learning Representations, 2024. RRRLtttps://ppereviiee net forum?
@d=wm4W1Hoxpc)
Claudio Zeni, Robert Pinsler, Daniel Zügner, Andrew Fowler, Matthew Horton, Xiang Fu, Sasha Shysheya, Jonathan Crabbé, Lixin Sun, Jake Smith, et al. Mattergen: a generative model for inorganic materials design. arXiv preprint (xXiv:2312.03681 2023.
Huan Zhang, Yu Song, Ziyu Hou, Santiago Miret, and Bang Liu. Honeycomb: flexible Ilm-based agent system for materials science. arXiv preprint arXiv:2409.00133, 2024.
Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, and Zheyan Luo. Llamafactory: Unified efficient fine-tuning of 100+ language models. arXiv preprint arXiv:2403.13372 2024.
14

---

Published as a conference paper at ICLR 2025
A REPRODUCIBLITY
A.1 ENVIRONMENT
The experiments were conducted machines with the following specifications.
Machine for MatExpert with Llama-2 8B and Llama-3 8B:
Hardware: NVIDIA A5000 GPU (24 GB on-board memory) GPU 4, AMD Ryzen
Threadripper PRO 3975WX CPU, 256 GB RAM
Software: Ubuntu 22.04, Python 3.11, PyTorch +cu.1, CUDA 12.4
Machine for MatExpert with Llama-2 70B and Llama-3 70B:
Hardware: NVIDIA L40S GPU (48 GB on-board memory) 4, INTEL(R) XEON(R) GOLD 6526Y x 2, 512 GB RAM
Software: Ubuntu 22.04, Python 3.11, PyTorch 2cu.1+ CUDA 12.4
The list of environment requirements for the Python library will be made publicly available alongside the release of the open-source code of MatExpert.
A.2 IMPLEMENTATION DETAILS
Table 4: Hyperparameters for training T5-based encoders the retrieval stage.
Hyperparameter
Value
Description
Model Architecture
t5-base
Pretrained model to use as the base encoder.
Learning Rate
le-4
Learning rate for the optimizer.
Number of epochs
100
Number of epochs to train for.
Temperature
0,1
Temperature parameter for the NT-Xent loss.
Batch Size
32
Batch size for training.
Gradient Accumulation Steps
8
Number of batches for gradient accumulation.
Table 5: Hyperparameters for finetuning Llama family in the transition and generation stage.
Hyperparameter
Llama-3 8B
Llama-3 70B
Model Architecture
Mcta-L Instruct
Mcta- Meta-Llama-33 3-mm--3770B- Instruct
Learning Rate
le
Batch Size
4
Batch Size per Device
Number of Epochs
Optimizer
AdamW W
Adam W
Sequence Length
2048
1024
LoRA Rank
8
LoRA Alpha
16
16
LoRA Dropout
0.1
0.1
Warmup Ratio
0.1
Gradient Accumulation Steps
8
The proposed MatExprt framework consists of two training stages: 1) Training T5- ba-ba encoders
of the contrastive learning framework the stage Retrieval. 2) Finetuning the Llama-2/Llama- based LLMs in the stage Transition and stage Generation.
Training b5-based encoders In the retrieval stage, two T5-based encoders are employed. The model uses the "t5-base" architecture in Huggingface Wolf (2019) its foundational structure. The training process is governed by a learning rate of 0.0001. It undergoes training for 100 epochs with early stopping. A temperature parameter of 0.1 is utilized, which affects the NT-Xeet loss function, a common choice for contrastive learning. The batch size is set to 32 and gradient accumulation occurs over 8 steps, effectively simulating a larger batch size and allowing for more stable updates.
15

---

Published as a conference paper at ICLR 2025
Finetuning Llama family In the transition and generation stages, the Llama family of models is finetuned. Two variants are used: Llama-3 8B and Llama-3 70B, both with specific architectures designed for instruction-baseed tasks named Meeta-Llamaa--88-Inntttee Instruct" in Huggingface Wolff (019). The learning rate is set to 0.0001. Each device processes batch size of 1, with an overall batch size of The 8B model is trained for 70 epochs, while the 70B model undergoes 10 epochs, reflecting differences in their complexity and training needs. The AdamW optimizer is employed for parameter updates. Sequence lengths differ, with the 8B model handling sequences of 2048 tokens, and the 70B model processing 1024 tokens limited by the on- on-board memory of GPUs. LoRA settings include rank of an alpha of 16, and a dropout rate of 0.1, facilitating efficient adaptation. A warmup ratio of 0.1 helps stabilize initial training and gradient accumulation is set steps for both models, ensuring robust learning.
and "Meta-Llama-33700"
A.3 EFFICIENCY
We utilize the open-source LLM training framework named LLaMA-Factoon Zheng al. (2024) to finetune the LLMs. With the advantages of LoRA Hu et al. (2022) and FlashAttention Dao et al. (2022) mechanism, takes around 30 and 120 hours to fine-tune Llama-3 8B and Llama-3 70B in MatExpert, respectively.
The generation speed of MatExpert on our machines is as follows:
MatExpert with Llama-3 8B: 400 minutes per 10,000 samples.
MatExpert with Llama-3 70B: ~ hours per 10,000 samples.
B DATASETS DETAILS
Two datasets are included in our experiments: Material Project and NOMAD for unconditional generation and conditional generation, respectively.
Material Project This dataset also named as MP-20 consisting of 45,231 materials publicly
available and first proposed in Ong al. (2013). We follow the data-split setting proposed in Gruver et al. (2023) and Xie et al. (2022a).
NOMAD
This large-scale dataset consisting of 2,886, 120 materials is collected by ourselves and will be publicly available. We collect the datasets via NOMAD API as the following script:
url = http:///w....teeeeee.../t.//tttt
excluded #xcluded_elements.
"He", "Ne", "Ar", "Kr", "Xe", "Rn", "0", "Th", "Rn", "Tc", "Po", "Pu", "Pa",
query {
"VASP"
}, "quantities:ail":
"reaales.progrrt.essstttuu
required
"results";
"material"; {
Schemicalfformite fariiaaa__ommmlar_eddddddd "*"
b
"properties": (
16

---

Published as a conference paper at ICLR 2025
"structures": "*"
We filter out the materials with elements in excluded e elements s and only include the materials with structures computing by VASP Kresse Hafner (1993).
Then we further convert the structure in JSON to CIF files via the following script:
def convert_to_ci istr, lattice=' lattice==vectors')
lattice_vectors structure_jjaon (wwattinnaaoioocc cellaattce_eeeeeee cevettir_eecttrs" lattice_parametee - structure_jjon
atrusture__jsonnlatiii
a Lattice_parraeterssll
b lattice_parrmettess ["b]]
c = lattice_pprameterr3|
alpha l ("alphaa) 180 np.pi
beta = lattice_parrameterss
* 180 / np.pi
ganna
("gamaaa)
180 np.pi
if lattice vectors': lattice Lattice (lattice, [lattice_vectorn)
elif lattice 'parameters':
lattice Lattice www.eearttraatteereeeeeeeaeaeeee (a, b, alpha, beta, gamma)
positions /ttp://eeeaoaaaaaaraiaaaaaaaaaee
species structure_json| (ttpeasooosssiciesseseeee st_t__sss_t__
stracture_jssnn(ppeeee....
structure Structure (lattice, species,
positions, coodd__are_carttsiinn-T
structure.scalee_lattiieee
return structure. to (fructurr.tofffttciiii..
(offt="cif")
We randomly select 10,000 samples and another 10,000 samples as the validation and testing set, respectively. The remaining samples are all included in the training set.
C EVALUATION METRICS
C.I VALIDITY
Validity evaluated based on the structural and compositional integrity the generated materials. Following Xie et al. (2022a), a structure is considered valid if the shortest distance between any pair of atoms is greater than 0.5 Å, ensuring atomic stability. Furthermore, the overall charge of the material must be neutral to satisfy chemical feasibility. These criteria ensure that the generated materials conform to realistic physical and chemical constraints.
C.2 COVERAGE
Following the methods outlined in Xie et al. (022a) and Gruver et al. (2033) we measure coverage using two metrics: COV-R (Recall) and COV-P (Precision). These metrics quantify how well the generated materials resemble the ground truth materials the test set. COV-R (Recall) measures the percentage of ground truth materials accurately represented by the generated set, while COV-P (Precision) evaluates the quality of the generated materials in replicating the true material diversity. These metrics ensure that the generated materials not only capture the diversity of the test set but also maintain high-quality representations. Both validity and coverage are computed over 10,000 materials randomly sampled from the generated set.
17

---

Published as a conference paper at ICLR 2025
C.3 DISTRIBUTION
The distribution of properties is compared between the generated materials and the test set using the Wasserstein distance. The key properties evaluated density (p, measured in g/em") and the number of unique elements (Net). The property distribution is computed over 1,000 valid materials, randomly sampled from those that pass the validity test. This method ensures that the generated materials exhibit realistic physical properties comparable to those in the test set.
C.4 DIVERSITY
Diversity is assessed by calculating the pairwise distances between samples, based on structural and compositional features as described by Court al. (2020). This metric quantifies how distinct each generated sample is relative to others in the dataset, offering insights into the variety structures and compositions. Diversity is specifically measured samples deemed metastable by M3GNet, as these are more likely to contribute meaningful variation. All diversity values are normalized against corresponding metrics from the test set, providing clear comparison of the underlying data distributions.
C.5 NOVELTY
Novelty calculated by comparing each generated sample its nearest neighbor in the training set. A sample is considered novel if its nearest neighbor exceeds predefined threshold distance. Specif- ically, we use structural distance cutoff of 0.1 and a compositional distance cutoff 2. Novelty is assessed both in terms of structure and composition, with overall crystal novelty determined by the presence of a new structure or composition. These metries provide insights into the uniqueness of the generated samples, particularly those identified as metastable. Novelty values are normalized by corresponding values for the test set effectively convey the characteristics of the data distribution.
C.6 STABILITY
We assess the stability of materials using combination of machine learning potentials and density functional theory (DFT) to ensure a consistent evaluation framework. Specifically, we employ the M3GNet model Chen & Ong (2022), trained on total energy data from VASP calculations within the Materials Project dataset. This aligns the results with established correction schemes and absolute energy values. For consistency with Materials Project settings and prior works Xie et al. (2022a); Gruver et al. (2023), including the PBE functional and DFT/DFT+U, we perform single relaxation for each candidate structure using the default MPRelaSet parameters.
To determine the percentage of metastable compounds, we first filter out samples that fail basic struc- tural and compositional validity checks. The remaining samples are then relaxed using M3GNet to obtain final relaxation energies. The stability calculation includes the validity rate from initial filtering and the rate of compounds with relaxed hull energy Enail 0.1. For stable materials, those identified as metastable by M3GNet undergo further DFF relaxation, and the percentage with Enoll < 0.0 reported. This comprehensive approach integrates both machine learning and tradi- tional DFT methods for robust stability evaluation.
D CASES OF UNCONDITIONAL GENERATION
Please see Figure
g
E CASES OF CONDITIONAL GENERATION
Please see Figure
F NUMERICAL RESULTS OF ABLATION STUDY (FIGURE 6)
Please Table
18

---

Published as a conference paper at ICLR 2025
generated
U2PN2
SKCN20
HM PP
H-M
cel 3.70000000
csll 4.80000009
longth_b 3.70020200
4.80050200
0.44 0.68
longth. 5,60000000
0.260.120.90
4.80000000
58.00000000
90.00000000
angle t0.00000000
00000000000
08.000000000
cnll int Tables number
00.00000000
0.780.280.74
int Tablen number
0.280.120.42
chamical formula structural UZPN2
chemical formula structural SiCNzja
0.440.61 0.88
chamical formala cell volums 07.05154525
0.560.62..88 0.99
chemicaa formula sum Sin N4
-call volume 110.50200000 call formula units
cell formula units
0.760.120.42
0.780.82..40
symmetry
atte
equiv
.aymmttry equiv xxy.""
'xy.x
_atom sioo. symbol
_atom site. bitt__tpe__t.. type_symbol
site lobel
_atom_sh labol
multiplicity
wite.
multiplicity
fract.t
aits. fruct_a
site, Jrect_y
_atom site. fruct_y
sito, froct_t
_atom _atom_site, site, fract_t
Sito www.ppouenyyyyy
.aomm site, coo.uucuugyyy
U UO 0.76000000 o 0.30000000
0.10000000
SI SID 0.76000020 8 620000000 o 0,50000000
U U1 0.46000000 0,61000000
0.63000000
0.12000000 0.26005000
0.52000000
P P2 1000000
0,80000000 0.35000000
C2 0.26005050
0.62000500 0.42000000
N N3 0.78000000 6 33000000 0.74000000
NS 0.26305000
0.13000500 0.42000000
N N4 0.44000000
0.81000000
0.50000000
0,26005000
0.62000500 0.52000000
0,78005000
8:12000500 0.42000060
0,76005000
0.62000500 0.42000000
generated using pymatgen
90 90
data dita_L
Li
symmetry
name. H-M 'P-'
.coll langth. 4.9000000
cell length 4,90000000
0.39 0.07 0.20
cell tangth. 3.3000000
Li
cell angle alpha 00.00000000
angle
Li
Lngle gamma 90. 00.0000000
0.39 0.70
symmetry Int Tables number 1
.chemicalformmnn cacicca/ooramaasatai sstructurral Li2GeF6
chemical formula sum
Ge
voluue 223.29300000 call formula unita Z
0.89..5570.0 0.45
0.09 0.95
symmetry equiv pos site symmetry equiv pos as xyz 'x,,,''
0.09:0088
0.180.380.86
_stom_si wit. type symbol
.atem site label
0.180.38
wwwmmtry_mutiplleee
0.59 0.62
_atom_stt _ittt_ttacttty
atem site fract
0.80000000
0.57000000
0.7000000
0.680.77..62 0.62
LI LIT
0.39000000 0,07000000 0,20000000
Li 0.8900000
0,57000000 0.20000000
0.880.270.12
0.39000000
0.07000000 0,7000000
Ge4
0.36000000 0.07000000
0,85000000
1.69.0.770.88
Ge DeS 0.89000000 0.57000000
0.45000000
0.09000000
0.88000000 0.95000000
0.680.270.45
0.0900000
0,45000000
F F9 0.18000000 0,38000000 F F8
0.13000000 0.38000000 0.95000000
0.090.277..8
0.45000000
0.180.770.45
0.68000000
0.77000000
0.12000000
F F12 0.6800000
0.77000000
0.62000000
0.5900000
0.27000000
0.1200000
0.95000000
F F15 0.6800000
0.27000000
0,45000000
F F16
0.0900000
0,27000000
0.8500000
0.1500000
0.77000000
0,4500000
Figure 8: Cases of unconditional generation.
19

---

Published as a conference paper at ICLR 2025
am looking to design new material with the following property: The formation energy atom is -0.168. The band gap D.O. energy
Hg. spacegroup number
4.34.34..
generated using pymatgen
LiHg3
0.70 0.800.86...2 0.26
cell length. a 4.3000000
Hg
-cll length_b 4.30000000
Hg
anglo alpha 30.00000000
0.20 0.86 0.71
cell angle beta 30.00000000
10.0000000
eyymetry structural LiHg3
foemical[formuia..o
chemiccl sum "Li1 Hg3
79.50700000 _ell_tormuta_ume loop.
symmetry equivy _os_sto__ee
symmetry equiv pos xya 'x
loop
_atom_aite wyppe_yyyooollllooooooooo
_atom__ite, www.s/trr.mmtttipiioooooo
_atom_site x
atom fract
.atom site occupancy
Li0 0.70000000 0.86000000 0,21000000
Hg 0.20000000 0.38000000
Hg Hg2 0.20000000 066000000 0.71000000
Hg Hg3 0.70000000 0.36000000 0.71000000
am looking to design new material with the following property: The formation energy per atom Is -0 0000331, The band gap is 0.0. The energy above the convex huil is 0.0531, The elements are Min Ge. The spacegroup number is 194.
generated using pymatgen
Mn
symmetry spwce group name, H-M 'P''
0.120.66
_coll_ength_a 4.00000000
length
0.67 0.12
.cel. longth e 5.10000000
Mn
_cell pngte_ 00.0000000
0.70..58
00.0000000
Mn
garita 120.0000000
0.33 0..00.5...00 0.88
symmetry Int Tables number
De
chemical formula structural Mn20e
0.330.450.31
chemical
Ge
.cell volume 70.86767285 ccel formula, units formula_units_2
0.00 0.700.88
symmetryyeequ wwwppeppuopppooop houuv_pos_sit_eee
symmetty wewle__uu_____ pos_possspsppooooo xy_ y, loop
stom site type symbol
atom site label
_stom_site commeetrymmutiiliihn
_stom_site, fract_y
.atom site. fiat_fract_2
0.67000000 0.12000000
Mn Mn1
0.67000000 0.12000000
0.06000000
Mn Mn2
0.00000000
0.78006000 0.31000000
0.3300000
0.81000000
Ge Ge4 0.3300000
0.45000000
0.31000000
Ge Ge5 0.0000000 0.7800000 0.81000000
Figure 9; Cases of conditional generation.
Table Numerical results of Ablation Study (Figure 6)
MatExpert
-CoT Stage
-ooT Stage
Reetrval Stage
< 5 atoms
76%
72%
62%
< 10 atoms
69%
62%
59%
< 25 atoms
66%
58%
51%
Space group
31%
30%
24%
Energy above hull
60%
56%
53%
20


Reviews:
[{"review_id": "qMBv14xICE", "reviewer": "Reviewer_Krbm", "summary": "MatExpert is designed to streamline the discovery of new materials using LLMs and contrastive learning. Inspired by the traditional workflow of human experts, MatExpert operates in three stages: retrieval, transition, and generation. Experimental results demonstrate that MatExpert outperforms current sota models in material generation tasks.", "questions": "The heavy reliance on specific material databases, such as NOMAD, might lead to overfitting or model bias toward these datasets. Testing MatExpert on unseen data sources or a wider array of material properties could offer better insights into its generalization capabilities.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The design of MatExpert mirrors the expert-driven process in material science, breaking down material generation into retrieval, transition, and generation stages. This structured approach allows for iterative refinement.\n\n2. The transition stage uses a CoT reasoning process, enabling the model to outline logical, step-by-step modifications to meet target properties. This sequential reasoning contributes to the model's ability to achieve high accuracy in conditional generation tasks.\n\n3. By compiling a dataset of over 2 million materials from NOMAD, MatExpert provides a large-scale testbed to assess its performance.", "weaknesses": "1. While the multi-stage design of MatExpert improves accuracy, it adds computational complexity and potentially increases training time compared to single-step models. \n\n2. The proposed framework will have cumulative errors. If the result retrieved in the first step is far away from the target, it will be difficult to correct it later, thus affecting the results of subsequent steps.\n\n3. This paper focuses on innovation in application scenarios, and the technological innovation is relatively limited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vKV4OOCsj5", "reviewer": "Reviewer_vZKP", "summary": "The paper describes a novel LLM based framework for materials discovery (MatExpert). There are three stages to the MatExpert methodology: retrieval, transition, and generation. First, a material is retrieved from the database that most closely matches the description given (similarly was trained with contrastive learning). Next during transition, the model determines how to alter the retrieved material to match the desired properties. Lastly, in the generation phase the model produces a ALX representation that is converted into a CIF representation. The main contribution of the paper is the MatExpert framework and the accompanying benchmarking and ablation study.", "questions": "- The paper mentions that the wdist density and wdist number of elements metrics are greatly improved compared to CrystalLLM but if the model is given a template from the database does that undermine these metrics? Is there a way to test this? For example, how often does the generated structure change the number of elements compared to the retrieved/given material? \n- Can you compute the S.U.N metric from Zeni et al. (https://arxiv.org/abs/2312.03687)?\n- How does the inference speed of MatExpert compare to CrystalLLM?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The application of LLM to materials is interesting and materials discovery is important\n- The evaluation metrics includes stability computed with DFT not just proxy metrics \n- Writing style and related work are good", "weaknesses": "- Lacking details on what data was used for which tasks? For the unconditional results on MP-20, it is unclear if the NOMAD data was also used for training MatExpert. For the conditional results, were CrystalLLM and MatExpert trained on the same data?\n- The results in Figure 5 are not well quantified i.e. it is not clear MatExpert is better. Also, there are 11 bars but only 9 labels, not sure if I missed something? The colors are very similar in some cases, hard to parse quickly.   \n- There is no discussion on the limitations of using the retrieval stage. My interpretation is that the retrieved material is like a template. Before generative models, new materials were searched for using templating/substitution methods. One of the critiques of those methods is that materials generated are still quite similar to the template, would that also be a limitation here?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GnsvEsM7pG", "reviewer": "Reviewer_SCz7", "summary": "The paper presents MatExpert, a framework that leverages large language models (LLMs) and contrastive learning to automate materials discovery. The proposed approach mimics human experts by breaking the process into three stages: retrieval, transition, and generation. MatExpert first identifies a closely related material, then suggests modifications to meet target specifications, and finally generates new material structures. The authors test MatExpert with large datasets from NOMAD and the Materials Project, showing that it performs better than current methods on key measures like stability, validity, and how well it meets diversity and novelty.", "questions": "* Follow up on W1, given the reproducibility challenges in LLM-generated content, how does the framework handle multiple potential pathways for synthesizing a target material?\n* Follow up on W3, I wonder what’s MatExpert’s the success rate on each step? Specifically, the success rate for generating accurate ALX representation.\n* Can this method be useful for user query without specifying formation energy and band gap for the target material? For instance, can the user prompt the model like, I want to material composed of Mn, Ge and with high electrical conductivity.\n* What will the model response if the target material doesn’t exist?\n\n[1] Miret, Santiago, et al. “Are LLMs Ready for Real-World Materials Discovery?”\n\n[2] Zhang, Huan, et al. “HoneyComb: A Flexible LLM-Based Agent System for Materials Science”\n\n[3] Chiang, Yuan, et al. “LLaMP: Large Language Model Made Powerful for High-fidelity Materials Knowledge Retrieval and Distillation”\n\n[4] Microsoft Research AI4Science. “The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4”", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "* The integration of Robocrystallographer enriches crystal data with textual descriptions, enhancing the retrieval process and interpretability.\n* MatExpert achieves impressive performance on benchmarks, demonstrating its reliability in generating valid and diverse material structures.\n* Contrastive learning effectively maps structure and property embeddings, which is a novel approach for aligning multimodal material data.", "weaknesses": "* The novelty of multi-stages material generation is a bit limited as it’s being studied in other works [1, 2, 3].  In the introduction, the author mention the drawback of the current method is the single step material structure generation. However, some cited paper include multi-steps material generation and property query already [1, 2, 3]. It will be helpful to have more discussion on those methods.\n* The paper could benefit from more clarity on the pathway generation process. Specifically, it is unclear how the pathways generated by GPT-4 can be reliably reproduced in real-world lab settings. The authors might find [4] useful as a reference for evaluating the quality and safety of generated pathways.\n* Due to the complexity of multi-steps framework, the paper could discuss more on how the authors prevent the error propagation.\n* The visualization for Figure 6 is not clear enough. If there’s a table include the numerical value of the ablation study, it can better show the improvement of each component.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Bq3bkFMvyL", "reviewer": "Reviewer_2LRH", "summary": "The paper proposes MatExpert framework, designed to enhance LLM-based crystal structure generation through a 3-stage inference. The first stage searches an existing structure that closely resembles the user's input via T5 model trained with a contrastive learning objective on property and structure descriptions. Then, the fine-tuned LLM suggests modifications to the physical properties and structural attributes to achieve the target composition and characteristics. Finally, the LLM generates an ALX representation, which is converted to the final crystal structure. The authors also introduce a curated, large-scale dataset for training and demonstrate that this iterative, feedback-driven approach improves all evaluation metrics.", "questions": "1. In Figure 5, are all the structures relaxed or not? Does it contain all generated samples, or samples that passed the validity test? Please provide the details.\n\n2. Have you checked the energy value difference between the model inputs and generated samples? If there are OOD samples, does it pass the validity test or relaxation steps?\n\n3. Regarding Equation (1), what do you use as the similarity function?\n\n4. What prompts have you given to the model? Concretely, how much overlap is there between the conditioned properties and the NOMAD data?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. This paper presents a novel approach by modeling crystal structures with a chain-of-experts framework that utilizes multiple LLMs. As far as I know, this approach is novel, and appears effective in generating stable materials.\n2. Additionally, the use of fine-tuning methods, such as LoRA and distillation, enhances the framework’s efficiency and scalability, making it practical for real-world adaptation.", "weaknesses": "1. The authors mentioned in the paper that “In the unconditional generation task, we aim to assess the ability of MatExpert to produce novel and stable material structures without any specific property constraints”, while at the same time “... For unconditional generation, we randomly select a material from the database during the first stage of MatExpert.” According to the second referred sentence, the second, transition stage would take samples of training set embeddings or raw data would be given as input. This is contradictory to the claim that no structure or property is given to generate. For this reason, I believe the evaluation scheme is unfair. Please fix me in the author response if I understood incorrectly.\n\n2.  Following this concern, the performance improvement on generation is also questionable. The authors are encouraged to provide stabilities (Predicted energy over convex hull values, or DFT relaxation success rate) of the generated samples that are out-of-distribution. Also, stability measurements on Table 2 are also needed.\n\n3. There are stronger baselines following the CDVAE research, as the authors stated. However, comparisons between them, which is necessary, are missing in Table 1.  Furthermore, the authors also need to specify which dataset was used, or, the source of the CDVAE model. If the CDVAE model is trained only on MP-20, that would not be a fair comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)