Code-Generated Graph Representations Using Multiple LLM Agents for Material Properties Prediction
Authors: Jiao Huang
Qianli Xing
Jinglong Ji
Bo Yang
Venue: ICML 2025 poster (2025)
Keywords: Material structure representation
Material property prediction
Multi-agent systems
Large language models
Primary area: applications->chemistry_physics_and_earth_sciences

Abstract: Graph neural networks have recently demonstrated remarkable performance in predicting material properties. 
Crystalline material data is manually encoded into graph representations.
Existing methods incorporate different attributes into constructing representations to satisfy the constraints arising from symmetries of material structure.
However, existing methods for obtaining graph representations are specific to certain constraints, which are ineffective when facing new constraints.
In this work, we propose a code generation framework with multiple large language model agents to obtain representations named Rep-CodeGen with three iterative stages simulating an evolutionary algorithm. 
To the best of our knowledge, Rep-CodeGen is the first framework for automatically generating code to obtain representations that can be used when facing new constraints. 
Furthermore, a type of representation from generated codes by our framework satisfies six constraints, with codes satisfying three constraints as bases. 
Extensive experiments on two real-world material datasets show that a property prediction method based on such a graph representation achieves state-of-the-art performance in material property prediction tasks.

Code-Generated Graph Representations Using Multiple LLM Agents for Material Properties Prediction
Jiao Huang Qianli Xing 1 Jinglong Ji
Bo Yang 13
Abstract
material data (Damewood et al., 2023). Recently, graph neu- ral networks (GNN) have shown remarkable performance in predicting material properties and become the dominant technology (Chen et al., 2024b; Fang & Yan, 2024).
Graph neural networks have recently demon- strated remarkable performance in predicting ma- terial properties. Crystalline material data is man- ually encoded into graph representations. Exist- ing methods incorporate different attributes into constructing representations to satisfy the con- straints arising from symmetries of material struc- ture. However, existing methods for obtaining graph representations are specific certain con- straints, which are ineffective when facing new constraints. In this work, we propose a code gen- eration framework with multiple large language model agents to obtain representations named Rep-CodeGen with three iterative stages simu- lating an evolutionary algorithm. To the best of our knowledge, Rep-CodeGen is the first frame- work for automatically generating code to obtain representations that can be used when facing new constraints. Furthermore, type of representation from generated codes by our framework satisfics six constraints, with codes satisfying three con- straints bases. Extensive experiments on two real-world material datasets show that property prediction method based on such graph repre- sentation achieves state-of-the art performance in material property prediction tasks.
For most GNN- material properties predicting meth- ods, a crystalline material data instance is represented by attributed graph, where nodes represent the atoms and the edges define the interatomic reactions of neighboring atoms (Xie & Grossman, 2018). The attributes of the graph commonly contain elemental properties as node features and interatomic distances as edge features. With the manually designed codes, the attributed graph can translated into numerical form, called a graph representation, which can be processed by GNN. As many important material prop- erties are highly sensitive to the structure of materials, fundamental challenge for these methods is to obtain the representation that can capture the structure of crystalline material(Choudhaa & DeCost, 2021).
A widely adopted solution by researchers is to incorpo- rate different attributes into the process of constructing the graph and the representation. CGCNN (Xie & Gross- man, 2018), Matformer (Yan et al., 2022), and PotNet(LLin et al., 2023) mainly focus on the distance of atoms, while ALIGNN(Choudhary & DeCost, 2021), PerCNet(Huang et al., 2025), and ComFormer( Yan et al., 2024) further in- corperates angle attributes. With the different attributes, representations satisfy different constraints arising from symmetries of material structure. Ideally, representations capture structures better when satisfying more constraints. However, existing methods for obtaining graph representa- tions are specific to certain constraints. which is ineffective when facing new constraints. Moreover, the above research strategy requires expert knowledge in both material and computer science, which costs researchers years to learn.
1. Introduction
Over the past decades, machine learning methods have been widely used in material science. These methods normally employ hand-crafted descriptors as the representation of
Key Laboratory of Symbolic Computation and Knowledge Engincering of Ministry Education, Jilin University, Changchum, Jilin, 130012, China CCollege of Artificial Intelligence, Jilin University, Changchun, Jilin, 130012, China 'College of Com- puter Science and Technology, Jilin University, Changchun, Jilin, 130012, China. Correspondence to: Qianli Xing <qianlix- ing @ @ww..cdu.un........ Bo Yang <ybo@jlun bybo@@lu.edu..n>..
Inspired by the achievements of large language models (LLM) in code generation and material science, we propose code generation framework with multiple LLM agents to obtain representations named Rep-CodeGenn The codes generated by large language models are always of low qual- ity when the task is difficult. Specificly, the main difficulty is to guide LLM agents to generate codes to obtain repre- sentation satisfying constraints, which requires LLM agents
Proceedings of the 4rnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 20255 by the author(s).

---

Code-Generated Graph Representations Using Multiple LLM Agents for Material Properties Prediction
to accomplish the complex design of the material graph con- struction process with material knowledge and the coding process with computer science knowledge. Thus, we sim- ulate the process of an evolutionary algorithm by dividing the overall framework into three stages: crossover genera- tion, evaluation summary, and parent selection. In crossover generation stages, we incorporate two agents: the first agent used to take the prompts and generate plans on how to write the codes, and the second agent is used to generate codes according to plans. Then, in the evaluation summary stage, we score the generated codes with test dataset and use the third agent to provide a detailed analysis. Finally, in the parent selection stage, we set a metric to select the codes with high scores as parent codes for the next loop. The main contributions of this paper are as follows:
potentials through infinite distance summations.
Subsequently, to ensure that the graph representations of material structures satisfy reflectional symmetry, methods such as ComFormer( Yan et al., 2024) and PercNet(Huung et al., 2025) extended the multigraph representation by intro- ducing higher-dimensional features, including bond angles and dihedral angles. These enhancements further improved the prediction accuracy of machine learning algorithms in material property prediction tasks.
To satisfy Lipschitz continuity, AMI((Wdddowsn et al., 2022) and PDD(Widdowson & Kurlin, 2022) proposed mod- eling materials as periodic point clouds. Unlike CGCNN (Xie Grossman, 2018), which represents bond lengths as fixed scalars, these methods suggest using distance distribu- tions to model bond lengths.
We propose an interpretable framework for automati- cally generating codes to obtain graph representations that can be used when facing new constraints.
With the advancement of large language models (LLMs), leveraging their generative capabilities across various do- mains has emerged as a new research direction. Works such as material LLM (Tao et al., 2024) and MatExpert (Ding et al., 2024) propose modeling material generation tasks directly as string generation tasks, producing materials represented in CIF format. However, these works cannot theoretically guarantee that the generated material structures satisfy physical symmetry constraints, and the results lack interpretability. This significantly undermines the trust of domain experts in LLMs and even neural network outputs, limiting the practical application of LLMs in the field of materials. In contrast, we propose a novel approach to ap- plying LLMs in the material domain. Specifically, instead of directly generating representations, our method generates interpretable code. Human experts can understand the un- derlying principles of the results through the code, thereby enhancing trust in the outputs of LLMs.
We obtain a type of graph representation from gener- ated codes by our framework satisfying six constraints with codes satisfying three constraints as bases.
Extensive experiments in two real-world material datasets show that property prediction method based on such graph representation achieves srate-of-the-arr performance in material property prediction tasks.
2. Related Works
2.1. Material Representation with Deep Learning
In recent decades, significant advancements in machine learning have been achieved in the study of crystalline mate- rials (Meredig et al., 2014; Oliynyk et al., 2016; Ward et al., 2016; Ramprasad et al., 2017). For material property pre- diction task, the development of representations that satisfy constraints of crystalline material structures is crucial for success. CGCNN (Xie & Grossman, 2018) introduced a multigraph- based approach, leveraging periodicity and in- variance under rotation and translation by encoding atomic interactions through multiple edges between nodes. Subse- quent studies have expanded on this foundation by incor- porating additional physical insights. For example. SchNet (Schütt et al., 2017) integrates force data into training, Meg- Net (Chen et al., 2019) includes thermodynamic variables like temperature and pressure, and ALIGNN (Choudhary & DeCost, 2021) utilizes dual graph structures to represent bond lengths and angles. To better capture the contributions of individual atoms within a unit cell, GATGNN (Louis et al., 2020) employs attention mechanisms to model local atomic interactions. MatFormer (Yan et al., 2022) advances this by introducing a framework that ensures periodic in- variance, while PotNet (Lin et al., 2023) models interatomic
The statistics of constraint fulfillment for the aforemen- tioned algorithms are provided in Table 3 in the Appendix.
2.2. Multi-agent System
To better leverage the potential of large language models (LLMs), multi-agent systems often use specific frameworks to connect and invoke multiple L.MMs, enabling the comple- tion of tasks that are more complex specialized than what single LLM can achieve.
Due to the confabulations (or hallucinations) (Romera- Paredes et al., 2024) inherent in LLM outputs, incorporating evaluators to verify LLM--gnnrrted results is a common approach in agent systems. For instance, Haluptzok et al. (Haluptzok et al., 2022) employs Python interpreter to fil- ter the correctness of data generated by the language model, thereby enhancing its performance. Similarly, Zelikman et al. (Zelikman et al., 2022) adopts this idea by validating previously generated data (such as explanations for answers)
2

---

Code-Generated Graph Representations Using Multiple LLM Agents for Material Propertics Prediction
(a) Permutation Invariance.
(b) Rotation Invariance.
(c) Translation Invariance.
(d) Reflection Equivariance.
(e) Periodicity Equivariance.
(f) Lipschitz Continuity.
Figure 1. Constraints of Crystalline Material Structures. For simplicity,
a 2D structure for illustration. (a) Permutation Invariance:
The atomic indices within the unit cell change. (b) Rotation Invariance: The lattice and all atomic coordinates undergo the same rotation. (c) Translation Invariance: All atomic coordinates undergo the same translation. (d) Reflection Equivariance: The material structure is mirrored along specific plane. (e) Periodicity Equivariance: The lattice coordinates change. (f) Lipschitz Continuity: The atomic coordinate undergoes small displacement
to ensure that the LLM can be fine-tuned on correctly gener- ated answers. Comera-Parrdde et al. (Romera- Pareddes et al., 2024) utilizes code executor to test LLL--generatee code, ensuring the accuracy of the generated data.
Here, = [a1, aN] E AN denotes atom types, with A representing the set of chemical elements and N rep- resenting the number of atoms in the unit cell. X aI...., ØN| RN specifies the three-dimensional coordinates of atoms in the Cartesian coordinate system. L 12, E R3x3 represents the periodic lattice, indicating the directions in which the unit cell extends in- finitely in three-dimensional space.
More relevant to our approach is the use of LLMs as source of mutations in genetic programming processes. Haluptzok et al. (Haluptzok et al., 2022) was the first to propose the concept of "evolution through large models", leveraging code-geneee gode-generating LLMs as mutation operators in genetic programming (GP). This idea was subsequently applied to various fields, including neural architecture search (Chen et al., 2024a; Zheng et al., 2023; Nasir et al., 2024), code generation (Romera-Paredee et al., 2024; Lehman et al., 2023), symbolic regression (Meyerson et al., 2024), and game exploration (Wang et al., 2023). Unlike previous tasks, applying LLMs in real-world domains requires care- ful consideration of physical rules and the generation of interpretable results to enhance trustworthiness. To the best of our knowledge, we are the first to apply the concept to the field of material representation, proposing representation of material structures that surpasses current human designed representations.
3.2. Constraints of Crystalline Material Structures
We consider six widely recognized constraints that mate- rial molecular representations should satisfy: permutation invariance, rotation invariance, reflection equivariance, lip- schitz continuity, periodicity equivariance, and translation invariance. The illustrations of these constraints shown in Figure 1. Their detailed definitions are provided below.
Permutation Invariance: For any permutation p E Sy, we have P(A, X, L) = P(Ap, X, L). This means that chang- ing the order of the atoms does not affect the representation of the material structure.
Rotation Invariance:
For any rotation matrix R E 3 we have P(A, L) = P(A, X LR). This indicates that applying the same rotation to both the atoms and the lattice does not alter the representation of the material structure.
3. Preliminaries
3.1. Material Notations
Translation Invariance: For any translation vector p3x1 we have P(A, X, X, L) = P(A, X + 11 I. + 11T), where 1 æ2xx This means that translating both the atoms and the lattice by the same vector does not change the representation of the material structure.
A material's structure can be viewed as the infinite exten- sion of its unit cell, with the unit cell serving as the small- est representative entity illustrating the material's struc- ture. material can be expressed as M = = (A,
3

---

Code-Generated Graph Representations Using Multiple LLM Agents for Material Propertics Prediction
Crossover Generation
Evaluation Summary
Agent A
Plan
Agent_
Code
Agent_
Code
LLM
LLM
LLM
Reason
Code Population
Prompt
Parent Selection
Figure 2. Pipeline of code generation framework with multiple I.M agents to ohtain representations (Rep-CodeGenn).
Reflection Equivariance: For any reflection matrix R E R3x3 we have L) + P(A, X,, LR). This im- plies that performing the same reflection on both the atoms and the lattice will change the representation of the material structure.
problem is generated by Agent_A. In the second step, complete code is generated based on the outlined plan by Agent B. Below, we describe the detailed process each step.
1.1.1. SOLUTION GENERATION
Periodicity Equivariance: For any orthogonal matrix O E R3x3, if L2 * OL1, then P(A, X, L1) * P(A, L2). This indicates that changing the lattice coordinates of the material structure will change the representation.
The goal of solution generation utilize the Agent_A devise plan for generating material representation code that satisfies all constraints, based on the physical constraints of the parent code. In this section, the input to the Agent mainly consists of three parts: the problem description, the parent code, and the task requirements. An example template of such a prompt shown the Appendix D.1.
Lipschitz continuity: the material structure m2 is ob- tained by shifting each atom's position in my by at most e, then dm2( P(m,) (m,)) < Ce, where C constant. This means that the change in the representation of the material structure should be continuous.
In the first part of the prompt, the problem description, begin by providing the Agent_A with natural language explanation of the task, followed by brief description of the constraints to be satisfied. Then, we present all the environmental code in the form of code, with the core heing an evaluate function that scores the generated representation code. The second part, the description of the parent code, includes the code of the parent code, the satisfaction of the constraints, and the potential reasons for each constraint being either satisfied or unsatisfied. These reasons and the satisfaction status are obtained from the code evaluation and Agent_C analysis in the second stage, evaluation summary. Finally, we describe the task to Agent
4. Methodology
In this section, we first introduce a code generation frame- work with multiple LLM agents obtain graph representa- tions named Rep-CodeGen. An overview of Rep-CodeGenn is shown in Figure 2. We simulate the process of lutionary algorithm by dividing the overall framework into three stages: Crossover Generation, Evaluation Summary, and Parent Selection. We will describe each component in Sections 4. 4.2, and 4.3, respectively.
4.1. First Stage: Crossover Generation
The Agent_A serves as the creative core of our evolution- ary framework, generating plans for mutation code by referencing the parent code and their constraint satisfaction status, along with the reasons provided in the prompt. Based on this information, Agent_A attempts to generate a plan for code that satisfies all constraints.
The purpose of the first phase, the Crossover Generation, to generate new material structure representation codes based on existing parent codes using Large Language Mod- els (LLMs). To further improve the quality and interpretabil- ity of the generated output, adopted two-step gener- ation strategy. In the first step, the plan for solving the

---

Code-Generated Graph Representations Using Multiple LLM Agents for Material Propertics Prediction
4.1.2. CODE GENERATION
constraint are used as labels, and the code, along with its reasons, are stored in the code population.
The goal of code generation is use Agent B to generate material representation code that satisfies all constraints, based on the modification plan generated by Agent_A. Sim- ilar to the previous section on solution generation, the input to the Ageni_B still consists of three parts: the problem description, the parent code, and the task requirements. An example template of such a prompt is shown in D.2.
4.3. Third Stage: Parent Selection
The main purpose of code population is twofold. First, it stores the codes obtained from the previous step of the Eval- uation Summary along with the reasons for their correspond- ing constraint satisfaction. Second, it serves as a source for parent codes for the first stage, Crossover Generation.
Here, the problem description and the description of the parent code the same in the previous prompt for Agent_A. The third part, the task description for the Agent_B, B, incorporates the output from Agent_A, i.e., the modification plan, as part of the input. Fach iteration gen- crates four codes, after which all the generated code is evaluated and analyzed.
Parent selection, which refers to the process of sampling codes from the code population to generate prompts in Crossover Generation, is a crucial component of this frame- work. For this, we cluster codes based their signa- tures, with codes having the same label being placed in the same cluster. The signature of the i-th code is defined as S = [sf....., sxx] where is the number of constrains, and s E {0, 13 represents the satisfaction of the k-th con- straint by the i-th code within its respective cluster. Specif- ically, s = indicates that the code's output does not satisfy the k-th constraint, while si indicates that the constraint is satisfied. The number represents the total number of constraints, which is set to 6 by default.
4.2. Second Stage: Evaluation Summary
The purpose of the second stage, Evaluation Summary, is to assess the constraint satisfaction of the generated code using the test data, and to utilize Agent_C to analyze the reasons behind the satisfaction or violation of these constraints. The detailed process of each step is described below.
To ensure diversity in sampling and comprehensive con- straint satisfaction, we prioritize selecting two codes that satisfy a larger number of constraints. For this purpose, we propose joint label for two clusters, S = [s] ,..,,,.......................... (t1 where K is the number of constrains, and the definition
The code is evaluated and scored using constructed dataset. During the evaluation process, the input to the evaluation function consists of test data for each constraint, and the output the score of the code for each constraint. We have built dedicated test data for each constraint based on the MP- 20 dataset, with the construction process and the scoring criteria for each constraint detailed below. The construction of test sets for each constraint is shown in Appendix C. To accelerate the evaluation process, a random subset of data is sampled from the constructed test set for each evaluation.
of s/e is as follows: if s = 1 or oy = 1, then a/ = 1; otherwise, 8%
2.5 =0
Based this, the probability that the i-th cluster and the j-th cluster are jointly selected as parent codes is given by:
If the code passes all test cases for a specific constraint, we consider it to satisfy that constraint; otherwise, it is deemed unsatisfied. Finally, incorrect codes (those that fail to execute within the specified time and memory limits, or produce invalid outputs) will be discarded. The correctly running code, along with its constraint satisfaction status, will be sent to the Agent_C for cause analysis.
exp
T
Σ
Ey= =1
exp
g4'
T
(1)
K
K
SWd
8k
St'
K
K
To enable Agent_A to have a more accurate and deeper understanding of the parent code, and to generate a mod- use
k=
k=
ification plan that better meets the requirements, the LLM to analyze the reasons for the code satisfying or violating certain constraints before sending the generated code to the code repository. Similar to the previous prompt for the agent, the input to Agent_G _C still consists of three parts: the problem description, the code with its constraint satisfaction status, and the task requirements. An example template of such prompt is shown in D.3.
where, N is the total number of clusters, K is the number of constrains, and the temperature parameter related to the current number of codes. Following Boltzmann distribution (Maza & Tidor, 1993), code pair with larger S . (i.e., satisfies more constraints) is more likely to be selected.
Considering the token limitations of the input and output of the large model, when selecting parent codes from a cluster, we tend to prefer shorter codes. In this case, the probability of selecting the i-th code within cluster given by:
Finally, the satisfaction status of the code on the various
5

---

Code-Generated Graph Representations Using Multiple LLM Agents for Material Propertics Prediction
0u
0
(b)
(e)
0,0
6. 0
0 6
o 0
0
0.0
0
0
0
0
0
0
0
(a)
(c)
(g)
(h)
(k)
Figure 3. Illustrations of graph representation constructed by ComFormer (a-c), PerCNet (d-h) and ours (i-k). For simplicity, we use 2D structure for illustration. (a) Illustration of ComFormer's cutoff- based neighbor selection, where the red dashed circle (with radius equal to the cutoff value) encloses all atoms (blue nodes) that are neighbors of atom u (orange node), and black nodes represent non-neighhor atoms. (b-c) Illustrations of ComFormer's 's distance and angle features, where u1, U2, and uy are the three nearest neighbors of atom and v is the neighbor of atom u. (d) Illustration PerCNet's cutoff-based neighbor selection. (e-h) Illustrations of PerCNet's distance, angle, and dihedral angle features, where U1 is the nearest neighbor of atom t2 is the neighbor with the smallest angle relative to the edge uv, and 81 is the nearest neighbor of atom v. (i) Illustration of our proposed satice-based neighbor selection, where all atoms (blue nodes) within the red dashed box neighbors of atom (orange node). (i-) Illustrations of our proposed distance and dihedral angle features, where in and U2 are atoms obtained by extending atom a along the lattice directions.
all six constraints) is found. Ultimately, in the 14,076th iteration, we identified code that successfully passed all test cases. In the process of finding codes that satisfy six con- straints, the framework generated total of 56,304 codes, of which 39. 99.60% (22,297) were executed successfully, Among these, 4,009 codes satisfied four constraints, 1 14,645 codes satisfied five constraints, and I code satisfied six constraints. These results demonstrate that Rep-CodeGen is capable of generating code representations that satisfy new constraints.
exp (-G
Pj
Ex= exp
where, Ne. is the total number of codes in the cluster, and C2 is the character count of the i-th code. With Equation and 2, we achieve the goal that code pairs satisfying more constraints and having shorter lengths are more likely to be selected as parent codes.
Note that we employ three pre-trained QWen (Hui et al., 2024) LLMs as distinct agents, and the entire process does not require further retraining of the models and the results other LLMs are shown in section 5.3.
5. Experiments
In this section, we address three key questions: (Q1) Can our framework identify representation that satisfies all six aforementioned constraints, and how does this repre- sentation differ from existing methods? (Q2) How does the proposed material representation perform in material property prediction tasks? (Q3) How effective is our pro- posed multiple LLM agent architecture in generating graph representation code when facing new constraints?
Material Graph Representation. The material represen- tation code obtained by Rep-CodeGen is provided in Ap- pendix B. Specifically, the proposed representation that sat- isfies the physical constraints can be expressed as [u, v, attr]. where u represents the atomic number of all atoms in the unit cell, and v represents the atomic number of each neigh- bor of atom u. The attribute attr- {duues Quv) includes the distance and dihedral angle information between atoms and v. The illustrations of dus and ou. are shown in Figure 3 (i) and (k), where du represents the edge distance be- tween atom u and atom v, with atom e being any neighbor of atom u. Owe represents the dihedral angle between the half-planes wok and uol. This angle is also related to the direction of normal vectors of the two half- half-planes and vector wv. Here, atoms i and are obtained by extending atom along the lattice direction.
5.1. A Representation Obtained by Codes of Rep-CodeGenn
In this section, we answer question Q1. First, we introduce the results of Rep-codeGen. Then, we provide a detailed ex- planation of the material graph representation that satisfies six constraints. Finally, we thoroughly discuss the differ- ences between our proposed representation and existing graph representation methods.
Results of Rep-CodeGen. The framework employs codes satisfying three constraints as bases and iterates until a ma- terial representation that passes all test cases (i.e., satisfies
Differences from Existing Graph Representation Meth- ods. The proposed representation differs from existing
6

---

Code-Generated Graph Representations Using Multiple LLM Agents for Material Propertics Prediction
Table Comparison hetween our proposed material representation and other baselines in of test MAE JARVIS dataset and The Materials Project dataset. The best results are shown in bold, and the second best results are shown with underlines. The number in parentheses after algorithm represents the number constraints that the algorithm satisfies.
JARVIS
Materials Project
Method
Energy Hull
Total Energy
BandGap
Formation Energy
Formation Energy
BandGap
meV
me V/tom
meV
me
me metom
meV
CGCNN (4)
170
78
410
63
292
ALiGNN (4)
76
37
310
33.1
22.1
218
MatFormer (4)
64.2
35
300
32.5
21
211
PotNet (4)
55.4
32.4
273
29.4
18.8
204
PerCNet (5)
50.3
30.7
265
28.7
18.1
200
ComFormer (5)
47
28.8
260
27.2
18.3
193
Ours (6)
43.3
28.3
253
27.2
18.0
191
graph algorithms primarily in two aspects: the method of acquiring neighbors v and the construction of features attr. Below, we illustrate the distinctions of our representation by comparing with the most constrained graph representation methods, namely ComFormer and PerCNet.
Detailed proofs demonstrating that our representation sat- isfies all constraints proposed in Section 3.2, as well as the role of the aforementioned distinctions, are provided in Ap- pendices E. I through E.6. To the best of our knowledge, this material representation is the first to simultaneously satisfy all of these constraints.
First, regarding the method of acquiring neighbors, exist- ing methods, such as ComFormer( Yan et al., 2024) and PerCNet(Huang et al., 2025) in Figure 3, predominantly determine neighbor relationships based on a cutoff distance. Specifically, atoms within a distance smaller than the cutoff are considered neighbors. This approach lacks Lipschitz continuity for atoms near the cutoff boundary, meaning that minor positional changes of boundary atoms can lead to significant alterations in neighbor relationships. In contrast, our representation innovatively identifies neighbors based on periodicity. For any atom within a unit cell, all atoms within a 3 x 3 x 3 periodic region surrounding the unit cell are considered neighbors, ensuring continuity to positional variations.
5.2. Performance Comparisons on Material Property Prediction tasks
In this section, we aim answer question Q2 by evaluating the performance of the proposed presentation in terms of Mean Absolute Error (MAE), consistent with prior studies (Huang et al., 2025; Lin et al., 2023; Yan et al., 2022; Xie & Grossman, 2018).
Setup. The Materials Project-2018.6 dataset contains 69,239 materials, and the JARVIS-DFT-2021.8 dataset contains 55,722 materials. To ensure fair com- parison, we adopt the same data settings as previous works (Huang et al., 2025; Xie & Grossman, 2018). The statistics of data settings and neural network configurations are shown in the Appendix F.I.
3D
Second, in terms of feature construction, ComFormer fo- cuses on distance and angle information, while PerCNet incorporates distance, angle, and dihedral angle informa- tion. Our representation, on the other hand, emphasizes distance and dihedral angle information. Notably, although both our representation and PerCNet model dihedral angles, our construction method differs significantly. As shown in Figure 3, in PerCNet's representation, among the four atoms forming two faces, the two atoms besides atom u and its neighbor e are selected based on their distance to atom u. This distance-based selection still suffers from the lack of Lipschitz continuity. In contrast, our representation selects the remaining two atoms based on lattice coordinates related periodicity. Consequently, even if atomic coor- dinates fluctuate, the derived distance and dihedral angle information maintain Lipschitz continuity.
Results. The results of various graph-based representation algorithms on the property prediction task are shown in Ta- ble 1, and point cloud-based algorithms shown in Table 5 in the Appendix. Across all tasks, based the material representation proposed by our multiple-agent architecture, our method achieves the best performance. Notably, while we employ the same network architecture as PerCNet for processing graph representations, our method demonstrates improved prediction accuracy across all tasks compared to PerCNet. The most significant improvement is observed in the prediction accuracy of Energy Hull, with an increase of 13.9%. This result highlights that the performance gains stem from the superiority of our representation rather than the neural network architecture
7

---

Code-Generated Graph Representations Using Multiple LLM Agents for Material Propertics Prediction
Table 2. Constraint satisfaction results of representation codes generated by different large models under different frameworks. Algorithm
variants with (Ie3) denote 1,000 candidate code generations. The goal of this experiment is to satisfy five constraints. Bold values highlight bond metric superiority of our Rep-CodeGen framework over direct generation baselines.
Constraint Satisfaction Spectrum
Algorithm
Improve Rate
Zero
One
Two
Three (Initial)
Four
Five
G.T' (1e3)
0.00%
0.00%
0.00%
14.39%
65.61%
0.00%
0.00%
GPT +
(1e3)
5,41%
2.70%
10.81
5.41%
75.68%
5.41 %
0.00%
Deep Seek (1e3)
0.00%
0.00%
0.00%
0.00%
100.00%
0.00%
0.00%
Seek + Rep-CodeGeen (Ie3)
26.27%
0.00%
0.00%
0.42%
73.31%
26.27%
0.00%
QWen (Ie3)
13.51%
2.70%
2.70%
10.81%
70.27%
13.51 %
0.00%
Qwen + Rep-CodeGen (1c3)
66.53%
1.15%
1.54%
6.92%
23.85%
40.38% %
26.15%
5.3. Performance Comparison on Evolution Ability of Rep-CodeGen
contrast, the Deepseek with Rep-CodeGen shows the explo- ration ability and finds the codes satisfying four constraints. Finally, the QWen shows the exploration ability and evolu- tion ability. However, it can't generate codes satisfying five constraints. By contrast, Rep-CodeGen can significantly improve the abilities of QWen and generate codes satisfy- ing five constraints. Overall, Rep-CodeGies demonstrates evolutionary capabilities across all three LLMs, supported by its enhanced exploration ability.
In this section, we aim to address question Q3 by conduct- ing a comparative analysis between two distinet experimen- tal conditions: the performance of large language models (LLMs) when augmented with CodeGen versus their baseline capabilities in standalone operation.
Setup. We conduct experiments on three different large language models: QWen (Hui et al., 2024), DeepSeek (DeepSeek, 2025), and ChatGPT-3.5 (OpenAI, 2023). Each LLM is evaluated on graph representation code generation tasks under two operational settings: (1) LLM solely, (2) LLM enhanced with Rep-CodeGen. To ensure a fair com- parison, identical prompt templates are applied across corre- sponding configurations, with the baseline LLM prompting strategy detailed in Appendix F.2. Notably, in contrast to the aforementioned experiments, our objective is to identify representations that satisfy five known constraints, with the number of evolutionary iterations restricted to 1000 due to the resource limitation. As the code of graph representations satisfies three constraints is used as bases, the framework shows evolutionary ability if the generated codes of graph representations satisfy four and five constraints. By contrast, the framework shows degeneration if the generated codes of graph representations satisfy two or fewer constraints.
6. Conclusion
In this work, we focus on the challenge of obtaining graph representations of materials that are effective when facing new constraints. We propose Rep-CodeGen with iterative stages, code generation framework with multiple LLM agents, to generate high-quality codes obtain graph rep- resentations. To the best of our knowledge, Rep-CodeGen is the first framework for automatically generating codes to obtain graph representations. We take representation con- struction task as a code-generation task, which can reduce expert knowledge to a large extent. The researchers in mate- rial science can focus more on investigating the constraints brought by materials, while the researchers in computer sci- ence can focus on improving the quality and efficiency of generating methods. Furthermore, type of representation from generated codes by our framework satisfies six con- straints with codes satisfying three constraints as bases. We detailed analysis the difference between the obtained graph representations and existing graph representations, and pro- vide chemical insights into such representations. Extensive experiments on two real-world material datasets show that property prediction method based on such graph repre- sentations achieves -tate-of-the-art performance in material property prediction tasks. Furthermore, to evaluate the gen- eralizability of our framework, we conduct experiments on different LLMs. The Rep-CodeGen shows evolution abil- ity based on different I.I.Ms, while the single LLM agent shows limited ability when facing new constraints. In the fu- ture, the proposed framework will be utilized to streamline material structure generation tasks.
In this section, employ two complementary metrics: Im- prove Rate: The proportion of executable codes that satisfy more constraints than the initial code. Constraint Satisfac- tion Spectrum: Distribution of codes satisfying incremental constraint counts (0-5) among successful executions.
Results. As shown in Table 2, our framework significantly improves the quality of generated code compared directly using LLMs. For GPT3.5, we can see the quality of codes shows a degeneration to satisfy the two constraints, while GPT3.5 with Rep-CodeGen shows both evolution and de- generation. Note that, both the degeneration and evolution show the exploration ability, which is key to the success of generation models. The Deepseek shows no exploration ability for this task and fails to identify satisfied codes. By
8

---

Code-Generated Graph Representations Using Multiple LLM Agents for Material Propertics Prediction
Acknowledgements
Haluptzok, P., Bowers, M., and Kalai, A. T. Language models can teach themselves to program better. arXiv 2022.
This work was supported by the National Natural Sci- ence Foundation of China underGrant Nos. U22A2098, 62172185, 62206105, 62202200, and 02406127; the Major Science and Technology Development Plan of JilinProvince under Grant No. 20240302078GX; and the Major Science and Technology Development Plan of Changehun under Grant No. 2024055
preprint
Huang, J., Xing, Q. Ji, J., and Yang, B. Perenet: Peri- odic complete representation for crystal graphs. Neural Networks, 181:106841, 2025.
Hui, B., Yang, J., Cui, Z., Yang. J., Liu, D., Zhang, L., Liu, T., Zhang, J., Yu, B., Dang, K., et al. Qwen2. 5-coder technical report. arXiv preprint wrXiv:2409.121660 2024.
Impact Statement
Keqiang, Y., Cong. F. Xiaofeng, Q., Xiaoning, Q., and Shuiwang, J. Complete and efficient graph transformers for crystal material property prediction, 2024. URL https: /openreview net forum? i id=Bngy9Xi RAS.. Accessed: 2025-1-30.
This paper presents work whose goal advance the filed of AI4Science, especially for material science. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Lehman, J., Gordon, J., Jain, S., Ndousse, K., Yeh, C., and Stanley, K. O. Evolution through large models. In Handbook of Evolutionary Machine Learning, pp. 331- 366. Springer. 2023.
References
Chen, A., Dohan, D., and So, D. Evoprompting: language models for code-level neural architecture search. Ad- vances in Neural Information Processing Systems, 36, 2024a.
Lin, Y.C.,, Yan. K., Luo. Y., Liu, Y.. Qian, X., and Ji, S. Efficient approximations of complete in- teratomic potentials for crystal property prediction. In International Conference on Machine Learning, 2023. URL https: ptttpa://api..smmnt... ptg/CorrusID::5520000 semanticcchonlll................
Chen, C., Ye, W., Zuo, Y., Zheng, C., and Ong, P. Graph networks universal machine learning framework for molecules and crystals. Chemistry of Materials, 31(9): 3564-3572, 2019.
Chen, C., Nguyen, D. T., Lee, S. J., Baker, N. A., Karakoti, A. S., Lauw, L., Owen, C., Mueller, K. T., Bilodeau, B. A., Murugesan, V., et al. Accelerating computational materials discovery with machine learning and cloud high- performance computing: from large-scale screening to experimental validation. Journal of the American Chemi- cal Society, 20009--000000020000 2024b.
Louis, S.-Y., Zhao Y., Nasiri, A., Wang, X., Song, Y., Liu, F., and Hu, J. Graph convolutional neural networks with global attention for improved materials property pre- diction. Physical Chemistry Chemical Physics, 22(32): 18141-18144 (1881141440 2020.
Maza, M. d. I. and Tidor, B. An analysis of selection proce- dures with particular attention paid to proportional and boltzmann selection. In Proceedings of the 5th Interna- tional Conference on Genetic Algorithms, pp. 124-131, 1993.
Choudhary, K. and DeCost, B. Atomistic line graph neural network for improved materials property predictions. npj Computational Materials, 7(1):185, 2021.
Damewood, J., Karaguesian, J., Lunger, J. R., Tan, A. R., Xie, M., Peng, J., and Koomez-Bombarell,, R. Representa- tions of materials for machine learning. Annual Review of Materials Research, 53(1):399 426, 2023.
Meredig, B., Agrawal, A., Kirklin, S., Saal, J. E., Doak, J. W., Thompson, A., Zhang, K., Choudhary, A., and Wolverton, C. Combinatorial screening for new mate- rials in unconstrained composition space with machine learning, Physical Review B, 89(9)0994104, 2014.
DeepSeek. Deepseek platform, 2025. URL https: //platform. deepseek. cwm/cs//a/aaaeeeeeeeeeeee usage. Accessed: 2025-01-24.
Meyerson. E., Nelson, M. J., Bradley, H., Gaier, A., Moradi, A., Hoover, A. K., and Lehman, J. Language model crossover: Variation through few-shot prompting. ACM Transactions on Evolutionary Learning, 4(4):1-40, 2024.
Ding, Q., Miret, S., and Liu, B. Matexpert: Decomposing materials discovery by mimicking human experts. In A/ for Accelerated Materials Design-NeurIPS, 2024.
Nasir, M. U., Earle, S., Togelius, J., James, S., and Cleghorn, C. Llmatic: neural architecture search via large language models and quality diversity optimization. In proceedings of the Genetic and Evolutionary Computation Conference, pp. 1110-1118. 2024.
Fang, Z. and Yan, Q. Towards accurate prediction of con- figurational disorder properties in materials using graph neural networks. npj Computational Materials, 10(1):91, 2024.
9

---

Code-Generated Graph Representations Using Multiple LLM Agents for Material Propertics Prediction
Oliynyk, A. O., Antono, E., Sparks, T. D., Ghadbeigi, L.,, Gaultois, M. W., Meredig, B., and Mar, A. High- throughput machine-teaming-drrveee synthesis of full- heusler compounds. Chemistry of Materials, 28(20): 7324-7331, 2016.
Yan, K., Fu, C., Qian, X., Qian, X., and Ji, S. Complete and efficient graph transformers for crystal material property prediction. In The Twelfth International Conference on Learning Representations, 2024.
2016.
Zelikman, E., Wu, Y., Mu, J., and Goodman, N. Star: Boot- strapping reasoning with reasoning. Advances Neural Information Processing Systems, 35:15476-15488, 2022.
OpenAl. Introducing chatgpt. b pllg/chhttgp... 2023. Accessed: 2025-01-24.
Ramprasad, R., Batra, R., Pilania, G., Mannodi- Kanakkithodi, A., and Kim, C. Machine learning in materials informatics: recent applications and prospects. upj Computational Materials, 3(1):54, 2017.
Zheng, M., Su, X., You, S., Wang, F., Qian, C.,, Xu, C., and Albanie, S. Can gpt-4 perform neural architecture search? arXiv preprint arXiv:2304. 10970, 2023.
Comera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M. P., Dupont, E., Ruiz, F. J., Ellenberg, S., Wang, P., Fawzi, O., et al. Mathematical discoveries from program search with large language models. Nature, 625 (7995):468 475, 2024.
Schütt,, K., Kindermans, P.-J., Sauceda Felix, H. E., Chmicla, S., Tkatchenko, A., and Müller, K.-R. Schnet: A continuous-filee convolutional neural network for model- ing quantum interactions. Advances in neural information processing systems, 30, 2017.
Tao, T., Li, Tan, B., Wang. H., Marshall, W., Kanakiya, B. M., Hestness, J., Vassilieva, N., Shen, Z., Xing, E. P., et al. Crystal: Illuminating Ilm abilities language and code. arXiv preprint arXiv:241 1. .41666, 2024.
Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., and Anandkumar, A. Voyager: An open-ended embodied agent with large language models. Transac- tions on Machine Learning Research, 2023.
Ward, L., Agrawal, A., Choudhary, A., and Wolverton, C. A general-purpose machine learning framework for predict- ing properties of inorganic materials. npj Computational Materials, 2(1):1-7, 2016.
Widdowson, and Kurlin, V. Resolving the data ambiguity for periodic crystals. Advances in Neural Information Processing Systems, 35:24625-2466 2022.
Widdowson, D., Mosca, M. M., Pulido, A., Cooper, Andrew, I, and Kurlin, V. Average minimum distances of periodic point sets foundational invariants for mapping periodic crystals. Match, (3):87, 2022.
Xie, T. and Grossman, J. C. Crystal graph convolutional neural networks for an accurate and interpretable predic- tion of material properties. Physical review letters, 120 1443:145301, 2018
Yan, K., Liu, Y., Lin, Y., and Ji, S. Periodic graph transform- ers for crystal material property prediction. Advances in Neural Information Processing Systems, 35:15066- 15080, 2022.
10

---

Code-Generated Graph Representations Using Multiple LLM Agents for Material Propertics Prediction
A. The fulfillment of physical constraints of related works
The fulfillment of physical constraints by various algorithms is shown Table
Table 3. The fulfillment of physical constraints by various algorithms, where indicates that the algorithm does not satisfy the constraint, indicates that does.
Permutation Invariance
Rotation Invariance
Reflection Equivariance
Lipsehitz Continuity
Periodicity Equivariance
Translation Invariance
Algorithm
CGCNN
ALIGNN
MatFormer
PotNet
X
materialLLM
X
X
X
MatExpert
X
PerCNet
X
ComFormer
X
AMD
PDD
Ours
B. Material representation code proposed by Rep-CodeGen
The code material representation generated by our proposed Material Representation Code Generation (Rep-CodeGen) with multi-agent architecture is shown in Code 1. To enhance readability, we have manually adjusted the variable naming conventions the generated code align with those used in the paper and added necessary comments, while keeping other parts unchanged.
Listing 1. Code of material representation generated hy code generation framework with multiple L.M agents (Rep-CodeeGen)
@unsearch eeelvv
def representatien (dataset list: list) -> list:
otoms_ comm__ttt_list []
for atoms_date in dataset_list
coords aa..rrrylattss_daattt ("ccords"])
elements atoms_data [ [" ements"] ("eletttta))
lattice_ma np.array (atoms_data ("latticc_matt"
num atoms (coords)
extended atoms []
Extend atcms based lattice translations
for dx [0, -1, 1]:
for dy in [0, -1, 1]: for dz in [0, -1, 1]:
13
14
for i in range :oum_atoms):
ccord_u coords
element_u elements [i]
new_ccord np.array (ccord_u dx mattice_mation + dy Lattice mat [1] + dz
18
Lattice_mati [2])
extended_atoms. wtwpdaattmssppppppss ([elements [i], new_coord])
Attr_list - []
for i in range (num_aat atoms)
22
coord_u u extended atoms [i] [1]
element_te aetomic_numban Textended atoms [i] [0]]
distances []]
26
I Calculate distances to all ether atoms
for j in range (len (extended_atomms)
if !- j: coord_ extended_atom [i] [1]
30

---

Code-Generated Graph Representations Using Multiple LLM Agents for Material Propertics Prediction
Pllement_to_attomm___umeeees Lettenedeeeeeeetteeeeeeetttsss []] [0]]
distance np. (cooo coord_v)
32
sistancce..appedd ([), distance])
distances. sort (key=lambda x[1])
nearest_neighhors distances
36
Process nearest neighbors
37
for neighber in nearest_neighbozz
neighbor
distance neighbor[1]
u element_to_atomice wtttooomimuummaaae c_numberrrreeeeeeeeeeeeeeeeeee
hataoooooooootttteeeeeeedeeddttr (extended_atooms [1] [0]]
v e
c_numbrrrmnnnneerreereee [eetrtoeeeeoneneeeeeeeeeeeeeeeee [extended_atome []] [0]]
42
A roun stance, 2)
Attr .ip... [u, v, Attr
Calculate dihedral angle
46
Iccordd - extended_atoms [i] [1]
v_ceord extended_atoms [i] [1]
c_cordd extended_atoms [i num_atoms] [1]
c_crr extended_atooms [i + num_atoms [1]
50
uv np.array (v__oray
app.rrayyuu_corr)
uk ap.array (k_coord) np. np.array (u_coord)
ul ap. ap.array sarray(__coon))
ap.array (u_coor))
normal_ukk np. (uk, ul)
55
normal_uve = crocross (uv, uk)
56
cos_angle np. (normal_ukl, normal_uvk) / (np.linalg, norm (normal_uk1) np. linalg.norm (n..ommmmoomma__uxx11
cos oas_angle pp. mp.cclppcoos_nngee -1.0, 1.0)
angle = ap.arcoo cross_ (
59
cross (normal_ukk, ukk, normal_uvk)
if np. (cross_prodd uv) 0:
dihedral_angle np deggrecssanggll elsc:
63
dihedral_angle = np.degrees (np.pi angle)
64
Attr_ 1ir__isse Attr_list. [u, dihedral_anglet
Attr Attr_list. sert Lkey-laaddaa x: (x10), x111, x
atoms Att lesttr_listt appt. (Att_
68
return atoms_Attr_list
C. The construction of test sets for each constraint
Permutation Invariance: For each material structure in the mp-20 dataset, rearrange the order of all the atoms. If the representation of the material structure changes after rearranging, it is considered that the representation does not satisfy permutation invariance.
For each material structure in the mp-20 dataset, we rotate all the atomic coordinates and lattice coordinates within the unit cell by random angle. If the representation of the material structure changes after rotation, it considered that the representation does not satisfy rotation invariance.
Rotation Invariance:
Translation Invariance: For each material structure in the mp-20 dataset, we translate all the atomic coordinates within the unit cell by the same random vector. If the representation of the material structure changes before and after translation, it is considered that the representation does not satisfy translation invariance.
Reflection Equivariance:
For each material structure in the mp-20 dataset, we perform a mirror reflection of all the atomic coordinates and lattice coordinates along a randomly chosen plane. If the representation before and after the reflection remains the same, it considered that the representation does not satisfy reflection equivariance.
Periodicity Equivariance: For each material structure in the mp-20 dataset, we modify the lattice coordinates in various ways such that the modified coordinates cannot be obtained by rotating or translating the original lattice
12

---

Code-Generated Graph Representations Using Multiple LLM Agents for Material Propertics Prediction
coordinates. If the representation of the material structure before and after the modification remains unchanged, it is considered that the representation does not satisfy periodicity equivariance.
Lipschitz Continuity: For each material structure in the mp-20 dataset, make small changes to the coordinates of some atoms in the unit cell. If the material representation undergoes abrupt changes (such as change in neighbor relationships) or the difference between the representation before and after modification is too large, it is considered that the representation does not satisfy Lipschitz continuity.
D. Prompt of all agents
D.I. Prompt Agent_A
An example template of the prompt of Agent_Ais:
want to design function that can construct graph representation
of material molecules that satisfies specific constraints. The graph representation of the material molecule should meet the following requirements: <Symeetry.onstraaite
|n Below is the code that describes this problem, which does not require modification: <Problem.Code>, \n Next is the function that needs hnl.fiaaaionnn:eeeseeettee
Code>. \n The constraint satisfaction the material graph representations generated by the above representation function is as follows: <Satisfaction In Your task is to outline a clear plan and detailed steps to design new representation function that fulfills all the specified constraints.
D.2. Prompt of Agent_B
An example template of the prompt of Agent_B is:
I want to design function that can construct a graph representation of material molecules that satisfies specific constraints. The graph representation of the material molecule should meet the following requirements: <Symmetry Constraints>. in Relow is the code that descríbes this problem, which does not require modification: <Froblem.code>.bk In Next is the function that needs opt imtimize maiionni:: <Representation.olee >. \n hope to rewrite new representation funct ion to replace the previous representation function, representation it generates satisfies all constraints. Here is my proposed plan: Moodification.PP.. >. |n Your task is to rewrite a new representation function to replace the previous one, based on the plan outlined above.
that the graph
D.3. Prompt of Agent_C C
An example template of the prompt of Agent_C'in is:
want to design a function that can construct a graph representation material molecules that satisfies specific constraints. The graph representation of the material molecule should meet the following requirements: <symmttry.onnstzaantt> \n Below is the code that describes this problem, which does not require modification: <Froblem_code>.1 optimization: <RepresentatioonCode >. |n The constraint satisfaction of the material graph representations generated by the above representation function is as follows: <constraint.aatisfcciiio >. in Your task is to analyze the reasons for each constraint being satisfied or not satisfied, one by one. Please present the complete reasoning.
Next is the function that needs
D.4. Description of Constraints
Below the full description of the constraints:
13

---

Code-Generated Graph Representations Using Multiple LLM Agents for Material Propertics Prediction
Permutation invariance: Changing the atomic indices should not alter the graph representation.
Rotation invariance: Rotating the atomic coordinates should not change the graph representation.
Reflection equivariance: Performing a mirror symmetry on the lattice and atomic coordinates should result in a change in the graph representation.
Lipschitz continuity: If the coordinates of the crystal undergo continuous changes, the corresponding graph repre- sentation should also change continuously. This means the neighbor relationships of each atom (i.e., which atom is a neighbor) should not change, but the attributes of the corresponding neighbors (such as distance or angle) should vary accordingly.
Periodicity equivariance: The graph representation should implicitly incorporate lattice information, meaning that any modification to the lattice coordinates (e.g., scaling the lattice by 1.5 times) should result in a change in the graph representation.
Translation invariance: Translating the atomic coordinates should not affect the graph representation.
E. Proof that the proposed representation satisfy material symmetry properties
Below, we will prove that the material representation found by Rep-CodeGen satisfies all the constraints proposed in Section 3.2, based on the representation formula in Section 5.1.
E.1. Permutation Invariance
As shown in Section 5.1, the features of atom u and its neighboring atom v depend solely on the atomic indices. For example, atom u is a carbon (C) atom, its feature value would 6. Therefore, changing the arrangement of atoms does not affect the final representation.
E.2. Rotation Invariance
The variables in attr, which contain distance and dihedral angle information, are relative quantities. Therefore, applying rotation the coordinates does not affect the material representation
E.3. Reflection Equivariance
When the coordinates of the material undergo reflection, the directions of the vectors within the material will change. a result, the product of the variables cross.prai and uv on line 61 of Code will be altered. Therefore, the representation obtain satisfies reflection equivariance.
E.4. Lipschitz Continuity
The feature triplet [u, v, attr] for each atom within the unit cell exhibits continuity primarily in two aspects: the feature value attr and the neighbor relationship v. Since attr directly related to the coordinates, it inherently satisfies continuity. The current method, however, employs cutoff approach, which can lead to abrupt changes in the neighbor relationships. In contrast, the representation we propose considers all atoms within 3 x 3 x 3 lattice neighbors, ensuring that the neighbor relationships do not change due to small variations in the coordinates, thereby satisfying Lipschitz continuity.
E.5. Periodicity Equivariance
For any atom u within the unit cell and its neighboring atom the coordinates of v are dependent on both the atomic coordinates within the lattice and the lattice coordinates. Therefore, when the periodicity of the material changes, i.e., when the lattice coordinates are modified, the coordinates of neighbor e will also change. As a result, the distance feature between and will be adjusted accordingly, which ensures that the material representation satisfies periodicity equivariance.
14

---

Code-Generated Graph Representations Using Multiple LLM Agents for Material Propertics Prediction
E.6. Translation Invariance
The variables in attr, which contain distance and dihedral angle information, are relative quantities. Therefore, applying translation to the coordinates does not affect the material representation.
Table 4. Statistics of datasets.
JARVIS
Materials Project
Dataset
Formation Energy
Total energy
Bandgap
Ehull
Formation Energy
BandGap
# training
44578
44578
14537
44296
60000
60000
# validation
5572
5572
1817
5537
5000
5000
# testing
5572
5572
1817
5537
4239
4239
F. Experimental Details
F.1. Experimental Setup of material Property Prediction tasks
The statistics of data settings are shown in Table 4. For all tasks, we utilize one NVIDIA RTX 24G 3090 GPU for computation. To ensure fair comparison, employ the same network architecture and training methodology as PerCNet(Huang et al., 2025), which also incorporates dihedral angle information. In terms of implementation, all models are trained using the Adam optimizer with a one-cycle learning rate scheduler. The training configuration includes learning rate of 0.001, batch size of 64, and total of 500 training epochs.
F2. Prompt of LLM without architecture assistance
An example template of the prompt of LLM :s:
I want to design function that can construct a graph representation of material molecules that satisfies specific constraints. The graph representation of the material molecule should meet the following requirements: <symeetry.constraintts.
\n Below is the code that describes this problem, which does not require modification: <Problem.code>.1 \n Next is the function that needs modi :icaaain:n Reeresenntteeeeetattttiioo Code>.\n Your task is to provide new, modified function ion that will replace the current priority function. This new function should generate graph representation that satisfies all the given constraints, and the function signature should remain the same as the original priority function. Please ensure that the new function adheres to all the necessary constraints and provides a correct implementation. Only the full code of the new function should be returned, with no additional comments, explanations, or suggestions.
F.3. Comparison with point cloud-based representation methods
The results of point cloud-based algorithms are shown in Table 5.
Table Comparison between proposed material representation and other baselines in terms of test MAE JARVIS dataset in formation energy task. The results of baselines are from related work (Keqiang el al., 2024)
Method
Number training materials
Test MAE (e (eVatom)
PDD( WWiddoddosnnn & Kurlin, 2022)
29342
0.047
AMD(Widdowson al., 2022)
24067
0.78
Ours
24067
0.027
15


Reviews:
[{"review_id": "xm7PzwyocZ", "reviewer": "Reviewer_qN76", "summary": "This paper introduces Rep-CodeGen, a framework that uses multiple LLM agents to autonomously generate graph representations for material property prediction. Unlike traditional methods, Rep-CodeGen iteratively refines representations through crossover generation, evaluation summary, and parent selection, ensuring adaptability. The framework optimizes graphs to satisfy six material constraints, improving prediction accuracy. Experimental results demonstrate its superiority over conventional approaches. By integrating LLMs for automated representation learning, this work reduces reliance on domain expertise, offering a promising solution for AI-driven materials science.\n\n## update after rebuttal\nDuring the rebuttal period, I interacted with the authors. The authors have answered my questions well, so I decided to raise my score for the paper.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable time and your recognition of our work's motivation, novel contributions, and experimental design. We provide point-to-point responses to your questions as follows.\n\nQ1: The neighborhood selection is the main difference between the 'new' representation and the existing methods. Why does this change affect the property predictions in Table 1?\n\nR1: The neighbor selection method in the generated representation enhances the graph's capability to capture long-range atomic interactions. As a result, the energy-related properties are improved.\nSpecifically, the interactions between atoms are not limited to adjacent atoms, and long-range interactions can extend to distant atoms, which play a key role in the stability of crystals. Taking ionic crystals as an example, the Coulomb force between ions is a type of long-range interaction with a relatively large range of action. Cations and anions are attracted to each other through the long-range Coulomb force, which maintains the structural stability of the crystal. In our representation, neighbors are determined based on periodic regions instead of a cutoff value, which can cover the atoms over a longer distance. Besides, the long-range interactions also affect the electrical and thermal properties of crystal materials.\n\nQ2: The LLMs may train on methods and their codes in Table 3. Thus, the 'new' representation may be a combination of the existing codes. Then, how can the framework satisfy the unseen constraints? \n\nR2: LLMs can be trained not only on materials science research papers but also on works from other fields, such as protein design and drug discovery. With all these training resources, LLMs have a better understanding of atom interactions inside the materials. Therefore, we aim to guide the LLMs in generating the desired representations. To the best of our knowledge, the neighborhood selection for the new representation, which is based on periodic regions, has not been observed. As shown in Table 2, the representations satisfying four and five can be viewed as unseen constraints. Among these representations, we also observed 'new' representations that are different from existing methods.\n\nQ3. This framework costs a lot of tokens to generate a target code. How to compare the cost from both time and money to DFT calculations? Besides, what is the size of LLMs in Table 2? As the LLMs become more and more powerful, a new version of GPT or Deepseek may accomplish this task well.\n\nR3: The cost of Rep-CodeGen is minimal compared to DFT calculations. We utilize the Qwen2.5-Coder-7B model to develop three agents, opting for the 7B version over the Qwen2.5-Coder-32B primarily due to cost considerations. While the Qwen2.5-Coder models are open source, the operational expense of the 32B model is significantly higher than that of the 7B model. Additionally, the 7B model offers considerably faster inference times. Our aim is to create a framework accessible to a broad spectrum of researchers. Specifically, we only utilize one NVIDIA RTX 24G 3090 GPU for all experiments. \n\nA larger LLM can benefit the framework, but a single one may not accomplish the task well, as shown in Table 3. GPT-3.5 (gpt-3.5-turbo used in our experiments) and DeepSeek (DeepSeek-Coder-v2 used in our experiments) represent a larger general-purpose LLM and a larger code-generation LLM, respectively. Nonetheless, their performance remains unsatisfactory.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "Yes.", "experimental_designs_or_analyses": "Yes.", "relation_to_broader_scientific_literature": "The 'new' representations can benefit the material virtual screening.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "**Strength:**\nThe generated codes are more interpretable, allowing humans to read and edit them easily. This AI knowledge may further inspire human researchers.\nThis paper shows that a good representation can improve the prediction results with the same model.\n\n**Weakness:**\nPlease see the questions.", "other_comments_or_suggestions": "I keep up with the literature in this area.", "questions_for_authors": "1. The neighborhood selection is the main difference between the 'new' representation and the existing methods. Why does this change affect the property predictions in Table 1？\n\n2. The LLMs may trained on methods and their codes in Table 3. Thus, the 'new' representation may be a combination of the existing codes. Then, how can the framework satisfy the unseen constraints? \n\n3. This framework costs a lot of tokens to generate a target code. How to compare the cost from both time and money to DFT calculations? Besides, what is the size of LLMs in Table 2? As the LLMs become more and more powerful, a new version of GPT or Deepseek may accomplish this task well.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cQHm8tlAms", "reviewer": "Reviewer_RrQ6", "summary": "This paper proposes a novel code generation framework for material property prediction. The LLM agents are employed to replace the human experts and automatically generate codes to process the CIF files fitting GNN-based models. After the processing, the obtained input vectors are called representations. Due to the symmetry of crystals, the representation should satisfy a few constraints. This paper selects six widely used constraints as the target of the generated representations. Note that the code used from the beginning satisfies three types of constraints. As the framework evolves, the generated representations can satisfy six types of constraint, which did not exist before. This ability is summarized as \"obtain representations that can be used when facing new constraints\" by authors. The experiments show how the generated codes process the CIF files. The main difference is the neighborhood selection compared to the expert-designed methods. Furthermore, the generated representations with an existing GNN-based property model can achieve the SOTA performance in property prediction tasks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much to the reviewers for your time and for recognizing the innovation of our method and its contributions to the field. We will revise the paper carefully. Below, we will summarize your reviews and provide a response to each point separately.\n\nQ1: The experiments are stopped at five due to resource limitations. Why the results of GPT+Rep-CodeGen and Seek+Rep-CodeGen are 0 for five constraints in Table 2? Does this mean that the framework is only effective for QianWen?\n\nR1: Although the choice of five constraints is employed due to the resource limits, it can show a clearer comparison between LLMs with and without our framework. The zero results for GPT+Rep-CodeGen and Seek+Rep-CodeGen occur because we capped the experiment at 1,000 programs to ensure a fair comparison. We should also pay attention to the results other than three constraints which also reflect the evolutionary capability of our framework. Thus, the framework is not limited to QWen and can also be applied to other LLMs. \n\nQ2: Although the generated codes are interpretable, the paper does not provide detailed insights into understanding the logic and structure of these codes. There is also no discussion on how to further optimize these codes to improve performance. \n\nR2: In this paper, the framework goal is to identify crystal representations that satisfy all constraints, and thus we treat the entire evolutionary process as a black-box procedure. Our framework enables better utilization of the interdisciplinary knowledge of LLMs while guiding them to optimize and generate code effectively. Since the entire process is automated by the LLM, not interpreting the intermediate codes does not affect the final outcome.\n\nQ3: The experiments used specific network architectures of PerCNet. Can the impact of network choices on the results be further discussed?\n\nR3: We use PerCNet because it is currently the only network architecture in this field capable of handling dihedral angles. We employ this network to ensure a fair comparison between the representations obtained by our framework and those of PerCNet. In the future, we plan to use LLMs to simultaneously generate both the representations and the corresponding network architectures, rather than solely relying on network architectures from other algorithms.\n\nQ4: The test set is constructed manually, will it be better to employ the LLMs to generate the test set?\n\nR4: Using LLMs to generate the test set is possible, but in terms of effectiveness, a manually designed test set would be better. Due to the hallucination issue of LLMs, they might generate answers that appear correct but are actually wrong. Since the test set determines the direction of evolution, a manually designed test set would be more reliable and effective.\n\nQ5: Will the framework get stuck in local optima when facing more complex constraints? If so, how can mechanisms be designed to avoid this?\n\nR5: It is unlikely to fall into local optima because our program selection method (i.e., Equation 1 in the paper) ensures that the two parent programs have a higher probability of satisfying different constraints, thereby maintaining the diversity of the parent programs.", "claims_and_evidence": "Yes. There are two important claims in this paper. First, the authors claimed they found a new graph representation for crystalline materials. Second, the authors claimed the proposed code-generation framework can satisfy new constraints. \nThe two claims are supported well by experimental results. For the first claim, the new representation is shown in Fig.3 compared with two methods. For the second claim, the results in Table 2 show the evolution results among different LLMs.", "methods_and_evaluation_criteria": "Yes. The proposed framework is tested on two widely used material datasets, named JARVIS and Material Project separately. And the LLM agents are tested on GPT3.5, DeepSeek, and QianWen.", "theoretical_claims": "Yes, I have checked the proofs in E. The proofs show why the generated representation can satisfy the six constraints. Most methods based on graph representations(e.g. CGCNN, ALiGNN, ComFormer) do not satisfy the reflection equivariance and Lipschitz Continuity. How to define \"too large\" in Lipschitz Continuity?", "experimental_designs_or_analyses": "Yes. The experiments are designed in accordance with the work on predicting crystal properties.", "relation_to_broader_scientific_literature": "1. The idea of this work holds significant importance for both material and computer science. The test sets bridges the two fields. Normally, researchers need to investigate how to satisfy the constraints by sophisticated design. By contrast, this work converts the constraints to test sets and discards the representations that failed in test sets. In other words, this framework can not only solve the problem of material science but can also deal with problems in protein with desired test sets accordingly.\n\n2. A novel graph-based representation that satisfies six constraints is found by the proposed framework, thereby achieving enhanced accuracy in crystal property prediction.", "essential_references_not_discussed": "No important references are found to be missing.", "other_strengths_and_weaknesses": "The main strengths of this paper are as follows:\n- Most existing studies based on LLMs focus on directly using these models to predict crystal structures, i.e., generating new structures. In contrast, this work explores how to generate representations of crystal structures using LLMs, which is both novel and fundamental.\n- The framework is effective in generating new codes for the representation. And the results are interesting and impressive. In particular, the results shown in Figure 3 are especially noteworthy, as they may inspire human experts with their unique representation.\n\nThe weaknesses of this paper are as follows:\n- Section 5.3 is not very convincing. The main focus of the proposed framework is to generate codes (representations) satisfying six constraints, but the experiments are stopped at five due to resource limitations.\n- Although the generated codes are interpretable, the paper does not provide detailed insights into understanding the logic and structure of these codes. There is also no discussion on how to further optimize these codes to improve performance.", "other_comments_or_suggestions": "typos: line 014 \"crystalline material data...\"", "questions_for_authors": "1. Why the results of GPT+Rep-CodeGen and Seek+Rep-CodeGen are 0 for five constraints in Table 2? Does this mean that the framework is only effective for QianWen? \n\n2. The experiments used specific network architectures of PerCNet. Can the impact of network choices on the results be further discussed? \n\n3. The test set is constructed manually, will it be better to employ the LLMs to generate the test set?\n\n4. Will the framework get stuck in local optima when facing more complex constraints? If so, how can mechanisms be designed to avoid this?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "glxGx3Nq7y", "reviewer": "Reviewer_DgpU", "summary": "This paper introduces a novel framework named Rep-CodeGen, which leverages multiple Large Language Model (LLM) agents to automatically generate code for obtaining graph representations of material properties. The primary contributions of this work are threefold. First, the paper proposes an interpretable framework capable of automatically generating code to derive graph representations, particularly effective when addressing new constraints. This represents the first framework of its kind to automate the generation of code for graph representations. Second, through the generated code, the paper achieves a graph representation that satisfies six distinct constraints, including permutation invariance, rotation invariance, reflection equivariance, Lipschitz continuity, periodicity equivariance, and translation invariance. Lastly, extensive experiments on two real-world material datasets demonstrate that the material property prediction method based on this graph representation achieves state-of-the-art performance across multiple tasks.\n\n## update after rebuttal\nAfter reviewing the authors’ rebuttal, my questions regarding experimental details, the evolution process, and constraint handling have been satisfactorily addressed. I have no further concerns and maintain my recommendation of Accept.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your time and for recognizing the originality of our work, the contributions to the problem, and the experimental design. Below, we will address your suggestions and questions point by point.\n\nQ1: It is recommended to provide the initial code to help readers better understand the extent of changes during the evolution process.\n\nR1: Due to space limitations, we are unable to present the complete code here.The initial code will be provided in the appendix. In terms of design philosophy, the initial code determines neighbors of atoms within the unit cell based on a distance cutoff, and the features are limited to the distances between neighbors. However, the representation code that satisfies all conditions after the framework evolution differs in both the neighbor determination method and the features.      \n\nQ2: It is recommended to provide the complete prompts, particularly the descriptions of the constraints, to enhance reproducibility and clarity.\n\nR2: We will include the complete prompts in the appendix. Below is our full description of the constraints:\n\nPermutation_invariance: Changing the atomic indices should not alter the graph representation.\n\nRotation_invariance: Rotating the atomic coordinates should not change the graph representation.\n\nReflection_equivariance: Performing a mirror symmetry on the lattice and atomic coordinates should result in a change in the graph representation.\n\nLipschitz_continuity: If the coordinates of the crystal undergo continuous changes, the corresponding graph representation should also change continuously. This means the neighbor relationships of each atom (i.e., which atom is a neighbor) should not change, but the attributes of the corresponding neighbors (such as distance or angle) should vary accordingly.\n\nPeriodicity_equivariance: The graph representation should implicitly incorporate lattice information, meaning that any modification to the lattice coordinates (e.g., scaling the lattice by 1.5 times) should result in a change in the graph representation.\n\nTranslation_invariance: Translating the atomic coordinates should not affect the graph representation.\n\nQ3: Could there be situations where constraints conflict with each other, meaning that satisfying some constraints might make it impossible to satisfy others?\n\nR3: Theoretically, this situation is highly unlikely to occur because these physical constraints are grounded in real-world conditions, making conflicts almost improbable. Moreover, even if conflicts were to arise, the framework would evolve in a direction that satisfies the majority of the constraints.\n\nQ4: In the conclusion, the authors mention that the proposed representation method could also be applied to crystal generation. Could the authors provide more details on how this representation would be utilized in crystal generation tasks? \n\nR4: First, taking state-of-the-art algorithms in crystal generation tasks, such as CDVAE[1], DiffCSP[2] and DiffCSP++[3], as examples, the graph representation we propose can serve as input to denoising models and be directly applied to crystal generation tasks.  \n\nSecond, the “constraints” that Rep-CodeGen can solve, are not limited to physical constraints but can also include arbitrary requirements from human experts. Therefore, our framework can generate new crystal representation codes that meet the needs of various scenarios, such as sequence-based crystal representations required by LLMs.  \n\n[1]T. Xie, X. Fu, O.-E. Ganea, R. Barzilay, T. Jaakkola, Crystal diffusion variational autoencoder for periodic material generation, arXiv preprint arXiv:2110.06197 (2021).\n\n[2]R. Jiao, W. Huang, P. Lin, J. Han, P. Chen, Y. Lu, Y. Liu, Crystal structure prediction by joint equivariant diffusion, Advances in Neural Information Processing Systems 36 (2024).\n\n[3]Jiao, Rui, et al. \"Space Group Constrained Crystal Generation.\" The Twelfth International Conference on Learning Representations.", "claims_and_evidence": "Yes. Specifically, the claims made in the paper can be divided into four parts. First, the paper asserts that Rep-CodeGen is the first framework for automatically generating codes to obtain representations that can be used when facing new constraints. This claim is supported by the fact that no prior work has adopted this approach for generating crystal representations, as demonstrated in Section 2.1. Second, the paper claims that the framework is indeed capable of generating representations that satisfy new constraints. This claim is substantiated by the experimental results presented in Sections 5.1 and 5.3. Third, the paper proposes a representation that satisfies six constraints using the Rep-CodeGen framework. This claim is validated in Section 5.1 and further supported by the code provided in Appendix Code 1 and the proof section. Finally, the paper claims that the obtained representations achieve state-of-the-art (SOTA) performance in crystal property prediction tasks, which is evidenced by the results of Experiment 2.", "methods_and_evaluation_criteria": "Yes. The authors' approach of using a multi-agent framework to generate crystal representation codes is well-founded and logical. Additionally, the experimental design is appropriate and aligns with the research objectives. The datasets and evaluation metrics employed for the crystal property prediction task are publicly available and widely recognized in the field, further validating the suitability of the proposed methods and criteria.", "theoretical_claims": "Yes. The authors provide proofs in the appendix demonstrating that the representations generated by the Rep-CodeGen framework satisfy the six proposed constraints.  No significant issues were identified in the reasoning or validity of the proofs.", "experimental_designs_or_analyses": "Yes, the experimental designs and analyses presented by the authors are sound and valid. Specifically:  \n\n1. Experiment 1: The authors introduce the generation results of the Rep-CodeGen framework, compare the generated representations with other representations, and provide theoretical proofs demonstrating that the generated representations satisfy the proposed constraints.  \n2. Experiment 2: The authors evaluate the performance of the generated representations in crystal property prediction tasks, demonstrating their effectiveness.  \n3. Experiment 3: The authors compare the generation results of three different large language models (LLMs) when used independently versus in combination with the proposed framework, highlighting the advantages of their approach.  \nOverall, the experiments are well-designed, and the analyses are thorough and logically consistent.", "relation_to_broader_scientific_literature": "The key contributions of this paper are closely tied to the broader scientific literature on crystal property prediction.   \nContributions to Literature:\n1. This paper introduces a novel solution by proposing a multi-agent framework for automatically generating crystal representations that inherently satisfy the desired constraints. This approach represents a departure from traditional manual design methods.  \n 2. The proposed framework generates representations that differ from existing ones and, for the first time, satisfies six specific constraints, addressing a gap in prior research.  \n3. The experimental results demonstrate that the generated representations outperform existing methods in crystal property prediction tasks, achieving higher accuracy.", "essential_references_not_discussed": "This paper have discussed all the essential references.", "other_strengths_and_weaknesses": "Strengths:  \n1. Originality: The approach to solving the problem is novel. The paper utilizes the multi-agent framework Rep-CodeGen to generate codes for crystal representations, which, compared to previous manual design methods, has the capability to satisfy unknown new constraints. Moreover, the paper is the first to propose a representation that satisfies six constraints using the Rep-CodeGen framework.  \n2. Experimental Design: The experimental setup is reasonable, and the results demonstrate that the proposed representation achieves state-of-the-art (SOTA) performance in crystal property prediction tasks.  \n\nWeaknesses:  \nSome experimental details should be more clearly specified. See suggestions and questions.", "other_comments_or_suggestions": "Suggestions:\n1. It is recommended to provide the initial code to help readers better understand the extent of changes during the evolution process.  \n2. It is recommended to provide the complete prompts, particularly the descriptions of the constraints, to enhance reproducibility and clarity.", "questions_for_authors": "Questions:  \n1. Could there be situations where constraints conflict with each other, meaning that satisfying some constraints might make it impossible to satisfy others? If such cases arise, how does the proposed framework handle them? A clear explanation of this scenario and the resolution strategy would significantly impact the evaluation of the framework's robustness.  \n\n2. In the conclusion, the authors mention that the proposed representation method could also be applied to crystal generation. Could the authors provide more details on how this representation would be utilized in crystal generation tasks?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rnJTHvzXIJ", "reviewer": "Reviewer_oQXj", "summary": "This paper presents Rep-CodeGen, a framework using multiple LLM agents to generate, evolve, and evaluate code for obtaining graph representation of crystal structures following physical constraints. The representation obtained thereby is tested for constraint satisfaction and performance in materials property prediction, in comparison to baseline methods.\n\n## update after rebuttal\nThe authors have answered my questions well. I've raised my score accordingly; however, my concern remains that proposing a new representation is not a task for which LLM could be very valuable. The contribution to materials science is limited.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your time and thoughtful evaluation, as well as your recognition of our methodological design, theoretical foundations, and experimental framework. We are particularly grateful for your recognition of our motivation, ”Creating new representations of materials structures that satisfy necessary physical constraints could help ML-based materials modeling and design”. Below, we will address your concerns point by point.\n\nQ1: Materials representations need to trade off conciseness and expressiveness. Are six types of constraints all necessary?\n\nR1: We agree that representations should balance simplicity and expressiveness. At the same time, the importance of the six types of constraints proposed in this paper has been substantiated in corresponding studies (ComFormer, PerCNet, and PDD e.t.). These constraints can help capture the structure information of crystals and enhance the accuracy of property prediction. Improving the prediction accuracy of ML-based methods can benefit material science in various aspects.\n\nMost importantly, our framework offers fully customizable constraints for crystal representation. Domain experts can freely modify these constraints, such as physical constraints, representational simplicity constraints, and other specific constraints, based on application needs, enabling the creation of tailored crystal characterization codes for various scenarios. We selected six representative constraints to demonstrate our framework's capability in generating crystal representations that simultaneously satisfy multiple constraints that existing algorithms fail to meet collectively. \n\nQ2: How to combine merits from different code pieces? Is the ‘base’ code provided by user?\n\nR2: In the initial iteration, we manually provide the 'base' code to Agent_A. This code is designed to meet fundamental properties, including permutation, rotation, and translation invariance. It is worth noting that the base code can alternatively be generated by LLMs.\n\nThe merits of the generated codes are integrated throughout our framework's process. We not only retain the codes (produced by Agent_B) that meet specific constraints but also document the reasoning (merits and demerits) by Agent_C. Then, the parent codes with merits can be effectively selected through Formula 1 and 2. Subsequently, the codes and their corresponding reasoning are sent back to Agent_A. Agent_A evaluates the codes and their rationale, devising an improved plan accordingly.\n\nQ3: Why choose to generate code instead of natural language description? (Using LLMs decreases trustworthiness.)\n\nR3: This question helps shed light on our motivation. The primary advantage of code lies in its superior clarity and interpretability. Its structured nature allows humans to easily understand the reasoning behind the LLM's outputs, thereby boosting trust in its results. In contrast, natural language descriptions of crystal structures lack the same level of transparency, making it harder to discern the rationale behind the LLM's conclusions and potentially undermining confidence. \n\nQ4: “Given these, I don't see an advantage of using LLM here.”\n\nR4: It is worth noting that our goal is to assist human experts in accelerating material research instead of completely replacing them. This acceleration manifests in two aspects.\n\nFirst, while human experts excel in their specialized fields, they often invest significant effort in acquiring cross-disciplinary skills, such as coding or understanding graph learning. Rep-CodeGen enables materials science professionals to concentrate on their core expertise by reducing the need to master complex computer science concepts, thereby saving both time and resources.\n\nSecond, the generated representations can be viewed as a novel source of knowledge that can inspire the researchers. Rep-CodeGen shows the capability to rapidly generate a diverse array of solutions. For instance, to ensure that representations satisfy periodic equivariance (i.e., changes in lattice coordinates should be reflected in the representation), current graph representation methods typically rely on obtaining sufficiently large cutoffs. By contrast, RepCode-Gen introduces innovative strategies beyond the traditional cutoff method, such as converting periodic data into length and angle metrics, incorporating angles between lattice and edge vectors, and proposing supercell configurations extending unit cells along periodic dimensions.", "claims_and_evidence": "The main claims, which are related to the 3 key questions in Sec. 5, are well-supported. However, my concerns are about the validity of 6 constraints and the usefulness of this Rep-CodeGen method (see Other S/W and Questions).", "methods_and_evaluation_criteria": "The methods for comparing materials property prediction performance and testing constraint satisfactions make sense.", "theoretical_claims": "The proofs of constraint satisfactions look correct. However, whether all constraints in Sec. 3.2 are necessary is questionable (see Questions).", "experimental_designs_or_analyses": "Experimental design and analyses are sound. The comparison of evolution ability (Sec. 5.3) needs a little clarification (see Questions).", "relation_to_broader_scientific_literature": "Creating new representations of materials structures that satisfy *necessary* physical constraints could help ML-based materials modeling and design.", "essential_references_not_discussed": "Related works to my knowledge are comprehensively discussed.", "other_strengths_and_weaknesses": "This work uses LLM to “create” a new graph representation through code generation. However, designing this new representation and is not beyond human’s capability given the goal of which constraints to satisfy. Once a function for generating such representation is written, it can be reused for all materials. There are not many constraints of this kind, so this “representation creation” process will not need to be repeated so many times that it needs an LLM to automate (not to mention using LLMs decreases trustworthiness). Given these, I don’t see an advantage of using LLM here.", "other_comments_or_suggestions": "None.", "questions_for_authors": "* Materials representations need to trade off conciseness and expressiveness. Sec. 3.2 lists 6 types of constraints, but are they all necessary for representing materials?\n  - Enforcing reflection equivariance (rather than invariance) helps distinguish chirality, which has little or no impact on the properties tested in this study but increases the complexity of representation.\n  - For crystals with lattice type and atomic radii known, a cutoff radius for formulating graphs could be reasonably chosen. Whereas enforcing Lipschitz continuity may result in different structures with a small distortion (e.g., Jahn Teller) not well distinguished.\n* Lots of different code pieces are generated during the evolution process, some may contain certain merits that others miss. Is there a systematic way to combine them?\n* In Sec. 5.3, is the “base” code provided by user or generated by each LLM from prompt?\n* (Just for curiosity) Why choose to generate code instead of natural language description?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: All reviewers are positive about this paper. I agree with the reviewers that this paper has proposed a novel code generation framework with multiple large language model agents. This research direction is important and still at an early stage. Overall, this paper presents a solid work and is potentially useful to the community. I am confident that this paper can be accepted.
Decision: Accept (poster)