Local-Global Associative Frames for Symmetry-Preserving Crystal Structure Modeling
Authors: Haowei Hua
Wanyu Lin
Venue: NeurIPS 2025 poster ()
Keywords: local frame
crystal symmetry
SO(3) invariance
Primary area: machine_learning_for_sciences

Abstract: Crystal structures are defined by the periodic arrangement of atoms in 3D space, inherently making them equivariant to SO(3) group. A fundamental requirement for crystal property prediction is that the model's output should remain invariant to arbitrary rotational transformations of the input structure. 
One promising strategy to achieve this invariance is to align the given crystal structure into a canonical orientation with appropriately computed rotations, or called frames.
However, existing work either only considers a global frame or solely relies on more advanced local frames based on atoms' local structure. A global frame is too coarse to capture the local structure heterogeneity of the crystal, while local frames may inadvertently disrupt crystal symmetry, limiting their expressivity.
In this work, we revisit the frame design problem for crystalline materials and propose a novel approach to construct expressive {\bf S}ymmetry Preserving  Frames, dubbed as SPFrame, for modeling crystal structures. Specifically, 
this local-global associative frame constructs invariant local frames rather than equivariant ones, thereby preserving the symmetry of the crystal. In parallel, it integrates global structural information to construct an equivariant global frame to enforce SO(3) invariance.
Extensive experimental results demonstrate that SPFrame consistently outperforms traditional frame construction techniques and existing crystal property prediction baselines across multiple benchmark tasks.

Local-Global Associative Frames for Symmetry--reserrvng Crystal Structure Modeling
Haowei Hua', Wanyu Lin
DDepartment of Computing. *Department of Data Science and Artificial Intelligence The Hong Kong Polytechnic University, Hong Kong SAR, China haowei hua@connect, polyu. hk, can-yu.lin@pon edu. hk
Abstract
Crystal structures are defined by the periodic arrangement of atoms in 3D space, inherently making them equivariant SO(3) group. A fundamental requirement for crystal property prediction is that the model's output should remain invariant to arbitrary rotational transformations of the input structure. One promising strategy to achieve this invariance is to align the given crystal structure into canonical orientation with appropriately computed rotations, or called frames. However, existing work either only considers global frame or solely relies on more ad- vanced local frames based on atoms' local structure. global frame is too coarse to capture the local structure heterogeneity of the crystal, while local frames may inadvertently disrupt crystal symmetry. limiting their expressivity. this work, we revisit the frame design problem for crystalline materials and propose novel ap- proach to construct expressive
Sernetrr-Preserving Frames, dubbed as SPFrame, for modeling crystal structures. Specifically, this local-global associative frame constructs invariant local frames rather than equivariant ones, thereby preserving the symmetry of the crystal. In parallel, it integrates global structural information to construct an equivariant global frame to enforce SO(3) invariance. Extensive experimental results demonstrate that SPFrame consistently outperforms traditional frame construction techniques and existing crystal property prediction baselines across multiple benchmark tasks.
Introduction
Fast and accurate prediction of crystal properties is essential for accelerating the discovery of novel materials, it enables efficient screening of promising candidates from vast material space (38]. Traditional approaches based on high-fidelity quentum-mechanics calculations, such density functional theory (DFT), can provide acceptable error margin for property predictions but they require high computational resources (58), thereby limiting their practical deployment. As alternative, machine learning models have shown great potential for predicting crystal material properties with both efficiency and accuracy. These methods typically leverage 3D geometric graph representations of crystals in conjunction with geometric graph neural networks (GGNNs) 2 36
3 55 or transformer-
based variants of GGNNs [37 49 56 data and their properties.
27
5T
19 to establish mappings between crystal structural
When establishing the structure-property mapping, crystal structures are defined by periodic arrangement of atoms 3D space, which inherently makes them equivariant under SO(3) group transformations (i.e., rotations). However, many crystal properties, such as formation energy, are scalar quantities that remain invariant under such transformations. Consequently, to ensure accurate property prediction, GGNNs must maintain invariance when input structures are subjected to SO(3)
"Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

transformations. For this purpose, early studies have employed SO(3)-invariant features, such as simple interatomic distances [55] [6]. which prevent the designed GGNNs from capturing rich interatomic directional information. More recent works has attempted to incorporate interatomic directional information [57] while preserving SO(3)-invariance by carefully designing the network architecture, such as converting directional vectors into angle-based representations [57]. Although this approach successfully ensures invariance, it imposes architectural constraints that can limit the flexibility and scalability of neural networks in modeling complex crystal structures.
Alternatively, the global frame approach can be integrated with any GGNNs without imposing constraints on the network architecture, while still ensuring compliance with the SO(3) invariance requirement. These global frames are constructed in an equivariant manner (such as PCA frame 99) with respect to the input structure, effectively aligning the structure to canonical orientation However, because the same frame is applied to all atoms in the structure, the global frame approach lacks the ability to capture the local structure heterogeneity, limiting their expressivity. To address this limitation, recent developments have shifted toward local frame strategies, where distinct frames are dynamically constructed for each atom based on its local structure [19 34 50)
19 15 50
This approach allows for greater differentiation among atomic local environment, thereby improving the expressivity and enhancing the model performance [34 [5)]. Despite these advantages, directly applying this general equivariant local frame to crystal structures may unintentionally disrupt the symmetry of the crystal, discussed in Section 3.1 This disruption hinders the model's ss ability to distinguish atoms located at Wyckoff positions, thereby weakening its capacity to capture structural details.
To address these challenges, we revisit the problem of frame design for crystals and analyze the root cause of the symmetry breaking observed in previous equivariant local frame methods. Specif- ically, we identify that constructing local frames based solely on the local atomic structure often breaks the symmetry of the crystal. Motivated by this insight and the symmetry characteristics of crystal structures, we propose a Cenumetry-Preserving Frame (SPFrame) method for property prediction. SPFrame constructs invariant local frames rather than equivariant ones. For atoms located positions, SPFrame assigns identical invariant local frames, which allows
at wymmetry-equivallee their relative local relationships to be preserved after frame transformation. Based these invariant local frames, an equivariant global frame is further incorporated. Since the same global frame is applied to all atoms, it enforces SO(3) invariance across the structure without disrupting the symmetry of the crystal structure. This local-global associative design enables SPFrame to overcome the symmetry-breakee -breaking issue observed in previous local frame methods, enhancing the model's ability to differentiate between distinct atomic local structures. The effectiveness of the SPFrame approach is evaluated on two widely used benchmark datasets for crystalline materials.
2 Preliminaries
2.1 Coordinate Systems for Crystal Structure Representation
Crystalline materials are defined by a periodic 3D arrangement of atoms. where the smallest repeating unit, known as the unit cell, fully determines the entire crystal structure. Prior studies [56] 52 21 have established two primary coordinate systems for representing such crystal structure mathematically.
Cartesian Coordinate System. A crystal structure is formally defined by the triplet M = (A, L).
The matrix
[a1, a22 a,,]" E Raxd, contains feature vectors for n atoms within unit cell, where each row a; E dRd describes the individual atom feature. The 3D Cartesian coordinates of n atoms in the unit cell are encoded in X = [x), T2, : .T,]" E R The lattice maxtrix = [41, 12, 43] E 3x33 consists of the lattice vectors 4, 22, and 43. which form the basis of the 3D) space. The complete crystal structure emerges through periodic repetition: (Á, = {(á,, $:){ = x; + 4,4, + hal2 kz4g, d, a, kn k2, ka Z, i Z, < E n), where integer coefficients k1, k2, ky generate all possible atomic positions in the periodic lattice.
Fractional Coordinate System. This system employs lattice vectors 4, 12, and l as basis, with atomic positions expressed as fi = [f fi2 fral E [0, 1)3. The conversion to Cartesian co- ordinates is defined as x, = Σ, figlij where j = 2,2,3. This yields crystal representation T M (A, F, L), where F = [f1, TT E [0, 1)ax3 contains the fractional coordinates of all atoms in the unit cell.

---

@
p4mm
(b) Wyckeff Positions
Mulliplicity
Wyckolt letter
Practional coordinatess
(0,0)
(1/2,122)
2
(1/20), (0.1/2)
(x,0), (-x.0), (0x), (0xx)
4
(x/12),, (x,1/2) (1/2x0, (1/2-x)
4
(XX), (8,00, (600), (0000)
(cy)) (t-y). (-xg), (x.5)) (yxe), (-ys) (905)
12.01
(a) The figure illustrates the lattice of the P4mm space group, visually demonstrating equivalent positions; the symmetry-equivaleen positions are indicated by the same color. (b) The table depicts the Wyckoff positions present in this lattice.
Figure 1: An illustration with 2D plain group P4mm 52
14 21
2.2 The Symmetry of Crystal Structures
SO(3) group. The SO(3) group consists of all rotations in 3D space. Its elements are rotation matrices defined as {Q Q E R 23x3 Q = (Q) = 1}. When applied to crystal data M, an SO(3) transformation yields M' (A, QX.QQ.QQ.
Space group. The E(3) group encompasses all rigid transformations including rotations, reflections, and translations. Its elements can be denoted by the pair ((Q.t) E R3x3 Q I, R*) where Q is an orthogonal matrix and is translation vector. When an E(3) transformation is applied the crystal data M, certain elements (Qg: tg) can map a crystal structure back onto itself due to the inherent symmetry of the structure. These specific elements (Qg, ty) are collectively referred to space groups. Mathematically, (A, Q₂ + tg. Q2 (A, L), where the symbol '=' indicates the equivalence between geometric structures.
Wyckoff positions. The concept of space groups leads the definition of Wyckoff positions, which are sets of symmetry-equivalee characterized by three fundamental attributes: multiplicity, Wyckoff letter, and fractional coordinates. As shown in Fig. 1 for 2D plain group P4mm, the Wyckoff positions obey specific coordinate constraints, including 0 x 0.5, 0 y 0.5, x y, and the identical atomic occupation
atomic sites within a unit cell (20 21). Each Wyckoff position is
requirement [20 21
2.3 Crystal Structure Invariant Learning
S0(3)-invariance requirement for crystal properties prediction. The SO(3) group transformation can alter the orientation of crystal structure within 3D space [57]. Nevertheless, many critical material properties, such as formation energy, are invariant under the SO(3) group transformation. Consequently, for effective crystal property prediction, it is essential that the model can exhibit prediction capabilities. Specifically, for a prediction model denoted as fe if is SO(3)-invariant, for any rotation matrix Q E 3 the following equality holds:
(A, QX. QL) fo(A,
(I)
Frame. Frame-based methodologies have shown promising in enforcing equivariance and invariance In the context of SO(3) group transformations, a frame can be interpreted as a rotation matrix F E R3x3 deriving from -quiiirqaa map denoted hX)). This frame transforms the atomic positions X into SO(3)-invariant representation represented XFT, Crucially, this representation remains unchanged under arbitrary rotations Q:
in geometric deep learning [50 33
37
X 2, XQ(FQ)T = XQQTFT = XFT, thus decoupling the SO(3) invariance requirement for neural network design. Additionally, the concept of a global frame involves using single frame for all atoms within the atomic system, whereas a local frame is defined for each atom individually, with its calculation based on the atom's 's local structure, more detailed discussion related works can be found in Section! !

---

Graph Construction
Applying Local Frame
Crystal Structure
Local Structure
Transformed Local Structure
Data danio
F
Different local structure
Identical
f
loral structure
P
LT
F
F
53
f
f
PT
Not distnguishable
Local Information
Trausformed Local Structure
Global Information
F FIVV
F
Different
F
f
local structure
global
F
Q
A Aplying SPFrame
f,
f
F, F global
C
P.P
Distinguishable
Figure 2: Using the 2D plane group P4mm running example, we demonstrate why local frame methods may disrupt the symmetry of crystal. For atoms and that belong to the same Wyckoff position type, their local structures can be related by 90-degree rotation after graph construction. Since equivariant local frames are constructed solely based on local structural information, the resulting frames for p and q also exhibit a 90-degree rotational relationship. applying these frames eliminates the relative orientation between the two local structures. In contrast, SPFrame preserves these relative structural differences by incorporating global structural information during frame construction.
Methodology
In this section, we first outline the motivation behind our work, with a particular emphasis on the limitations of existing local frame methods when applied crystal structures, as discussed in Section 3.1 To address these challenges, we introduce the proposed SPFrame method, local-global associative frame. We further incorporate this method into the established crystal property prediction architecture, yielding new framework for crystal structure modeling, as described in Section 3.2
3.1 Symmetry Breaking Induced by the Local Frame
begin by presenting the general formulation of message passing at k-th layer in GGNNs using -quiiaaqui edge features, defined as follows:
Building upon previous works 22 57 51,,
f!() ()())
Σ JEN(S)
(N) (f!h-1)
f((t-1) Csj. ē.5)
where ((()) denotes the feature vector of atom eij Rd represents SO(3)-invariant edge features (e.g., embeddings of interatomic distances between atoms and j). and ēij E R corresponds to -Quii-equivariant edge features capturing directional information between atoms (e.g., the edge vector between atoms and j). The functions
(k)) (-) and ()(()()) are learnable non-linear mappings that define the message construction and aggregation processes, respectively.
When integrating the local frame into the message passing framework, the local frame is adaptively defined for each atom, thereby transforming the equivariant edge features into invariant features. Consequently, the message passing process is reformulated follows:
f((0 (b(()))))
[kk
Σ ((()) JEN())
(k-1)) esy. G.
where F; denotes the local frame for atom i. As illustrated in Section 2.3 the presence of local frames ensures that the output of the GGNNs remains invariant under the SO(3) group transformation.

---

However, as discussed in Section 2.2, the symmetry of crystal structures implies that atoms occupying the same Wyckoff position type exhibit similar local structures. As illustrated in Figure 2 when constructing the erystal graph, atoms p and which belong to the same Wyckoff position type, share identical atom features and invariant edge features (such as interatomic distances). The only distinction between these atoms lies in equivariant edge features. These equivariant features, representing the relative directional vectors of atoms p and with respect to their neighboring atoms, are related through 90 degrees rotation matrix Qs0
When local frames are incorporated into GGNNs, they serve to canonicalize the equivariant edge features Esy by mapping them to invariant representations e,, Since the local frames F, p and Fg are constructed equivariantly based on the local structures, follows that F, Q90 F Fp. As result, atoms and q. which initially exhibit distinct orientations in their respective local structures, are aligned to common orientation, rendering their local structures indistinguishable. This process diminishes the model's s ability to distinguish syymmery-equivaaln limiting the expressivity of GGNNs. Furthermore, such equivalence between atoms p and g under SO(3) transformations is commonly observed in crystals with screw axes rotational symmetries, such as those found in space groups like P2 among others []4].
yet spatially distinct atoms, thereby
3.2 Our SPFrame
To address the challenges outlined above, several critical considerations must be taken into account. First, it is essential to decouple the SO(3) invariance requirement imposed on GGNNs when employ- ing local frames. Simultancously, for atoms located at equivalent Wyckoff positions, it is imperative to preserve the relative relationships within their local structures following the application of the local frame. This preservation ensures that the GGNN can effectively differentiate between these atoms. Second, in line with the standard definition of local frames, distinct frames should be assigned to atoms occupying non-equivalent positions.
SO(3) symmetry decoupling and erystal symmetry preserving. As discussed Section 3.1 conventional local frame construction assigns different frames to symmetry-equivvaent atoms p and 4. which can inadvertently disrupt the crystal's symmetry. To mitigate this issue in Figure 2 a straightforward yet effective strategy is to assign identical frames to atoms p and g. This design ensures that the relative orientation relationships between their local structures are preserved after the frame transformation. For atoms that are not symmetry-equivallee distinct frames should be constructed to reflect the differences in their local structures. Therefore, We now introduce the SPFrame, defined
F = F FINV F global
where F FINV, denotes the invariant local frame for atom and Feichal denotes the equivariant global frame shared across the atomic system. Using SPFrame, we reformulate Equation 3as as follows:
J/00 (())
Hth (h-1
@(0) (I! eey,, JEN(
Σ
eiy, e.y global F Tiv.))
(5)
Under an SO(3) group transformation applied to the entire atomic system, the presence of global frames ensures that the output of the GGNNs remains invariant, thereby decoupling the SO(3) invariance requirement. Since the global frame is computed based global structural information, remains consistent across conmetry-equivaleee atoms and thus does not disrupt the symmetry of the crystal. At the same time, the invariant local frame FINV,; is constructed using invariant method based on local structural information. Thus, atoms p and g, which occupy the same type of Wyckoff position, are assigned identical local frames. Consequently, the transformed SO(3)-invariant representation, F FT global F FiNv. remains distinguishable for atoms p and while preserving their relative structural differences.
This local-g local-global associative design enables the model to satisfy SO(3) invariance while maintaining the crystal's symmetry. Furthermore, since the invariant local frame FINV,; still considers local information, the SPFrame for non-symmettr--equivalee addition, we provide a theoretical justification for the superiority of SPFrame over the local frame, as detailed in Appendix A.I This work also guarantees SE(3) invariance, detailed in Appendix A.2
positions are computed differently. In
5

---

Algorithm Quaternion to Rotation Matrix Conversion
Require: Quaternion q [a, b, c, d E I+
Ensure: Rotation matrix Q E 1R333
I: Normalize the quaternion:
V + + at a/o b+ b/b/8,
c/0, d d d/s
2: Compute the rotation matrix Q:
a² + - c² d²
2(bc ad)
2(bd + ac)
Q = 2 - ad)
a 1/ + c²
d²
2 ab) a - b² c² +
2(bd ac)
2(ed ab)
frame construction. As illustrated in Equation [ the proposed SPFrame consists of two components: a shared global frame F glooaal applied to all atoms, and set of atom- specific invariant local frames FINv,-- The global frame F golaaal plays key role in decoupling the SO(3) invariance requirement from the GGNN. For simplicity, we propose to use non-parametric approaches such as QR decomposition
33
Additional implementation details can found in
Appendix A.33 A.3
Correspondingly, the invariant local frame FINv,; is designed enhance the expressive power of the GGNN. Since it does not need to independently enforce SO(3) symmetry constraints, this component allows for better flexibility frame construction. To this end, we employ quaternions 42 46 13] as
a compact and numerically stable representation of the rotations.
Specifically, first predict quaternion for each atom based its local structure. Inspired by previous work [50]] 34 we leverage a message passing scheme to generate the quaternion eenteerionembbddinnnn....
9: = . fc. E
の (fi, (f.. f, cij) ff. JEN(()
FINV,- = LF(qs),
where q: E R24 denotes the predicted quaternion for atom i. The message function (6-)) and the aggregation funetion vi(-) can instantiated using any SO(3)-invariant message passing architecture. In this work, we adopt transformer-bast quaternion qi is obtained, it converted into the corresponding rotation matrix
implementations following Yan et al. [56). Once the
42 via the mapping
LF(-) (Further details can be found in Appendix A.4). The pseudocode for LF(-) is presented in Algorithm
Network architecture. As demonstrated in previous studies [50 34 local frames can be effectively integrated into GGNNs that utilize equivariant edge features for message passing. Among these models, eComFormer [57] represents the state of the art in crystal property prediction, leveraging equivariant edge features enhance message propagation. Based on this, we adopt eComFormer as the backbone architecture for implementing and evaluating the proposed SPFrame strategy. More details on the integration of SPFrame into eComFormer can be found Appendix A.5
4 Related Works
Crystal Property Prediction. GGNNs and their transformer-based extensions (hereafter collectively referred as GGNNs for convenience) have been widely adopted in crystal property prediction due to their capacity to model complex atomic interactions. Several representative methods, such as
CGCNN (55), MEGNet 2 GATGNN (36), Matformer (56). PotNet [32], DOSTransformer [27].
interatomic distances as edge features. By avoiding the use of equivariant directional vectors, these models ensure SO(3)- invariant predictions. Similarly, models such as ALIGNN [3 M3GNet []]. Crystalformer [51].
and CrystalFormer 49 construct crystal graphs by utilizing
and iComFormer [57 leverage invariant angular information as edge features to maintain SO(3)-
invariance in prediction. Beyond these, several methods adopt more specialized strategies. For example, eComFormer (57) utilizes equivariant edge features, which subsequently are transformed into two-hop invariant angular representations for preserving invariance.

---

Global Frame. Frames are widely used in both equivariant and invariant learning. However, earlier frame methods, such as the frame averaging (FA) method 4 9 rely on frame construction techniques like PCA, which produce non-unique frames. This necessitates the use of specially designed loss functions during training to learn invariant representations across all tranf-tranfoomme variants. More recently, minimal frame methods [33] have adopted frame construction techniques such as QR decomposition to produce unique frames, thereby improving the efficiency of frames. This is the type of global frame described in this work. It is worth noting that another approach in equivariant and invariant learning, i.e. canonicalization [37
43 is equivalent to the minimal
frame method some extent 377
Local Frame. Similar the global frame, the local frame is also method that transforms equivariant data representations into invariant ones [7] [50] 8 40 34 [9]. The difference lies in that the local frame approach generates separate frame for each atom the atomic system, which can enhance the expressiveness of GGNNs [50]. Recent work, Crystalframer [19], was the first to introduce local frames into the field of crystal property prediction. Building upon the attention mechanism from 49]. it designed two types of equivariant local frames and recalculated different local frames at various network layers. However, as mentioned above, the use of general equivariant local frames may unintentionally decouple the symmetry of the crystal structure.
Beyond the aforementioned studies, we also review approaches that incorporate crystal symmetry into method design, together with other key strategies for enhancing predictive accuracy. A more discussion is provided in Appendix A.6
5 Experiments
To validate the effectiveness of the proposed SPFrame, we performed a comprehensive series of experiments on crystal property prediction. Additionally, we conducted comparative analyses between our method and existing equivariant local frame approaches. A detailed summary of the experimental setup and results is provided below.
5.1 Experimental setup
Datasets. We utilize two widely used crystal property benchmark datasets: JARVIS-DFT and Materials Project (MP). Following previous work [56 57 19]. we perform predction tasks of formation energy, total energy, bandgap, and energy above hull (E hull) on HARVIS-DFT dataset. For the MP dataset, we perform predction tasks of formation energy, bandgap, bulk modulus, and shear modulus.
Baseline Methods. We selected several state-of-the-an methods the field, including CGCNN [55], SchNet (55. MEGNet [2]. GATGNN (36), ALIGNN 33, Matformer (56). PotNet (32). Crystal-
former 449. eComFormer [57]. iComFormer 57 and Crystalframer [9] as baseline methods for comparison.
Frame Comparison and Ablation Studies. In addition the aforementioned crystal property prediction methods, we also conducted comparative experiments by replacing the proposed SPFrame with other frame methods integrated into the backbone network. The first method is an SO(3)- equivariant local frame. Inspired by Wang and Zhang (50) and Lippmann et al. [34), we design this approach using Gram-Schmidt orthogonalization to construct SO(3)-equivariant local frames. This
method serves as baseline for examining the impact of breaking the symmetry of crystal structures on model performance. The second method is an SPFrame variant constructed using orthogonalization, allowing for more direct comparison with the first method, as both rely on the same orthogonalization procedure. This comparison further enables the evaluation the advantages of the quaternion- based SPFrame. Additional details the design of these two frame baselines can be found in Appendix A.7
Experimental Settings. Following prior work [57]. we evaluate model performance using Mean Ab- solute Error (MAE) and optimize all models using the Adam optimizer. We conduct our experiments on NVIDIA GeForce RTX 3090 GPUs, with complete hyperparameter configurations (including learning rates, batch sizes, and training epochs) provided in Appendix AA.8) In our evaluation, we highlight the best perfoornn results in bold and indicate second-best performances with underlining.
7

---

5.2 Experimental Results
JARVIS. Table I presents the experimental results on JARVIS. The eComFormer architecture, when combined with the our proposed SPFrame, achieves best performance on all prediction tasks, demonstrating consistent improvements over existing approaches. In the comparison of different frame methods, the performance of the SO(3)-equivariant of both the (rram-Schmidt-oee
Seaam- local frame is inferior to that SPFrame and the quaternion-basee SPFrame in all prediction tasks. This observation confirms that maintaining crystal symmetry while applying local frames enables GGNNs to better distinguish between atoms, leading to improved prediction accuracy. Furthermore, the superior performance of the quaternion-base SPFrame over the Bram-Schmidtt-ee emphasizes that quaternion-derived rotation matrices provide a more effective representation for frame construction in crystal materials. To further demonstrate the generality of SPFrame, we conducted additional experiments combining SPFrame with another backbone architectures. The corresponding results are provided in Appendix A.99
SPFrame
Table 1: Property prediction results on the JARVIS dataset,
Form energy
Total energy
Bandgup (OPT)
Bandgap (MBJ)
E hull
Method
eVattom
eVatom
eV
eV
CGCNN
0.078
0.17
SchNet
0,045
0.047
0.19
0.43
0.14
MEGNet
0.047
0.058
0.145
0.34
0.084
GATGNN
0.047
0,056
0.12
ALIGNN
0.0331
0.117
0.142
0.31
0,076
Matformer
0,0325
0.035
0.137
0,30
0,064
PotNet
0.0294
0.032
0.127
0.27
0.055
iComFormer
0.0272
0.0288
0.122
0.26
0.047
Crystalforner
0,0306
0.0320
0.128
0.30
0.046
Crystalframer
0.0263
0.0279
0.117
0,242
0.047
ComFormer
0.0284
0.0315
0.124
0,283
0,044
w/ -quivvrauaa Gram- Schamid local frame
0,0285
0.0296
0.115
0.271
0,043
w/ Seam---Siiidt-bbseedd SPFrame (ours)
0.0268
0.0281
0.109
0,259
0.043
w/ Quaternion- bassdd SPFrame (ours)
0.0261
0.0276
0.107
0.239
0.042
MP. Table 2 presents the experimental results on the MP dataset. Similar to the results obtained on the JARVIS dataset, the eComFormer architecture, when combined with proposed SPFrame, achieves the best performance on two out of four prediction tasks. The comparison with different frame methods further demonstrates the effectiveness of SPFrame. Furthermore, considering that the performance of the our method on bulk modulus and shear modulus prediction was not particularly strong on the MP dataset, we additionally conducted experiments on the JARVIS dataset for these two properties. The corresponding results are provided in Appendix A.9
Table 2: Property prediction results on the MP dataset.
Formation energy
Bandgap
Bulk modulus
Shear modulus
Method
eVatom
log(GPa)
log(GPa)
CGCNN
0.031
0.292
0.047
0.077
SchNet
0.033
0.345
0.066
0.099
MEGNet
0.030
0.307
0.060
0.099
GATGNN
0.033
0.280
0.045
0.075
ALIGNN
0.022
0.218
0.051
0.078
Matformer
0.021
0.211
0.043
0.073
PotNet
0.0188
0.204
0.040
0.065
iComFormer
0.0183
0.193
0.0380
0.0637
Crystalfonner
0.0186
0.0377
0.0689
Crystalframer
0.0172
0.185
0,0338
0.0677
:ComFormer
0.0182
0.202
0.0417
0.0729
-quiiiiriar Schmiih local frame
0.0183
0.187
0.0407
0.0721
w/ Gramm-Schmidt-base SPFrame (ours)
0.0174
0.191
0.0370
0.0678
Quaternion-basee SPFrame (ours)
0.0171
0.0371
0.0672
Efficiency comparison. Table 3 compares the model efficiency of several frame methods and the skeleton method, eComFormer. We show the average training time per epoch, total number of parameters, and average testing time per material, all evaluated
the JARVIS-DFT formation energy dataset. The batch size is kept consistent across all experiments, and all experiments are conducted using single NVIDIA GeForce RTX 3090 GPU. Due to the presence of trainable network

---

Crystal Structure (PrBPt)
Local Frame
Initial Local Structure
Layer1
Layer2
Layer3
SPFrame
toun:
Pr
Edge:
Initial Local Structure
Layer1
Layer2
Layer3
Figure 3: Visual analysis. After graph construction, atoms at symmerry-equivalen positions may exhibit distinet local structures. Local frame methods tend transform these local structures into identical representations, thereby removing the relative differences and making the atoms indistinguishable the model. In contrast, SPFrame preserves these structural distinctions, enabling he model to effectively differentiate between atoms located at symmetry-equivalle positions.
components in the frame calculations, all frame-based methods are less efficient than the skeleton method, eComFormer. The SO(3)-equivariant Gram-Schmiae local frame and Wrram-Schmidt-are SPFrame, which construct rotation matrices using the Gramm-Schmid orthogonalizatton method, require two distinct message passing and aggregation modules to generate and orthogonalize two different vectors (see Appendix A.7), Consequently, their efficiencies similar but lower than that of SPFrame. contrast, SPFrame only requires a single message passing and aggregation module to predict quaternions, which subsequently used to construct the rotation matrix, thereby reducing computational cost.
Table Efficiency analysis.
Method
Num. Params.
Time/epoch
Test time/Material
eComFormer
4,9 M
127.86
31.76 ms
-equiaaaua Gram Schmidt local frame
M
235.42
43.43
w/ Grum- Schmidt-bused based SPFrame
8.5 M
234.86
42.81 ms
w/ Quaternion-aee -aased SPFrame
6,3 M
143.75
37.07 ms
Visual analysis. To empirically evaluate the limitations the equivariant local frame method and the effectiveness of SPFrame, present concrete visual example, shown in Figure Specifically, visualize the crystal structure of PrBPt; (IVASP-16632) and illustrate how different frames influence atoms located at symmetry-equivalen positions in the context of the formation energy prediction task Pt atoms, the equivariant local frame
on the JARVIS-DFT dataset. For the two symmetry-equivalt transforms the edge features such that their local structures become indistinguishabl.. In contrast, SPFrame preserves the relative structural differences between these atoms after transformation. enabling the model to distinguish them. On this sample, the backbone network using the equivariant local frame yields a MAE of 0.0551, while the same backbone integrated with SPFrame achieves lower MAE of 0.0503, demonstrating superior predictive accuracy.
Conclusion
This paper investigates the limitations of applying conventional local frame methods to crystal structures. Although these local frame methods enable GGNNs to satisfy the SO(3) invariance requirement, they may inadvertently disrupt the symmetry of the crystal, limiting the model's is ability distinguish atoms situated at symmetry-equivallee positions. To address this challenge, we propose the SPFrame for crystal property prediction. SPFrame constructs frames by incorporating both local

---

atomic structural information and global structural information. Such local global associative frames ensure that GGNNs meet the SO(3) invariance requirement while preserving the crystal's 's symmetry. enhancing the model's is ability to differentiate between distinct atomic structures. Experimental results on multiple datasets demonstrate the effectiveness of SPFrame. We hope that SPFrame provides new perspective for machine learning and materials science, promoting the specific adaptation of machine learning techniques for materials science applications. Further discussion of SPFrame is provided in Appendix A. 10 and Appendix A.11
Acknowledgments
This work was partially supported by the Research Grants Counil (RGC) of the Hong Kong (HK) SAR (Grant No. 15208725 and 15208222), the Young Scientists Fund of National Natural Science Foundation of China (NSFC) (Grant No. 62206235), and the Hong Kong Polytechnic University (Grant No. A0046682 and PO057774).
References
[1] C. Chen and S. P. Ong. A universal graph deep learning interatomic potential for the periodic table. Nature Computational Science, 2(11):718-74) 728. 2022.
[2] C. Chen, W. Ye. Y. Zuo. C. Zheng. and S. Ong. Graph networks universal machine learning framework for molecules and crystals. Chemistry of Materials, 31(9):3564 3572, 2019.
[3] Choudhary and DeCost. Atomistic line graph neural network for improved materials property predictions. npj Computational Materials, 781):885, 2021.
[4] K. Das, B. Samanta, P. Goyal, S.-C. Lee, S. Bhattacharjee, and N. Ganguly. Crysxpp: An explainable property predictor for crystalline materials. npj Computational Materials, 8(1):43, 2022.
[5] K. Das, P. Goyal, S.-C. Lee, S. Bhattacharjee, and N. Ganguly. Crysmmnet: multimodal representation for crystal property prediction. Uncertainty Artificial Intelligence, pages 507-517. PMLR, 2023.
[6] K. Das, Samanta, Goyal, S.-C. Lee, S. Bhattacharjee, and N. Ganguly. Crysgnn: Distilling pre trained knowledge to enhance property prediction for crystalline materials. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages
2023.
[7] W. Du Н. Zhang, Y. Du. Q. Meng, W. Chen, N. Zheng, B. Shao, and T.-Y. Liu. Se (3) equivariant graph neural networks with complete local frames. In International Conference on Machine Learning, pages 5583-5608. PMLR, 2022.
[8] Y. Du, Wang, D. Feng. G. Wang. S. Ji, C. P. Gomes, Z.-M. Ma, et al. new perspective on building efficient and expressive 3d equivariant graph neural networks. Advances in neural information processing systems, 36:66647-6666 2023.
[9] A. A. Duval, V. Schmidt, A. Hernández-Garena. S. Miret, F. D. Malliaros, Y. Bengio, and D. Rolnick. Faenet: Frame averaging equivariant gan for materials modeling. In International Conference Machine Learning, pages 9013-9033. PMLR, 2023.
[10] N. Dym, H. Lawrence, and W. Siegel. Equivariant frames and impossibility of continuous canonical- ization. In Forty-first International Conference on Machine Learning, 2024.
[11] D. Eberly. Quaternion algebra and calculus. Magic Software Inc, 26:1-8, 2002.
[12] M. Geiger and T. Smidt. e3nn: Euclidean neural networks. arXiv preprint arXiv:2207.09455: 2022.
[13] A. R. Geist, J. Frey, M. Zhobro, A. Levina, and G. Martius. Learning with 3d rotations, a hitchhiker's guide (3). In International Conference on Machine Learning, pages 15331--13500 PMLR, 2024.
[14] T. Hahn, U. Shmueli, and W. Arthur. International tables for crystallography, volume Dordrecht, 1983.
Reidel
[15] J. Han, Cen, L. Wu, Z. Li, X. Kong, R. Jiao, Z. Yu, T. Xu. Wu, Z. Wang, et al. A survey of geometric graph neural networks: Data structures, models and applications. Frontiers of Computer Science, 19(1)): 191 1375. 2025.
10

---

[16] H. Hong, W. Lin, and K. Tan. Accelerating 3d molecule generation via jointly geometric optimal transport. In The Thirteenth International Conference on Learning Representations, 2025.
[17] T. Hsu, T. A. Pham, N. Keilhart, S. Weitzner, J. Chapman, P. Xiao, S. R. Qiu, X. Chen, and B. C. Wood. Efficient and interpretable graph network representation for angle-dependent properties applied to optical spectroscopy. npj Computational Materials, 8(1):111, 2022.
[18] H. Hua, W. Lin, and J. Yang. Fast crystal tensor property prediction: A general o ())-equivariant framework based on polar decomposition. arXiv preprint arXiv:24/0.02372, 2024.
[19] Y. Ito, T. Taniai, R. Igarashi, Y. Ushiku, and K. Ono. Rethinking the role of frames for se(3)-invariant crystal structure modeling. In The Thirteenth International Conference on Learning Representations, 2025.
[20] A. Janner, Janssen, and De Wolff. Wyckoff positions used for the classification of bravais classes of modulated crystals. Acta Crystallographica Section A: Foundations of Crystallogruphy, 3965)66676670, 1983.
[21] R. Jiao, W. Huang, Liu, D. Zhao, and Y. Liu. Space group constrained crystal generation. In The Twelfth International Conference on Learning Representations. 2024.
[22] O. Kaba and S. Ravanbakhsh. Equivariant networks for crystal structures. Advances Neural Information Processing Systems, 35:4150-4164, 2022.
[23] S.-O. Kaba, A. K. Mondal, Y. Zhang, Y. Bengio, and S. Ravanbakhsh. Equivariance with learned canonicalization functions. International Conference on Machine Learning, pages 15546-15566. PMLR, 2023.
[24] N. Kazeev, Nong. I. Romanov, R. Zhu, A. E. Ustyuzhanin. S. Yamazaki. and K. Hippalgaonkar. Wyckoff' transformer: Generation of symmetric crystals. In Forty-second International Conference on Machine Learning, 2025.
[25] F. E. Kelvinius, O. B. Andersson, A. S. Parackal, D. Qian, R. Armiento, and F. Lindsten. Wyckoffdiff-a generative diffusion model for crystal symmetry. In Forty-second International Conference on Muchine Learning, 2025.
[26] D. Kingma. Adam: method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations. 2015.
[27] N. Lee, H. Noh, S. Kim, D. Hyun, G. S. Na, and C. Park. Density of states prediction of crystalline materials via prompt-guided multi-modal transformer. Advances Neural Information Processing Systems, 36, 2024.
[28] D. Levy, S. S. Panigrahi, S.-O. Kaba, Q Zhu, K. L. K. Lee, M. Galkin, S. Miret, and S. Ravanbakhsh. Symmed: Symmetry-preserving crystal generation with diffusion models. In The Thirteenth International Conference on Learning Representations, 2025.
[29] Z. Li, X. Sun, W. Lin, and J. Cao. Unveiling molecular secrets: An llm-augeented linear model for explainable and calibratable molecular property prediction. arXiv preprint arxiv:2410.08829, 2024.
[30] W. Lin. H. Lan, and B. Li. Generative causal explanations for graph neural networks. In International conference machine learning, pages 6666 6679.666...76 PMLR, 2021.
[31] W. Lin, H. Lan, H. Wang, and B. Li. Orphicx: A causality-nnspired latent variable model for interpreting graph neural networks. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13729- 13738, 2022.
[32] Y. Lin, K. Yan, Y. Luo, Y. Liu, X. Qian, and S. Ji. Efficient approximations of complete interatomic potentials for crystal property prediction. In International Conference on Machine Learning, pages 21 21260-21287. PMLR, 2023.
[33] Y. Lin, J. Helwig, S. Gui, and S. Ji. Equivariance via minimal frame averaging for more symmetries and efficiency. In Forty-first International Conference Machine Learning, 2024.
[34] P. Lippmann, G. Gerhartz, R. Remme, and F. A. Hamprecht. Beyond canonicalization: How tensorial messages improve equivariant message passing. In The Thirteenth International Conference on Learning Representations, 2025.
[35] S. Liu, Y. Li. Z. Li, Z. Zheng, C. Duan, Z.-M. Ma. O. Yaghi, A. Anandkumar, C. Borgs, J. Chayes, et al. Symmetry-informed informed geometric representation for molecules, proteins, and crystalline materials. Advances in neural information processing systems, 36:66084 66101, 2023.
11

---

[36] S.-Y. Louis. Y. Zhao, A. Nasiri. X. Wang, Y. Song, F. Liu. and J. Hu. Graph convolutional neural networks with global attention for improved materials property prediction. Physical Chemistry Chemical Physics, 22 (32):18141-18145 2020.
[37] G. Ma, Y. Wang, D. Lim, S. Jegelka, and Y. Wang. A canonicalization perspective invariant and equivariant learning. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[38] Z. Mao, W. Li, and J. Tan. Dielectric tensor prediction for inorganic materials using latent information from preferred potential. upj Computational Materials, 10(1):265, 2024.
[39] S. Mukherjee, M. Ghosh, and P. Basuchowdhuri. Crysatom: Distributed representation atoms for crystal property prediction. In The Third Learning on Gruphs Conference, 2024.
[40] S. Pozdnyakov and M. Ceriotti. Smooth, exact rotational symmetrization for deep learning on point clouds. Advances in Neural Information Processing Systems, (0479969--75000 2023.
[41] O. Puny, M. Atzmon, E, J. Smith, I. Misra, A. Grover, H. Ben-Hamu, and Y. Lipman, Frame averaging for invariant and equivariant network design. In International Conference on Learning Representations, 2022.
[42] I... Rodman. Topics in quaternion linear algebra. Princeton University Press, 2014.
[43] R. Sajnani, A. Poulenard, Jain, R. Dua, L. J. Guibas, and S. Sridhar. Condor: Self-suuervised canonical- ization 3d pose for partial shapes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16969-16979, 2022.
[44] V. G. Satorras, Hoogeboom, and M. Welling, E (n) equivariant graph neural networks. In International conference on machine learning, pages 9323-9332. PMLR, 2021.
[45] K. Schütt, H. F. Sauceda, P.-J. Kindermans, A. Tkatchenko, and K.-R. Müller. Schnet-a deep learning architecture for molecules and materials. The Journal of Chemical Physics, 148(24), 2018.
[46] K. Shoemake, Animating rotation with quaternion curves. SIGGRAPH Comput. Graph., 19(3):245- 254, 1985. ISSN 0097-8930. doi: 10. 10.1144/2556553255224 URL https://doli org/ 10. 1144//25115
325242
[47] L. N. Smith and N. Topin. Super-convergence: Very fast training of neural networks using large learning rates. In Artificial Intelligence and Machine Learning for 1006, pages 369-386. SPIE, 2019.
Operations Applications, volume
[48] Z. Song, Z. Meng, and I. King. A diffusion-based pre-training framework for crystal property prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 8993-9001, 2024.
[49] T. Taniai, R. Igarashi, Y. Suzuki, N. Chiba, K. Saito, Y. Ushiku, and K. Ono. Crystalformer: Infinitely connected attention for periodic structure encoding. The Twelfth International Conference on Learning Representations, 2024.
[50] X. Wang and M. Zhang. Graph neural network with local frame for molecular potential energy surface. In Learning on Grapks Conference, pages 19-1. PMLR, 2022.
[51] Y. Wang, S. Kong, J. M. Gregoire, and C. P. Gomes. Conformal crystal graph transformer with robust encoding of periodic invariance. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 283-291, 2024.
[52] Z. Wang, H. Hua, W. Lin, M. Yang, and K. C. Tan. Crystalline material discovery in the era of artificial intelligence. arXiv preprint arXiv:2408.0880 2024.
[53] Z. Wang, Z. Lin, W. Lin, M. Yang, M. Zeng, and K. C. Tan. Explainable molecular property prediction: Aligning chemical concepts with predictions via language models. arXiv preprint arXiv: 16041, 2024.
[54] T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, et al. Transformers: State- satee-of-the ohethe-art natural language processing. In Proceedings of the 2020 conference empirical methods in natural language processing: system demonstrations, pages 38-45, 2020.
[55] T. Xic and J. C. Grossman. Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties. Physical review letters, 120(14): 1453000,, 2018.
[56] K. Yan, Y. Liu, Y. Lin, and S. Ji. Periodic graph transformers for crystal material property prediction. Advances in Neural Information Processing Systems, 35:15066- 150800 2022.
12

---

[57] K. Yan, Fu, X. Qian. X. Qian, and S. Ji. Complete and efficient graph transformers for erystal material property prediction. In The Twelfth International Conference Learning Represeniations, 2024.
[58] K. Yan, A. Saxton, X. Qian, X. Qian, and S. Ji. A space group symmetry informed network for (3)
equivariant crystal tensor prediction. In Forty-first International Conference on Machine Learning, 2024.
[59] S. Yang, K. Cho, A. Merchant, P. Abbeel, D. Schuurmans, I. Mordatch, and E. D. Cubuk. Scalable diffuusin for materials generation. In The Twelfth International Conference on Learning Representations, 2025.
[60] S. 7hang, Y. Tay, I. Yao, and Q. Liu. Quaternion knowledge graph embeddings. Advonces in neural information processing systems, 32, 2019.
13

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: As discussed in Section and and Section 5 the main claims made in the abstract and introduction accurately reflect the paper's contributions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
Limitations this work discussed in Appendix A.10
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
We do not include any theoretical results in the paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Detailed experimental settings, datasets, and computations needed are shared in Section S and Appendix A.8
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer:
[No]
Justification: The code not currently in a state ready for distribution. It will be released after have some time clean it up.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurlPS code and data submission guidelines (https
ssionpolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Detailed experimental settings are provided Section 5 and Appendix A.8 Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: The dataset used in the experiments are large and the results are relatively stable. Training and evaluating models for multiple times costly.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
16

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Detailed compute resources needed are provided in Section
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Question:
Answer: [Yes]
Justification:
This paper definitely follows the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We provide broader impacts in Appendix A.11
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All datasets and models used in this paper have been properly cited.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
19

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
20

---

A Appendix supplemental materials
A.1 Theoretical Justification of the Superiority of SPFrame over Local Frame
We provide a mutual information- proof to explain why the SPFrame approach yields more informative node/atom representations than the local frame method. Let crystal structure be represented as graph with N atoms, and denote the node/atom feature corresponding to atom as f.. The complete sets of node or atom features under two different framing schemes are defined as:
A = {flocal, 1.2, N), B (fep. 1, N..
(7)
where A represents node/atom features obtained using the local frame method, and B corresponds to those derived using the SPFrame method.
We assume that the atoms with indices p. (1 p, < N) are symmetry-equivatte The use of the local frame results in identical local environments for atoms p and q (as illustrated in Figure Consequently, after message passing via Equation 3 we have
flocat.
flocal.q:
(8)
which reduces the number of distinguishable atom representations. Denoting the cardinality we obtain
|.
A N <<
In contrast, the SPFrame approach preserves the differences in the local environments of atoms p and q. After message passing Equation [5] the node/atom representations satisfy
flocal.p + flocal.p
B N.
(10)
For scalar crystal property prediction, the final node features are first aggregated to get global graph-level representation, which is then passed through regression head. We denote the prediction target as Y E V. The neural network induces the following mapping:
Mocal A A V, hap B B
(11)
Since [B] > |A], there exists surjective mapping f1 B - A, such that hocen 91 hap- However, an injective mapping O2 : A + does not exist general due to loss of distinguishability in |A]. Consequently, the information flow can be described via the following Markov chain:
(12)
Applying the data processing inequality this chain yields
I(Y; B) > I(Y; A),
(13)
with equality if and only if
( and Y Y
(14)
i.e., the chain Y + A B is also valid. However, the absence of mapping 92 A + B implies that this reverse chain cannot be constructed. Therefore, the incquality is strict:
IY; > A);
(15)
This result implics that the node/atom representations obtained via SPFrame retain strictly higher mutual information with the target variable than those obtained via the local frame method. In other words, SPFrame-based features preserve more task-relevant information.
A.2 Proof of SE(3) invariance
This work also ensures SE(3) invariance 15
Let atoms and j be two neighboring atoms, with positions denoted by and Zy. respectively. In the message passing formulation of Equation 5 the edge scalar feature eiy can expressed as ||ær - æzyl. and the directional vector æig can be expressed as T; Zy. After applying global rotation Q and translation t, we have
16
21

---

(9l4 1a
I.h 1
E
giA)) (5 12 -1th 11 e,, E.y R. Fun Fliv.)
SaNar (1u-1)
E Jill (1 1/t ,(t- ||a 2yyl. (er, 2, 1 F Flavss)
Apply D
Q
ann
gMM)) (f.h-1)) f/--"1 [ax ( till (c=:Q (ar, +F (0 Q) Fiera)
f( (oia
Σ 6(h) (f f/--11 la a,l1, Qa F VianFoov.)
SCN(0)
1
@ (f s/t-" e. 2. Flaaa
SCN(
(16)
Applying a rotation Q and translation t does not change the expression in Equation Therefore, Equation unaffected by rotation and translation, indicating that it is SE(3)-invariant.
A.3 Implementation of Global Frame in SPFrame
As outlined Section 2.1 the entire crystal structure can be represented by its unit cell. When the
entire crystal structure undergoes rotation, the unit cell also changes accordingly. Therefore, the global frame can be computed from the lattice matrix L E R3x3 of the unit cell [2 52). Below, we introduce three commonly used methods for computing the global frame. Each of these methods can be applied to the global frame construction within the SPFrame.
QR Decomposition [33]. Given that the lattice matrix L E R3x3 is invertible, it can be uniquely decomposed via QR decomposition as = QR. where the diagonal elements of R are constrained to be positive. In this decomposition, Q p3x3 is orthogonal matrix, while R R3x3 is upper triangular matrix. By applying QR decomposition to L under this positivity constraint, we obtain the orthogonal matrix Q, which is naturally equivariant under O(3) transformations. To further restrict this equivariance to the SO(3) group. we flip the sign of the first column vector of Q if necessary to enforce dQt(Q) 1. The resulting matrix, now SO(3)-equivariint, serves choice for the global frame F Filobal in our proposed SPFrame.
Polar Decomposition [21] 18]. As an invertible matrix, the lattice matrix 3333 can be uniquely decomposed into L QH, where Q E 3333 is an orthogonal matrix, H E R3x3 is Hermitian positive semi-definite matrix. By applying polar decomposition to L, we obtain the orthogonal matrix Q, which is naturally equivariant under O(3) transformations. To ensure that Q is equivariant only under SO(3) transformations, we adjust the sign of its first column vector if needed to enforce det (Q) 1. The resulting matrix, now -quuaaaqui can be reliably utilized as the global frame F ggoobal in our SPFrame.
Principal Component Analysis (PCA) 9. For the lattice matrix L R333 we first compute the centroid of the lattice vectors as t = LL E R$, followed by the construction of the covariance matrix
(L- (L- (L-10 T). We then perform eigendecompossiiion on to obtain its eigenvectors
u1 u2 ug. Assuming the eigenvalues satisfy the condition A₁ > Ae > Xs the corresponding eigenvectors can be assembled into a x orthogonal matrix = [ui U22 ug), which defines one of eight possible O(3)-qquivariant frames. To obtain unique - global frame, apply Algorithmp to resolve the sign ambiguity (37 in U and enforce det(U) = 1. The resulting matrix U serves the global frame F global in SPFrame.
A.4 Implementation of Invariant Local Frame in SPFrame
Quaternion Generation via SO(3)nInvariant Message Passing, In this work, we adopt SO(3)- invariant message passing proposed by Yan et al. (56) to generate quaternions for constructing local frames. This process leverages the atom features fú. neighboring atom features fj. and invariant edge features eij to perform message passing from neighbor atom j to the central atom i. The messages
22

---

Algorithm 2 Unique SO(3)-equivarian global frame based on eigenvectors
Require: The orthogonal eigenvector matrices U [u U2; U3.
Ensure: The unique -Qui)-aquiaarian global frame U
for i=1,2 do
2: Let j be the smallest index such that u * 0
3:
if U 0 then
4:
11
5:
else
6:
u f- - end
7:
8: end for
9:
if (de(([u uz, us]) > 0 then
10:
ug Ug
11: else
12:
ug <- Ug
13: end if
14: U
[ui, u2 ug]
are aggregated across all neighbors, and the result is combined with the central atom's features fi to produce the quaternion q:
Specifically, first calculate three components: the query vector qi LNo (ft,, the key vector kij = (LNx x (f:) xe (fz)), and the value vector Vij = (LNv (f:] [LN=(())) where (- LNx(-) (-), LNv(-), (-), LNg()) denote the linear layers, and denote the concatenation. Then,
the message form atom j to atom is computed as:
4sj &K (k())
msgij = (B))) Ev
(17)
dag
where εK. εv represent mappings applied the key and value vectors, respectively, and the operators denote the Hadamard product, BN refers to the batch normalization layer, and /dq., indicates the dimensionality of qsj- Finally, the quaternion qi is generated as:
msg: Σ
msgij,
gi = Emns (IN s (fi + BN(msg:)),
(18)
JEN.
where Emag() (-) denotes the softplus activation function amd LN msgg denote the linear layer.
Quaternion to Rotation Matrix Conversion. Quaternions, while originating in pure mathematics, are extensively used for representing and computing 3D rotations [60]
46 unit quaternion,
represented by four real-valued components, encodes 3D rotation by specifying rotation axis and an associated rotation angle [.. Therefore, we normalize the network's 4-dimensional output to obtain a unit quaternion, which is then converted into rotation matrix 42
A.5
Backbone with SPFrame
The eComFormer has demonstrated strong performance across wide range of crystal property prediction tasks (57). This model integrates a node-wise transformer layer and node-wise equivariant updating layer to capture complex geometric relationships within the crystal structure. Given that the node-wise equivariant updating layer operates on equivariant edge features, we incorporate SPFrame into each of these layers. Furthermore, append node-wise equivariant updating layer following each wode-wise transformer layer. The detailed network architecture is provided below.
Node-wise transformer layer in eComFormer. The node-wise transformer layer in eComFormer updates node invariant features f. through a bessaag--pssiigg mechanism. This layer integrates three types of information: the node features f., neighboring node features f,- and invariant edge embeddings f The update process follows transformer-style architecture. Fristly, the message from node j node is encoded using three projected features query Zi = ..Ng (fi), key kij = (LN k (f.) |LN |LN k (f))), and value feature V.j (LNy (ff)) (f) (N))) where LNQ()).
23

---

Output
MLP
Linear+ SiLU
Node-w wise Equivariant Updating Layer
eg global FINv
Average
Pooling
SPFrame Module
M Message Passing with SPFrame
Node-wist wise Transformer Layer
Embedding Layer
ej
Invariant Edge Embedding
A tom Feature Embedding
ej
Figure 4: The detailed architectures of eComFormer with SPFrame.
(+). (-). L- denote the linear transformations, and denote the concatenation. Then, the attention mechanism computes:
4ig εK (key)
ai
msgis sigmoid((N(ax;))) (v
(19)
Vdas
where ÊK, εy are nonlinear transformations, and the operators denote the Hadamard product. BN(-) refers to the batch normalization layer, and Vdg., indicates the dimensionality of qıj- Then, node feature f; is updated as follows,
msg. Σ
msgij, J.uew Emag (fi + BN(msg:)),
(20)
JENS
where Emsg( (.) denoting the softplus activation function.
Node-wise equivariant updating layer using SPFrame. The node-wise equivariant updating layer in eComFormer employs two tensor product (TP) layers [12] to effectively capture geometric features. The equivalent edge feature Ejj is embedded using spherical harmonies, with the representations given by Y Yu(ēr;) = co, (ē,))) = C1
E R and Y((ē;) E R$. These harmonics form the TT
Z
input features to the TP layers.
Therefore, we apply the SPFrame the equivariant edge features before embedding them into spherical harmonics. Specifically, the first TP layer is defined as:
f/o= = f" +
1
TP Yole, F FivV.)). .)), JEN.
Σ
W
(21)
flA VI
TPXUF YA FT global FT J) X (1,2),
JEN.
where J is the linearly transformed atom feature derived from f. [V]] denotes the number of neighboring atoms of atom i, and TPx denotes the TP layer corresponding to rotation order X.
The second TP layer further aggregates the directional features across multiple orders as follows:
Σ
TP (2(e, global
F FTvv.))
(22)
JEAG A=
24

---

Finally, the outputs from the two TP layers are combined through both linear and nonlinear transfor- mations to produce the updated atom feature fl Jupdetedd
fupddated = o (BN((() LN(!).
(23)
where (-) denotes a nonlinear transformation consisting of two softplus layers with an intervening linear layer, while BN(() and LN(() represent batch normalization and linear layer, respectively.
Overall architecture. The overall architecture is illustrated in Fig. The key components of the network are summarized as follows. The architecture begins with embedding layers for node and edge features, followed by a series of stacked message passing modules. Each module consists of node-wise transformer layer, node-wise equivariant update layer, and SPFrame construction block. The network concludes with a global average pooling layer and a multi-layer perceptron (MLP) for property prediction. Notably, drawing inspiration from recent findings [19], which demonstrate that dynamically constructing frames at intermediate layers significantly enhances both model expressiveness and prediction accuracy, we integrate SPFrame construction modules at multiple stages throughout the network. Furthermore, since eComFormer produces $0(3)-invariam outputs, the global frame F gilobal in Equation 5 can be set as the identity matrix. This simplification enables more efficient implementation of our approach.
A.6 More Related Works
Other approaches for improving prediction accuracy. In crystal property prediction tasks, in addition to frame-based based methods (which can also be regarded as a form of representation learning), are two other important approaches
pretraining 48 and representation learning 17 35
for improving prediction accuracy.
Pretraining methods primarily focus on improving the backbone network architecture. CrysXPP designs an autoencoder for self-supervised pretraining, capturing key structural and chemical features from large amounts of unlabeled crystal graph data reduce prediction errors. CrysGNN 6 introduces a specialized pretrained GNN framework that combines feature reconstruction, connectivity reconstruction, and contrastive learning across different crystal systems. CrysDiff [48] employs diffusion-baase pretraining approach, where the pretraining phase reconstructs crystal structures via a diffusion process to learn the underlying edge distribution, and the fine-tuning phase generates target property values guided by structural data.
In contrast, representation learning focuses on constructing more expressive representations of crystal structures. Beyond commonly used bond angle information for encoding directionality, ALIGNN- d (17) incorporates dihedral angles, achieving memory-efficient graph representation that captures the full atomic geometry. CrysMMNet (5) integrates textual material descriptions into the crystal graph to encode global structural information, leading to richer and more robust representations. Geom3D (35) systematically benchmarks various geometric encoding strategies, including spherical harmonics, frame-based bases, and angle-based features. CrysAtomm [39] learns distributed atomic representations in an unsupervised manner from unlabeled crystal data, significantly improving downstream property prediction.
Incorporating crystal symmetry into method design. Crystal symmetry is a fundamental property of crystalline materials. Most existing methods for property prediction have only limited utilization of this symmetry, while many generative approaches explicitly leverage it to improve model performance. The core idea of these generative approaches is to simplify the data to be generated by exploiting crystal symmetry. Below, we introduce several representative methods.
Owing to crystal symmetry, the lattice matrix elements in different space groups are subject to specific constraints. DifICSP++ generates only the unconstrained lattice elements, simplifying the generation process. It also reconstructs atomic fractional coordinates and element types by deriving symmetry-equivart operations. This ensures that the generated crystals strictly satisfy space group constraints.
DiffCSP++ [2I]:
atoms from a single representative atom using symmetry
SymmCD 28 SymmCD generates only the asymmetric unit rather than the full lattice matrix, outputting its unit parameters. Unlike DiffCSP++, which generates complete lattice matrices using predefined templates, SymmCD demonstrates experimentally that DiffCSP++ may limit structural diversity and novelty.
25

---

Wyckoff Transformer [24]:
Wyckoff Transformer is an autoregressive generative method distinct
from diffusion- based approaches 21 28 25]. For symmetry-relatee atoms, generates only discrete attributes such as space group, element type, site symmetry, and enumeration. The complete crystal structure reconstructed by combining these discrete attributes with energy relaxation.
WyckoffDiff' [25]: Given a space group and Wyckoff positions, WyckoftDiff predicts the probability distribution of atom types occupying each position. This approach resembles UniMat [59], which predicts elemental probabilities from the periodic table, but WyckoffDiff explicitly embeds symmetry constraints into the generative process.
Our method, in contrast, is designed for scalar property prediction rather than generative modeling, primarily addresses the limitation of local frames, where atoms at symmetry -equivvleeeivvveenttt positions share identical local environments, leading to indistinguishable node features and information loss. To mitigate this, design the frame by combining an invariant local frame with equivariant global frame shared across the atomic system. This design preserves crystal symmetry and ensures that atoms at symmetry-equivaler positions maintain symmetric yet distinguishable local environments, allowing their node features to remain discriminative.
A.7 Frame Baseline
SO(3) equivariant frame constructed based on Gram-Schmidt orthogonalization [50 34 In- spired by previous work 44 50), we construct an equivariant frame by predicting two equivariant vectors using the Schmidt orthogonalizatton method, and use this frame baseline for comparison with the proposed cemmetry--rreserveee frame in this paper. Specifically, the two equivariant vectors Vi,1, Vi,2 E R are predicted as follows:
Vi,k =
E JEN(O)
on (fr. Cin)) E (1,2),
(24)
where fi denotes the feature vector of atom i, eij represents SOv3)-invariant edge features, and ē. corresponds SO(3)-equivarian edge features. (x(()) the message function from Yan et al. (56) The rotation matrix is then constructed using Gram-Schmidt orthogonalization as follows:
V
V1,1
V1,1
V1,2 = V12 (n..1 V.2) V4.2
|
(25)
V13
Vi2
FINV,
W V2,,, vr.ll
SPFrame constructed based on Gram-Schmide orthogonalizatton During the construction of the SPFrame, we need to establish invariant local frame and equivariant global frame. Similar to Eq. 24 we predict two invariant vectors using only invariant edge features:
Vi.k Σ du (i.)), E (1,2). JEN())
(26)
The rotation matrix FINV,; is then constructed using Eq. 25
The equivariant global frame F glohaal Ultimately, this yields the Symmetry- preserving frame F FINv, F.w,FFgthhll glohal based on Gram-Schmidt orthogonalization.
still derived using the method described in Appendix A.3
Incorporating angular information. When computing the local frame using the Craam-Schmidt orthogonalization method as defined in Equation 25 the intrinsic symmetry of the crystal can lead to cases during training where the vectors v,, and Vi,2 become collinear. This collinearity prevents the Gram-Schmidd orthogonalization method from producing a valid local frame.
To overcome this limitation, we incorporate angular information into Equation 25 Specifically, for each atom within the unit cell, we first compute the frame (such as PCA frame Appendix (A.) of the equivariant edge vectors and compute the frame of the vectors the lattice matrix. These vectors are then transformed into invariant representations. Next, we calculate the angles (57) between the transformed edge vectors and the transformed lattice vectors. These angle-based features are then integrated into the invariant edge features [57], enhancing the robustness of the frame construction.
26

---

A.8 Training Settings
In this subsection, provide the detailed hyperparameter settings for backbone integrated with SPFrame across different tasks. For the network architecture, the backbone follows the parameter settings outlined in the original paper [57], such as those for the graph construction and the embedding layers. The training hyperparameters are as follows.
JARVIS: formation energy. For the eComFormer backbone, the network is trained using LI loss with the Adam optimizer (26) for 500 epochs, employing the Onecycle scheduler 47] with a stt_start of 0.3 and an initial learning rate of 0.0005. The network consists of 2 message passing layers and 3 SPFrame modules. Each message passing layer is equipped with one SPFrame module, and an additional SPFrame module is placed before the first message passing layer. The intermediate features, such as node features and invariant edge features, are set to 256 dimensions, and the batch size is set to 64. For the iComFormer backbone, the network is trained using L1 loss with the Adam optimizer for 700 epochs. employing the Onecycle scheduler with a pet_starr of 0.3 and an initial learning rate of 0.001. The network consists of total of 4 message passing layers, each equipped with an SPFrame module except for the final layer. The dimensionality of all features is set to 256, and the batch size is 64.
JARVIS: band gap (OPT). For the eComFormer backbone, the network is trained using the L1 loss function and the Adam optimizer for 500 epochs. A cosine with warmup scheduler is employed [54], with an initial learning rate of 0.001 and a warmup phase corresponding to 5% of the total training steps. The network consists of a total of 2 message passing layers, each equipped with an SPFrame module. The feature dimension is set to 128, and the batch size is 64. For the iComFormer backbone, the network is trained using the L1 loss function and the Adam optimizer for 500 epochs. A cosine with warmup scheduler is employed, with an initial learning rate of 0.001 and a warmup phase corresponding to 5% of the total training steps. The network consists of total of message passing layers, each equipped with an SPFrame module except for the final layer. The feature dimension is set to 128, and the batch size is 64.
band gap (MBJ). For the eComFormer backbone, the network is trained using the LI loss function and the Adam optimizer for 500 epochs. A cosine with warmup scheduler is employed, with an initial learning rate of 0.003 and a warmup phase corresponding to 5% of the total training steps. The network consists of total of message passing layers, each equipped with an SPFrame module. The feature dimension is set to 128, and the batch size is 64. For the iComFormes backbone, the network is trained using L1 loss with the Adam optimizer for 1000 epochs, employing the Onecycle scheduler with a of 4 message passing layers, each equipped with an SPFrame module except for the final layer. The feature dimension is set to 256, and the batch size is 64.
JARVIS:
of 0.3 and an initial learning rate of 0.001. The network consists of a total
total energy. For the eComFormer backbone, the network is trained using L1 loss with the Adam optimizer for 1000 epochs, employing the Onecycle scheduler with a pet_start start of 0.3 and an initial learning rate of 0.001. The network consists of a total of 2 message passing layers, each equipped with an SPFrame module. The feature dimension is set to 128, and the batch size is 32. For the iComFormer backbone, the network is trained using L1 loss with the Adam optimizer for 1000 epochs, employing the Onecycle scheduler with a pct_ start of 0.3 and an initial learning rate of 0.001. The network consists of total of message passing layers, each equipped with an SPFrame module except for the final layer. The feature dimension is set to 256, and the batch size is 64.
JARVIS:
JARVIS: Ehull. For the eComFormer backbone, the network is trained using L1 loss with the Adam optimizer for 500 epochs, employing the Onccycle scheduler with a pct_ start of 0.3 and an initial learning rate of 0.001. The network consists of total of 2 message passing layers, each equipped with an SPFrame module. The feature dimension is set to 128, and the batch size is 64. For the iComFormer backbone, the network is trained using LI loss with the Adam optimizer for 1000 epochs, employing the Onecycle scheduler with a pet_start
start of 0.3 and an initial learning rate of 0.001. The network consists of total of message passing layers, each equipped with an SPFrame module except for the final layer. The feature dimension is set to 128, and the batch size is 64.
JARVIS: bulk modulus. The network is trained using the L.1 loss function and the Adam optimizer for 500 epochs. cosine with warmup scheduler is employed, with an initial learning rate of 0.001 and warmup phase corresponding to 5% of the total training steps. The network consists of a total
27

---

of 2 message passing layers, each equipped with an SPFrame module. The feature dimension is set to 128, and the batch size 64.
JARVIS: shear modulus. The network is trained using the L.1 loss function and the Adam optimizer for 500 epochs. A cosine with warmup scheduler is employed, with an initial learning rate of 0.001 and a warmup phase corresponding to 5% of the total training steps. The network consists of a total of 3 message passing layers, each equipped with an SPFrame module. The feature dimension is to 128, and the batch size is 64.
MP: formation energy. The network is trained using L1 loss with the Adam optimizer for 500 epochs, employing the Onecycle scheduler with a start of 0.3 and an initial learning rate of 0.001. The network consists of total of 2 message passing layers, each equipped with an SPFrame module. The feature dimension is set to 196, and the batch size is 32.
MP: band gap. The network is trained using L.1 loss with the Adam optimizer for 500 epochs, employing the Onecycle scheduler with a pet_staan of 0.3 and an initial learning rate of 0.001. The network consists of total of message passing layers, each equipped with SPFrame module. The feature dimension set to 128, and the batch size 32.
MP: bulk moduli. The network is trained using L1 loss with the Adam optimizer for 500 epochs, employing the Onecycle scheduler with pet__sttteetpe start of 0.3 and an initial learning rate of 0.001. The network consists of total of message passing layers, each equipped with an SPFrame module. The feature dimension set to 512, and the batch size 64.
MP: shear moduli. The network trained using MSE loss with the Adam optimizer for 500 epochs, employing the Onecycle scheduler with pet__taae of 0.3 and initial learning rate of 0.001. The network consists of total of message passing layers, each equipped with an SPFrame module, The feature dimension set to 128, and the batch size is 64.
A.9 Additional Experimental Results
To further demonstrate the generality of SPFrame, conducted additional experiments by integrating SPFrame with iComFormer [57]. Specifically, iComformer consists of the node-wise transformer layer and the edge-wise transformer layer. In implementation, an SPFrame construction block was added after each node-wise transformer layer. The resulting frame was applied to the interatomic edge vectors, and then the angles between these edge vectors and the lattice matrix before being fed into the edge-wise transformer layer for edge feature updates. Table 4 presents the experimental results combining iComFormer with SPFrame on the JARVIS dataset. Because iComFormer serves as a more powerful backbone, the combined model achieves superior performance compared to the model using eComFormer as the backbone across most prediction tasks.
computed
Table 4: Additional property prediction results on the JARVIS dataset.
Form. energy
Total energy
Bandgap (OPT)
Bandgap (MRJ)
E hull
Method
Vatom
eVatom
eV
Crystalframer
0.0263
0.0279
0.117
0.242
0.047
commFrrner
0.0284
0.0315
0.124
0.283
0.044
w/ -equiiqquaa Gram-Schmidt local frame
0.0285
0.0296
0.115
0.271
0.043
Quaternion-baid SPFrame (ours)
0.0276
ioomorrnr
0.0272
0.0288
0.122
0.26
0.047
w/ (30(()-equivaar)) Gram
Grram-Schmidt Schmidt local frame
0.0275
0.0287
0.112
0.255
0.045
w/ Quaternion- bused SPF rame (ours)
0.0250
0.0259
0.106
0.251
0.042
Table presents the experimental results for bulk modulus and shear modulus prediction on the
JARVIS IS dataset. CrystalFramer achieves the best performance on bulk modulus, consistent with the results observed on the MP dataset, while our method slightly outperforms CrystalFramer on shear modulus.
A.10 Limitations
This work investigates the issue that conventional local frame methods, when applied to crystal structures, may inadvertently disrupt the intrinsic symmetry of the crystal. To address this problem,
28

---

Table 5: Additional bulk and shear modulus prediction results the JARVIS dataset.
Method
Bulk Modulus (Kv)
Shear Modulus (Gv)
Matformer
11.21
10.76
CrysGNN 6
10.99
9.800
CrysDiff
9.875
9.193
Crystalframer
8.876
8,999
eComFormer
9.777
9,435
-Quua-equivariant Gram-Schmidt local frame
9.855
9.689
w/ Quaternion-based SPFrame (ours)
9.357
8.963
propose SPFrame. Comparative experiments against conventional local frame approaches demonstrate the effectiveness SPFrame. However, while empirical results validate the benefits of SPFrame, this study does not provide quantitative theoretical analysis of how symmetry breaking impacts the accuracy crystal property prediction. This question remains unexplored and closely related to the broader topic of model interpretability in neural networks [4] 30 31 53 29]. Future research could pursue a theoretical framework to quantify the effects of symmetry disruption and further elucidate its influence prediction performance.
A.11 Broader Impacts
As a frame-based method tailored for crystal structures, SPFrame enhances the accuracy prediction models and facilitates the discovery of new materials with desirable properties. Therefore, this work has the potential to make a meaningful impact in the field of materials science. Furthermore, SPFrame offers perspective at the intersection of machine learning and materials science. By adapting machine learning techniques to account for the unique characteristics of crystal systems, SPFrame significantly improve model performance.
demonstrates how domain-specte This highlights the importance of developing specialized methodologies to promote the effective application of machine learning in materials science.
modifications
29


Reviews:
[{"review_id": "Pe0WpILadh", "reviewer": "Reviewer_5W3r", "summary": "Machine learning tools are used for predicting various properties of crystals. Many of these properties, such as potential energy of the system, are invariant with respect to rotations, and more generally, to rigid transformations of an input crystal. A naive approach to constructing a model that predicts invariant outputs is to limit the used features to only those that are invariant with respect to rotations; however, this limits the expressivity of the corresponding models. An alternative approach is to define global or local frames, which would canonicalize the crystal's orientation, thereby making the predictions naturally invariant with respect to rotations. Both purely global and local frames, at least one particular method used as a reference, have their own limitations; thus, the paper introduces a new method that combines both ideas to efficiently construct rotationally invariant architectures expressive enough for the accurate prediction of potential energy and other properties of crystals.", "questions": "Is your method smooth or not?\n\nDid I understand correctly that the features f in equations 2 and 3 are invariant with respect to rotations, and not equivariant ones such as those of Tensor Field Network, Nequip, Mace, and many others?\n\nDo you think that your method is more or less efficient compared to equivariant Graph Neural Networks, which are the de facto standard in the field, and for which there exists the mentioned completeness theorem (along with a few others)?", "limitations": "Some of the limitations are addressed in the text. \n\nThe paper would benefit, in my opinion, from the discussion of such issues as the lack of a formal universal approximation theorem to back up the claim of superior expressivity, and potential questions of smoothness.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "The paper clearly identifies limitations of the previous work and proposes a way to overcome these issues. Additionally, it provides thorough benchmarks on more than one dataset against multitude of other state-of-the-art models. On top of that, it provides a clear analysis of the required computational cost. \n\nHowever, I have a few concerns regarding this paper. The first one is related to the lack of any sort of formal and systematic analysis of the expressive power, which the paper claims to improve compared to previous approaches. For many machine learning architectures, there exist explicit completeness theorems that establish that these machine learning architectures are universal approximators. There is, for instance, universal approximation theorem for MLPs, Kolmogorov Arnold representation theorem for KANs, and, the most relevant for machine learning on crystal structures, the one which states that certain class of rotationally equivariant graph neural networks is a universal approximator when predicting invariant properties for crystal structures, the one introduced here \"On the Universality of Rotation Equivariant Point Cloud Networks\" (I am not affiliated). The reviewed paper claims that it leads to increased expressivity of the corresponding machine learning models, but doesn't provide any formal definition of this increase. It is unclear whether the proposed method leads to architectures featuring a universal approximation property, similarly to the aforementioned approaches, or not. In the current way of representation, it is not even 100% convincing that the proposed method is strictly more expressive compared to the discussed approach based solely on local frames. It is well illustrated that the proposed method eliminates some of the issues of solely local frames, but it can simultaneously introduce new ones. \n\nMy another concern is the smoothness of the method with respect to geometric deformations of input crystal configurations. Can it be that a small movement of the atoms introduces rapid and discontinuous change in the predicted potential energy or other property? This property can be as important, if not even more, than the rotational invariance to run practical molecular dynamics simulations. While not 100% sure, I got the impression that the proposed method can sometimes introduce discontinuous jumps of local and global frames, which in turn would lead to discontinuities in the predicted target property. In the case of Gram-Schmidt diagonalization, this can occur when the initial vectors are collinear. For quaternions, the problematic case potentially leading to discontinuities is when a = b = c = d = 0. It would be helpful if the authors could clarify this matter.", "was_revised": true, "final_justification": "I appreciate the clarifications made by the authors, and I am willing to increase my score.", "rebuttal": "We sincerely thank you for your constructive suggestions. Below we will address your questions in detail.\n\n****\n\n**$\\bullet$ Response to weakness 1 (formal analysis of the expressive power)**: \n\nWe sincerely thank you for your constructive suggestions. \n\nWe provide a mutual information-based proof to explain why the SPFrame approach yields more informative node/atom representations than the local frame method.\n\nLet a crystal structure be represented as a graph with $N$ atoms. Denote the node/atom feature corresponding to atom $i$ as $f_i$.  We define the complete set of node/atom features under two different framing schemes as follows:\n\n$$A=\\\\{f_{local,i}|i=1,2,..,N\\\\}, B=\\\\{f_{sp,i}|i=1,2,..,N\\\\},$$ \n\nwhere $A$ corresponds to node/atom features in the network using local frames, and $B$ corresponds to those in the network using the SPFrame approach.\n\nWe assume that the atoms with indices $p,q$ ($1<p,q<N$) are symmetry-equivalent. \nThe use of the local frame results in identical local environments for atoms p and q (According to the illustration in Figure 2 in our paper, page 4). As a consequence, after message passing via Equation (3) (in our paper, page 4), we can have\n\n$$f_{local,p}=f_{local,q}. $$\n\nConsequently, the effective number of distinguishable node/atom representations is reduced. Denoting the cardinality as $|\\cdot|$, we can have:\n\n$$|A|=N-1<N.$$\n\nIn contrast, the SPFrame approach preserves the differences in the local environments of atoms p and q. After message passing via Equation (5) (in our paper, page 5), the node/atom representations satisfy:\n\n$$f_{local,p} \\neq f_{local,p}  \\implies |B|=N.$$\n\n\nFor scalar crystal property prediction, the final node features are first aggregated to get a global graph-level representation. This graph-level representation is then passed through a regression head. We denote the prediction target as a variable  $Y \\in \\mathcal{Y}$. The neural network induces the following mapping:\n\n\n$$h_{local}:A \\mapsto \\mathcal{Y}, \\quad h_{sp}:B \\mapsto \\mathcal{Y}.$$\n\nSince $|B|>|A|$, there exists a surjective mapping $g_1:B \\mapsto A$, such that $h_{local}\\circ g_1=h_{sp}$. However, an injective mapping $g_2:A \\mapsto B$ does not exist in general due to loss of distinguishability in $|A|$. Consequently, the information flow can be described via the following Markov chain:\n$$Y \\to B \\to A.$$\n\nApplying the data processing inequality to this chain yields:\n$$I(Y;B) \\ge I(Y;A),  $$\n\nwith equality if and only if:\n\n$$I(B;Y | A) =0 \\quad\\text{and}\\quad Y \\to A \\to B$$\n\ni.e., the chain $Y \\to A \\to B$ is also valid. However, the absence of a mapping $g_2:A \\mapsto B$ implies that this reverse chain cannot be constructed. Therefore, the inequality is strict: \n$$I(Y;B) \\gt I(Y;A),  $$\n\n\nThis result implies that the node/atom representations obtained via SPFrame retain strictly higher mutual information with the target variable $Y$ than those obtained via the local frame method. In other words, SPFrame-based features preserve more task-relevant information.\n\nWe hope our response can address your concerns.\n\n\n\n****\n\n**$\\bullet$ Response to weakness 2 (smooth) and questions 1**:  We sincerely thank you for your insightful comments. In practical applications, our method can be made smooth. Take the Quaternion-based SPFrame as an example. In our implementation, for a quaternion (a,b,c,d),  we only predict the values of b,c, and d, while a is fixed to 1 by default (see our Anonymous GitHub repository, file ./models/transformer.py,  line 229, function quaternion_1ijk_to_rotation_matrix(q)). This design prevents the network from encountering the discontinuity issue when a = b = c = d = 0 during the normalization step in rotation matrix construction. \n\nMoreover, this paper focuses primarily on scalar property prediction for crystals, which does not impose smoothness requirements. If one aims to extend the method to tasks such as force prediction, smooth rotation matrix construction methods can be adopted to inherently avoid discontinuities, such as the SVD-based rotation projection proposed in [1]. \n\nWe hope our response addresses your concerns.\n\n\n[1] Learning with 3D rotations, a hitchhiker’s guide to SO(3). *ICML2024*.\n\n****\n\n**$\\bullet$ Response to questions 2**:  \n\nThank you for your question. This work primarily investigates the application of frame-based methods in the prediction of scalar properties of crystals, where invariance is required. Therefore, the features f in equations 2 and 3 are invariant with respect to rotations. In contrast, methods such as NequIP focus on predicting interatomic potentials, where equivariance is necessary for accurate force prediction. As a result, these methods incorporate spherical harmonics and tensor products in their network architecture to construct equivariant representations. We hope our response addresses the reviewer’s question.\n\n****\n\n**$\\bullet$ Response to questions 3**:   \n\nWe sincerely thank you for your comments.\nFrame-based methods have already been applied to force prediction tasks [1,2], as they decouple the network from the strict requirement of equivariance. This allows them to avoid the high computational costs associated with architectures that rely on spherical harmonics and tensor products to achieve equivariance. Instead, frame construction methods offer lower computational overhead compared to tensor products, enabling more efficient models. This work primarily focuses on the application of frame-based methods for scalar property prediction in crystals, where the goal is to achieve invariance and to improve predictive accuracy. If our method is extended to force prediction tasks, it can theoretically achieve higher efficiency in the same manner as shown in [1,2].\n\n\n[1] Graph neural network with local frame for molecular potential energy surface. *LoG2022*.\n\n[2] A new perspective on building efficient and expressive 3D equivariant graph neural networks. *NeurIPS2023*."}, {"review_id": "1wkVK4Rv3H", "reviewer": "Reviewer_drCP", "summary": "This paper introduces SPFrame, a method for enforcing SO(3) invariance by combining local and global frames. It addresses a limitation of prior equivariant local frame methods, which fail to distinguish atoms in the same Wyckoff position that differ only by rotation. SPFrame resolves this by first aligning SO(3)-equivariant edge features to a global frame to ensure invariance, and then further refining the representation using a local invariant frame to distinguish atoms within the same Wyckoff position. The approach achieves state-of-the-art performance on the MP and JARVIS datasets.", "questions": "1. How is translational invariance handled in the proposed method?\n2. Have the authors considered using other architectures (besides eComFormer) or providing additional evidence to support the method’s general effectiveness?\n3. Although the method achieves SOTA performance, the results are very close to Crystalframer. Have the authors considered reporting metrics over multiple runs to assess statistical significance?\n4. (Minor) What exactly is $\\hat{e}_{ij}$ in this work? The paper describes it as SO(3)-equivariant edge features, but this is vague. Is it simply the edge vector between atoms i and j?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "# Strengths\n- The paper is clearly written and easy to follow, with self-contained preliminaries, well-designed figures, and helpful visualizations.\n- The proposed method is simple yet appears effective.\n\n# Weaknesses\n- The motivation is not entirely convincing. Atoms at the same Wyckoff position share identical local environments, so it seems reasonable for them to have the same representation. The argument that they should be distinguished based on their rotational differences is not convincing.\n- The paper lacks a discussion of translational invariance. While the proposed method enforces SO(3) invariance, prior works (e.g., Crystal Framer, CrystalFormer) guarantee full SE(3) invariance.\n- Although the method achieves state-of-the-art performance overall, the improvements over the baselines appear marginal (e.g., 0.0261 vs. 0.0263, 0.0172 vs. 0.0171), and it is unclear whether these differences are statistically significant.\n- Since the method is only tested with eComFormer, its general effectiveness remains uncertain.\n\n## Typos (and/or minor comments)\n- Equation 1: Should be $f_{\\theta}(A, QX, QL) = f_{\\theta}(A,X,L)$. The current equation is SO(3)-equivariance. \n- Line 98: The rightarrow notation seems a bit confusing\n- Line 217: CrystalFormer [32] -> Crystalformer \n- Line 219: Crystalformer [36] -> CrystalFormer", "was_revised": true, "final_justification": "The authors have clarified their motivation and addressed my concerns about translational invariance. They have also shown that their method works well on both iComFormer and eComFormer. I support the acceptance of this paper.", "rebuttal": "We sincerely thank you for your thoughtful feedback. Below we will address your questions in detail.\n\n****\n\n**$\\bullet$ Response to weakness 1**:   \n\nWe sincerely thank you for your valuable comments.  \nFirst, it is important to clarify that atoms at the same Wyckoff position share similar local environments, but not identical ones. As illustrated in Figure 2 of our paper (page 4), they can be transformed into the same local environment through symmetry operations such as rotations. Below, we discuss our motivation both from an intuitive and theoretical perspective.\n\n**From an intuitive perspective:** As mentioned in the Introduction of our paper, directional information between atoms is a crucial geometric feature. Leveraging this information can enhance network performance. For example, ComFormer [2] improves upon Matformer [1] by incorporating directional information into the message passing process. However, as shown in Figure 2 of our paper, applying a local frame to ComFormer causes the directional information of symmetry-equivalent atoms p and q to be lost. This results in the node features of atoms p and q becoming identical across all subsequent message passing layers of the network, effectively degrading ComFormer back to Matformer. In contrast, the original ComFormer preserves the directional distinction between p and q, ensuring that their node features remain different in all subsequent layers.\nSPFrame is designed to overcome this limitation of the local frame. It ensures that the node features of symmetry-equivalent atoms p and q remain distinct throughout all layers of message passing, just as in the original ComFormer, thereby preserving essential geometric information.                \n\n\n**From a theoretical perspective:** Let a crystal structure be represented as a graph with $N$ atoms. Denote the node/atom feature corresponding to atom $i$ as $f_i$.  We define the complete set of node/atom features under two different framing schemes as follows:\n\n$$A=\\\\{f_{local,i}|i=1,2,..,N\\\\}, B=\\\\{f_{sp,i}|i=1,2,..,N\\\\},$$ \n\nwhere $A$ corresponds to node/atom features in the network using local frames, and $B$ corresponds to those in the network using the SPFrame approach.\n\nWe assume that the atoms with indices $p,q$ ($1<p,q<N$) are symmetry-equivalent. \nThe use of the local frame results in identical local environments for atoms p and q (According to the illustration in Figure 2 in our paper, page 4). As a consequence, after message passing via Equation (3) (in our paper, page 4), we can have\n\n$$f_{local,p}=f_{local,q}. $$\n\nConsequently, the effective number of distinguishable node/atom representations is reduced. Denoting the cardinality as $|\\cdot|$, we can have:\n\n$$|A|=N-1<N.$$\n\nIn contrast, the SPFrame approach preserves the differences in the local environments of atoms p and q. After message passing via Equation (5) (in our paper, page 5), the node/atom representations satisfy:\n\n$$f_{local,p} \\neq f_{local,p}  \\implies |B|=N.$$\n\n\nFor scalar crystal property prediction, the final node features are first aggregated to get global graph-level representation. This graph-level representation is then passed through a regression head. We denote the prediction target as a variable  $Y \\in \\mathcal{Y}$. The neural network induces the following mapping:\n\n\n$$h_{local}:A \\mapsto \\mathcal{Y}, \\quad h_{sp}:B \\mapsto \\mathcal{Y}.$$\n\nSince $|B|>|A|$, there exists a surjective mapping $g_1:B \\mapsto A$, such that $h_{local}\\circ g_1=h_{sp}$. However, an injective mapping $g_2:A \\mapsto B$ does not exist in general due to loss of distinguishability in $|A|$. Consequently, the information flow can be described via the following Markov chain:\n$$Y \\to B \\to A.$$\n\nApplying the data processing inequality to this chain yields:\n$$I(Y;B) \\ge I(Y;A),  $$\n\nwith equality if and only if:\n\n$$I(B;Y | A) =0 \\quad\\text{and}\\quad Y \\to A \\to B$$\n\ni.e., the chain $Y \\to A \\to B$ is also valid. However, the absence of a mapping $g_2:A \\mapsto B$ implies that this reverse chain cannot be constructed. Therefore, the inequality is strict: \n$$I(Y;B) \\gt I(Y;A),  $$\n\n\nThis result implies that the node/atom representations obtained via SPFrame retain strictly higher mutual information with the target variable $Y$ than those obtained via the local frame method. In other words, SPFrame-based features preserve more task-relevant information.\n\n\n\n[1] Periodic graph transformers for crystal material property prediction. *NeurIPS2022*.\n\n[2] Complete and efficient graph transformers for crystal material property prediction. *ICLR2024*.\n\n\n\n\n\n\n****\n\n**$\\bullet$ Response to weakness 2 and question1**:   \n\nWe sincerely thank you for your constructive comments.\n\nThis work also ensures SE(3) invariance. We provide the proof below. Let atoms i and j be two neighboring atoms, with positions denoted by $p_i$ and $p_j$, respectively.\nIn the message passing formulation of Equation (5) (in our paper, page 5), the edge scalar feature $e_{ij}$ can be expressed as $||p_i-p_j||$, and the directional vector $\\widehat{ e}_{ij}$ can be expressed as $p_i-p_j$. After applying a global rotation $R$ and translation $t$, we have\n\n\\begin{align}\n\\boldsymbol{f}_i^{(k)} &= \\psi^{(k)} ( \\boldsymbol{f}_i^{(k-1)}, \\sum\\_{j \\in \\mathcal{N}(i)} \\phi^{(k)} (\\boldsymbol{f}_i^{(k-1)}, \\boldsymbol{f}_j^{(k-1)},e\\_{ij}, \\widehat{e}\\_{ij} \\mathbf{F}^{\\top}\\_{\\text{global}} \\mathbf{F}^{\\top}\\_{\\text{INV},i})) \\\\\\\\\n&=  \\psi^{(k)} ( \\boldsymbol{f}_i^{(k-1)}, \\sum\\_{j \\in \\mathcal{N}(i)} \\phi^{(k)} (\\boldsymbol{f}_i^{(k-1)}, \\boldsymbol{f}_j^{(k-1)},||p_i-p_j||, (p_i-p_j) \\mathbf{F}^{\\top}\\_{\\text{global}} \\mathbf{F}^{\\top}\\_{\\text{INV},i})) \\\\\\\\\n&\\text{Apply R and t:}\\\\\\\\\n&=  \\psi^{(k)} ( \\boldsymbol{f}_i^{(k-1)}, \\sum\\_{j \\in \\mathcal{N}(i)} \\phi^{(k)} (\\boldsymbol{f}_i^{(k-1)}, \\boldsymbol{f}_j^{(k-1)},||(p_iR-t)-(p_jR-t)||, ((p_iR-t)-(p_jR-t)) (\\mathbf{F}\\_{\\text{global}}  R)^{\\top}\\mathbf{F}^{\\top}\\_{\\text{INV},i})) \\\\\\\\\n&=  \\psi^{(k)} ( \\boldsymbol{f}_i^{(k-1)}, \\sum\\_{j \\in \\mathcal{N}(i)} \\phi^{(k)} (\\boldsymbol{f}_i^{(k-1)}, \\boldsymbol{f}_j^{(k-1)},||p_i-p_j||, (p_i-p_j)RR^{\\top} \\mathbf{F}^{\\top}\\_{\\text{global}} \\mathbf{F}^{\\top}\\_{\\text{INV},i})) \\\\\\\\\n&= \\psi^{(k)} ( \\boldsymbol{f}_i^{(k-1)}, \\sum\\_{j \\in \\mathcal{N}(i)} \\phi^{(k)} (\\boldsymbol{f}_i^{(k-1)}, \\boldsymbol{f}_j^{(k-1)},e\\_{ij}, \\widehat{e}\\_{ij} \\mathbf{F}^{\\top}\\_{\\text{global}} \\mathbf{F}^{\\top}\\_{\\text{INV},i})) \\\\\\\\\n\\end{align}\n\nApplying a rotation $R$ and translation $t$ does not change the expression in Equation (5). Therefore, Equation (5) is unaffected by rotation and translation, indicating that it is SE(3)-invariant.\n\nWe hope our response addresses your concerns.\n\n\n****\n\n**$\\bullet$ Response to weakness 4 and question 2**:   \n\nWe sincerely thank you for your valuable comments.\n\nWe conducted experiments using iComFormer [1] as the backbone, which outperforms eComFormer. Below, we provided the evaluation of JARVIS dataset. Notably, our solution outperforms or at least is on par with the baselines. The preliminary results are as follows.\n\n\n### Table: Property prediction results on the JARVIS dataset\n\n| Method                                               | Form. energy (eV/atom) | Total energy (eV/atom) | Bandgap (OPT) (eV) | Bandgap (MBJ) (eV) | E hull (eV) |\n|---------|--------|----------|-----|--------|--------------|\n| Crystalframer      | 0.0263    | 0.0279     | 0.117    | 0.242     | 0.047        |\n| eComFormer       | 0.0284      | 0.0315   | 0.124   | 0.283       | 0.044        |\n| — w/ SO(3)-equivariant Gram-Schmidt local frame      | 0.0285     | 0.0296     | 0.115| 0.271 | 0.043|\n| **— w/ Quaternion-based SPFrame (ours)**  | 0.0261  | 0.0276 | 0.107  | **0.239**           | **0.042**    |\n| iComFormer          | 0.0272     | 0.0288   | 0.122   | 0.261  | 0.047  |\n| — w/ SO(3)-equivariant Gram-Schmidt local frame      | runing    |runing       | 0.112     |  0.255     | 0.045        |\n| **— w/ Quaternion-based SPFrame (ours)**        | **0.0250**  | **0.0259** | **0.106** | 0.251               | **0.042**    |\n\nThe performance improvement of SPFrame on another backbone (iComFormer) demonstrates its general effectiveness.\n\nWe hope our response addresses your concerns.\n\n\n\n[1] Complete and efficient graph transformers for crystal material property prediction. *ICLR2024*.\n\n\n\n\n****\n\n**$\\bullet$ Response to weakness 3 and question 3**:   \n\n\n\nWe sincerely thank you for your comments. For fair comparisons, we just follow the experimental settings in the literature [1,2,3,4]. We agree that statistical results can be more rigorous. To reflect the statistical results, we evaluate various backbones for addressing the significance of our solution. Specifically, we replaced the backbone with a more powerful model to achieve a more significant improvement over the baseline. The detailed results are as iComFormer results (in Response to weakness 4 and question 2).\n\nWe hope our response addresses your concerns.\n\n\n[1] Periodic graph transformers for crystal material property prediction. *NeurIPS2022*.\n\n[2] Complete and efficient graph transformers for crystal material property prediction. *ICLR2024*.\n\n\n[3] Crystalformer: Infinitely Connected Attention for Periodic Structure Encoding. *ICLR2024*.\n\n[4] Rethinking the role of frames for SE (3)-invariant crystal structure modeling. *ICLR2025*.\n\n\n\n\n\n\n****\n\n**$\\bullet$ Response to question 4**:  \nThank you for your comment. In fact, $\\widehat{\\boldsymbol{e}}_{ij}$ refers to the edge vector between atoms $i$ and $j$.\n\n\n\n****\n\n**$\\bullet$ Response to Typos**:   \n\nWe sincerely thank you for your careful review. We will revise these typos. We will remove the \"rightarrow\" notation on Line 98 and retain only the equation $(\\mathbf{A},\\mathbf{Q}_g\\mathbf{X}+\\mathbf{t}_g,\\mathbf{Q}_g\\mathbf{L})=(\\mathbf{A},\\mathbf{X},\\mathbf{L})$."}, {"review_id": "P0FyT8E6Q6", "reviewer": "Reviewer_7QqS", "summary": "In this paper, the authors addressed an important task in material design: crystal property prediction. A fundamental requirement for crystal property prediction is that the model's output should remain invariant to arbitrary rotations of the input structure. This is essential because crystal structures are inherently equivariant under SO(3) group transformations (i.e., rotations).\nThe global frame approach can be integrated with any GGNN without imposing constraints on the network architecture, while still ensuring adherence to the SO(3) invariance requirement. However, using only local or only global frames comes with certain limitations.\nThe authors proposed Symmetry-Preserving Frames for modeling crystal structures, where a combination of local and global associative frames is integrated with GNN for crystal property prediction. This approach helps preserve the inherent symmetry of the crystal.", "questions": "Check the weaknesses", "limitations": "Check the weaknesses", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n- The paper is well-written, with a clearly motivated problem and sufficient background information provided to support the study.\n- The authors introduced both global and local frames and integrated them with GNN models. The local frames correspond to invariant local frames for each atom ii, while the global frame is an equivariant frame shared across the entire atomic system. The global frame ensures that the output of the GGNNs remains invariant, whereas the invariant local frames are constructed using a method based on local structural information, ensuring rotational invariance at the atomic level.\n- The authors also ensure that atoms pp and qq, which occupy the same type of Wyckoff position, are assigned identical local frames. This guarantees that their transformed representations remain distinguishable while still preserving their relative structural differences.\n- Source code is provided\n\n**Weaknesses**\n\n- **Limited Novelty** – While frame-based methodologies have shown strong potential in enforcing equivariance and invariance in geometric deep learning, the authors have primarily applied these existing concepts to the domain of crystal structure prediction. This adaptation, though technically sound, positions the work as more incremental than groundbreaking, as it builds upon well-established techniques rather than introducing fundamentally new methods or insights.\n\n- **Limited Performance Improvement** – When compared to CrystalFramer, the performance gains achieved by the proposed methodology are marginal across both datasets (Table-1/2). This raises concerns about the practical significance and effectiveness of the approach, especially given the added complexity introduced by integrating symmetry-preserving frames.\n\n- Table 2 shows weaker bulk and shear modulus results on the MP dataset. How does the model perform on these properties in the JARVIS dataset?\n\n- **Typo in Equation 1** – There appears to be a typographical error in Equation 1\n\n- Some recent and relevant works in this domain appear to be missing from the related literature discussion. These include:\n\nhttps://www.nature.com/articles/s41524-022-00716-8\n\nhttps://www.nature.com/articles/s41524-022-00841-4\n\nhttps://arxiv.org/abs/2301.05852\n\nhttps://openreview.net/pdf?id=06jLJiUAKX\n\nhttps://arxiv.org/pdf/2306.09375\n\nhttps://ojs.aaai.org/index.php/AAAI/article/view/28748\n\nhttps://arxiv.org/abs/2409.04737", "was_revised": true, "final_justification": "NA", "rebuttal": "We sincerely thank you for your useful comments. Below we will address your questions in detail.\n\n\n****\n\n**$\\bullet$ Response to weakness 1**:   \n\n\nWe sincerely thank you for your comments.\n\nFrame-based methodologies are indeed a general geometric learning approach. Our contribution first lies in identifying and articulating the challenges that arise when directly applying general frame methods to crystalline materials (highly symmetric atomic systems). We then propose a targeted solution to address these challenges, which, to the best of our knowledge, has not been considered in prior work.\n\nMoreover, we provide a theoretical justification demonstrating the superiority of SPFrame over conventional local frame methods (please refer to our response to Reviewer 5W3r’s weaknesses), which further strengthens the theoretical significance of our work.\n\nFinally, SPFrame provides a new insight for both machine learning and materials science, namely, that general-purpose machine learning techniques must undergo specific adaptation to effectively address the unique characteristics of materials science problems.\n\n\nWe hope our response addresses your concerns.\n\n\n\n****\n\n**$\\bullet$ Response to weakness 2**:   \n\nWe sincerely thank you for your valuable comments. Our method primarily focuses on enhancing the performance of the backbone (As demonstrated by the performance gains achieved when applying SPFrame to eComFormer). Using a more powerful backbone is theoretically expected to yield even better results. To address the reviewer’s concerns, we conducted additional experiments by replacing the backbone to achieve more significant improvements over the baseline. Specifically, we used iComFormer [1], which outperforms eComFormer, as the new backbone. Below, we provided the evaluation of JARVIS dataset. Notably, our solution outperforms or at least is on par with the baselines (Crystalframer). The preliminary results are as follows.\n\n\n\n\n### Table: Property prediction results on the JARVIS dataset\n\n| Method                                               | Form. energy (eV/atom) | Total energy (eV/atom) | Bandgap (OPT) (eV) | Bandgap (MBJ) (eV) | E hull (eV) |\n|---------|--------|----------|-----|--------|--------------|\n| Crystalframer      | 0.0263    | 0.0279     | 0.117    | 0.242     | 0.047        |\n| eComFormer       | 0.0284      | 0.0315   | 0.124   | 0.283       | 0.044        |\n| — w/ SO(3)-equivariant Gram-Schmidt local frame      | 0.0285     | 0.0296     | 0.115| 0.271 | 0.043|\n| **— w/ Quaternion-based SPFrame (ours)**  | 0.0261  | 0.0276 | 0.107  | **0.239**           | **0.042**    |\n| iComFormer          | 0.0272     | 0.0288   | 0.122   | 0.261  | 0.047  |\n| — w/ SO(3)-equivariant Gram-Schmidt local frame      | runing    |runing       | 0.112     |  0.255     | 0.045        |\n| **— w/ Quaternion-based SPFrame (ours)**        | **0.0250**  | **0.0259** | **0.106** | 0.251               | **0.042**    |\n\n\n\nWe hope our response addresses your concerns.\n\n\n[1] Complete and efficient graph transformers for crystal material property prediction. *ICLR2024*.\n\n\n\n\n****\n\n**$\\bullet$ Response to weakness 3**:   \n\n\nWe sincerely thank you for your valuable comments. We completed preliminary experiments on bulk and shear modulus properties in the JARVIS dataset, as shown below.\n\n\n\n\n### Table: Bulk and shear modulus in the JARVIS dataset prediction results\n\n\n| Method          | Bulk Modulus (Kv) |Shear Modulus (Gv) |\n|---------|--------|-------| \n| Matformer    | 11.21  | 10.76| \n| CrysGNN [1]     | 10.99  | 9.800 | \n| CrysDiff [2]     | 9.875  | 9.193| \n| Crystalframer     | **8.876** | 8.999  | \n| eComFormer      |9.777   | 9.435 |\n| — w/ SO(3)-equivariant Gram-Schmidt local frame      | 9.855  |  9.689| \n| **— w/ Quaternion-based SPFrame (ours)**  | 9.357 | **8.963** | \n\nIt can be observed from the table that SPFrame significantly improves the performance of the backbone method eComFormer, achieving the best prediction result for Shear Modulus (Gv).\n\n\n[1] CrysGNN: Distilling pre-trained knowledge to enhance property prediction for crystalline materials. *AAAI2023*.\n\n[2] A Diffusion-Based Pre-training Framework for Crystal Property Prediction. *AAAI2024*.\n\n\n\n\n\n****\n\n**$\\bullet$ Response to weakness 4**:  \n\n\nThank you for your careful review. We will revise these typos. Equation 1 was mistakenly written as an expression describing equivariance; we will correct it to properly describe invariance.\n\n\n\n\n\n****\n\n**$\\bullet$ Response to weakness 5**:  \n\n\n\nWe sincerely appreciate your useful comments. We will include a discussion below of these related works in our paper.\n\nIn crystal property prediction tasks, in addition to frame-based methods (which can also be regarded as a form of representation learning), pretraining [1,3,6] and representation learning [2,4,5,7] are two other important approaches for improving prediction accuracy.\n\nPretraining methods mainly focus on pretraining the backbone network architecture. CrysXPP [1] designs an autoencoder, CrysAE, to perform self-supervised pretraining, thereby capturing essential structural and chemical features from large amounts of unlabeled crystal graph data and reducing prediction errors. CrysGNN is a dedicated pretrained GNN framework that, in addition to common self-supervised strategies such as feature reconstruction used in CrysXPP, incorporates connectivity reconstruction and contrastive learning over different crystal systems, which further enhances prediction performance. CrysDiff [6], on the other hand, is a diffusion-based pretraining method. During the pretraining phase, CrysDiff performs a crystal structure reconstruction task based on a diffusion model to learn the underlying edge distribution of crystal structures. In the fine-tuning phase, the model generates the target property values under the guidance of the structural data.\n\nUnlike pretraining, representation learning primarily focuses on constructing more effective representations of crystal structures. Beyond the commonly used bond angle information to encode directional features, ALIGNN-d [2] incorporates dihedral angles into the crystal representation. This simple extension leads to a memory-efficient graph representation that captures the complete geometry of atomic structures.\nCrysMMNet [4] leverages textual descriptions of materials to encode global structural information into the graph structure, enabling the learning of a more robust and enriched representation of crystalline materials. Geom3D [5] is a benchmarking study on geometric representations, systematically evaluating the effectiveness of various geometric encoding strategies such as spherical harmonics basis, frame-based basis, and angle-based features.\nCrysAtom [7] uses unlabeled crystal data in an unsupervised manner to learn a distributed representation of atoms, which significantly improves the performance of various property prediction models.\n\n\nWe hope our response addresses the reviewer’s concerns.\n\n\n[1] CrysXPP: An explainable property predictor for crystalline materials. *npj Computational Materials, 2022*.\n\n[2] Efficient and interpretable graph network representation for angle-dependent properties applied to optical spectroscopy. *npj Computational Materials, 2022*.\n\n[3] CrysGNN: Distilling pre-trained knowledge to enhance property prediction for crystalline materials. *AAAI2023*.\n\n[4] CrysMMNet: Multimodal Representation for Crystal Property Prediction. *UAI2023*.\n\n[5] Symmetry-informed geometric representation for molecules, proteins, and crystalline materials. *NeurIPS2023*.\n\n[6] A Diffusion-Based Pre-training Framework for Crystal Property Prediction. *AAAI2024*.\n\n[7] CrysAtom: Distributed Representation of Atoms for Crystal Property Prediction. *LoG2024*."}, {"review_id": "QZj7zHihIS", "reviewer": "Reviewer_nQBH", "summary": "The paper highlights and aims to address the issues with existing global and local frames for modelling crystal structures by proposing a new symmetry-preserving frame (SPFrame) which combines local invariant and global equivariant frames. Next, the authors show that the results improve for the materials property prediction task when SPFrame is used with a state-of-the-art architecture.", "questions": "Please refer to the Strengths and Weaknesses section.", "limitations": "Please refer to the Strengths and Weaknesses section.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strength**:\n- The paper highlights and addresses an important problem in crystal modelling. The paper is structured and written.\n- The problem with the current methods is discussed in detail, and suitable examples are provided to support the claims, particularly in the case of existing local frames.\n- The proposed idea is straightforward to integrate with existing works, making it easy to implement the modifications.\n- The authors have benchmarked their method on two datasets, JARVIS and MP, showing that their method is either better or slightly worse than the state-of-the-art. Overall, when a baseline model is compared with the baseline $+$ SPFrame, there are notable improvements.\n\n**Weaknesses**:\n- I don't fully understand Figure 3. It seems that the behaviour exhibited in both SPFrame and equivariant local frame is similar. Can you point out the exact differences with the help of the colours present in the figure?\n- Can you discuss why the quaternion-derived SPFrame provide a better representation than the Gram-Schmidt-based SPFrame? Are there other ways through which $F_{INV, i}$ could be designed, and if so, are there ablations on them?\n- Can the proposed SPFrame be used for other tasks related to crystals, such as crystal structure prediction or materials generation?\n- Discussion on related works (DiffCSP++ [1], SymmCD [2], WyckoffTransformer [3], WyckoffDiff [4], etc.), which use the crystal symmetry in the architecture design, is missing. Can the authors compare and contrast these ideas with their proposed SPFrame?\n- Can you show improvements with SPFrame for another SOTA model? Currently, it is only demonstrated with eComFormer.\n\nReferences:\n1. Space Group Constrained Crystal Generation. Jiao et al., ICLR 2024. \n2. SymmCD: Symmetry-Preserving Crystal Generation with Diffusion Models. Levy et al., ICLR 2025.\n3. Wyckoff Transformer: Generation of Symmetric Crystals. Kazeev et al., arXiv 2025.\n4. WyckoffDiff-A Generative Diffusion Model for Crystal Symmetry. Kelvinius et al., ICML 2025.", "was_revised": true, "final_justification": "I recommend acceptance of this paper for its contribution to improving crystal modelling by enhancing frame creation to handle crystal symmetry.", "rebuttal": "We sincerely thank you for your constructive suggestions. Below we will address your questions in detail.\n\n****\n\n**$\\bullet$ Response to weakness 1**:   \n\nThank you for your comments.\n\nIn the figure, edges of different colors represent connections between atoms of different types (see the lower left corner of Figure 3). Please note the red and green edges in Figure 3, which are highlighted with glowing effects for better visual distinction. It can be observed that applying the local frame leads to identical local structures (the red and green edges of two different atoms point in the same direction). In contrast, applying SPFrame preserves the differences in local structures (the red and green edges of the two atoms point in different directions). This structural identity in the local frame causes the node/atom features to become identical after message passing, resulting in information loss. For more details, please refer to our response to Reviewer 5W3r's weaknesses.\n\nWe hope our response addresses the reviewer’s question.\n\n****\n\n**$\\bullet$ Response to weakness 2**:   \n\nWe sincerely thank you for your insightful comments. Due to the highly symmetric nature of crystal structures, using the Gram-Schmidt orthogonalization-based rotation matrix construction method (detailed in Appendix A.4) may result in two invariant vectors being nearly identical, leading to redundancy and duplication of information. This can increase the difficulty of network training and hinder performance improvements. In contrast, quaternion-based methods do not suffer from this limitation. \n\nIn addition to Gram-Schmidt and quaternion, Ref. [1] proposes a rotation matrix construction method based on singular value decomposition (SVD). \nWe conducted supplementary experiments using this method to construct $F_{\\text{inv}}$ on the JARVIS dataset, and the results are shown below.\n\n### Table: Property prediction results on the JARVIS dataset\n\n| Method   | Form. energy (eV/atom) | Total energy (eV/atom) | Bandgap (OPT) (eV) |Bandgap (MBJ) (eV) | E hull (eV) |\n|-----|-----|------|-----|-----|----|\n| eComFormer | 0.0284  | 0.0315 | 0.124|0.283 |0.044|\n| — w/ Gram-Schmidt-based SPFrame (ours)| 0.0268  | 0.0281   | 0.109 | 0.259 |  0.043 |\n| — w/ Quaternion-based SPFrame (ours)  | **0.0261**  | **0.0276** | **0.107** |  **0.239** |**0.042** |\n| — w/ SVD-based SPFrame (ours) | 0.0266 | 0.0281| **0.107**  | 0.258 | **0.042** |\n\nCompared to the quaternion-based approach, the SVD-based method did not yield significant improvements. In [1], the SVD-based method demonstrated better performance on scalar regression tasks (e.g., Experiment 4, where an MLP is used to predict scalar values). However, since our task is more complex—mapping from graph structures to scalar properties, this may account for the performance difference between the quaternion-based and SVD-based methods. A more detailed investigation will be left for future work.\n\n\nWe hope our response addresses the reviewer’s question.\n\n[1] Learning with 3D rotations, a hitchhiker’s guide to SO(3). *ICML2024*.\n\n****\n\n**$\\bullet$ Response to weakness 3**:   \n\nWe sincerely thank you for your constructive comments.\n\nPreviously, DiffCSP++ [1] applied the frame method to crystal structure prediction and materials generation, where frames were used to construct invariant representations of crystals, thereby removing the requirement for equivariance in the subsequent denoising model. Our method can serve a similar purpose. Since our approach is learnable frames, a more recent concurrent work - RADM [2] (for molecular generation) offers an insightful application strategy. RADM employs frames to eliminate the need for equivariance in the denoising model, allowing the use of a more advanced non-equivariant diffusion transformer (DiT) as the denoising model to enhance generation quality. Additionally, it leverages an autoencoder to learn richer feature representations in the latent space.\n\nSimilarly, our SPFrame can be applied in crystal generation. During autoencoder training, SPFrame can preserve crystal symmetries and enable the learning of more expressive latent features. Moreover, by removing the equivariance constraint from the denoising model, SPFrame allows the adoption of powerful non-equivariant DiT to further improve generation performance.\n\nWe hope our response addresses the reviewer’s question.\n\n\n[1] Space Group Constrained Crystal Generation. *ICLR2024*.\n\n[2] Scalable Non-Equivariant 3D Molecule Generation via Rotational Alignment. *ICML2025*.\n\n****\n\n**$\\bullet$ Response to weakness 4**:   \n\nWe sincerely thank you for your constructive comments.\n\nFirst, we discuss how these generation methods leverage crystal symmetry during the generation process. The core idea of these approaches is to simplify the data to be generated by exploiting the crystal symmetry (For example, by generating only the asymmetric unit and then reconstructing the full crystal structure using symmetry operations).\n\nDiffCSP++ [1]: Due to crystal symmetry, the elements of the lattice matrix in crystals belonging to different space groups are subject to specific constraints, as described in Table 1 of [1]. Therefore, when generating the lattice matrix, DiffCSP++ only needs to generate the elements that are not restricted by the crystal symmetry, thereby simplifying the generation process.\nFor the generation of atomic fractional coordinates and types, crystal symmetry allows further simplification. Since atoms at symmetrically equivalent positions can be derived from a single representative atom, it is sufficient to generate only one such atom, and the rest can be obtained through symmetry operations.\nDiffCSP++ separately and simultaneously generates the lattice, fractional coordinates, and atomic composition under the reduced form of the space group constraint. The generated crystals strictly satisfy the space group constraint.\n\nSymmCD [2]: SymmCD only generates the asymmetric unit and, during the generation process, it produces only the unit parameters of the asymmetric unit rather than the full lattice matrix (as illustrated in Figure 1 of [2]). This approach differs from that of DiffCSP++, which focuses on generating the full lattice matrix and uses predefined structural templates to enforce space group constraints. SymmCD also provides experimental evidence suggesting that this approach in DiffCSP++ (using predefined structural templates) may limit the diversity and novelty of the generated samples.\n\nWyckoff Transformer [3]: Wyckoff Transformer is an autoregressive generation method, distinct from diffusion-based methods [1,2,4]. For atoms located at symmetric positions, it generates only discrete data such as the space group, atomic element, site symmetry, and enumeration, rather than the explicit atomic coordinates or the crystal lattice matrix. The final atomic structure is obtained by combining generated discrete information with an energy relaxation method.\n\n\nWyckoffDiff [4]: WyckoffDiff, given a space group and Wyckoff positions, directly predicts the probability of the atom type occupying the corresponding position, as shown in Figure 1 of [4]. This approach bears some similarity to [5], where the probability of an element’s occurrence is predicted given the periodic table. However, WyckoffDiff incorporates symmetry constraints into the structure generation process.\n\nOur method is applied to scalar property prediction of crystals, which differs from the use cases of the aforementioned generative methods. Our approach primarily addresses the limitation of local frames, which cause atoms at symmetry-equivalent positions to have identical local structures. This structural uniformity leads to identical node features and consequently results in information loss (see our response to Reviewer 5W3r's Weaknesses). Therefore, when designing the frame, we adopt an associative approach that combines an invariant local frame with an equivariant global frame shared across the atomic system. This design prevents the destruction of crystal symmetry and ensures that atoms at symmetry-equivalent positions retain their symmetric local environments, thereby enabling their node features to remain distinguishable.\n\n[1] Space Group Constrained Crystal Generation. *ICLR2024*.\n\n[2] SymmCD: Symmetry-Preserving Crystal Generation with Diffusion Models. *ICLR2025*.\n\n[3] Wyckoff Transformer: Generation of Symmetric Crystals. *ICML2025*.\n\n[4] WyckoffDiff -- A Generative Diffusion Model for Crystal Symmetry. *ICML2025*.\n\n[5] Scalable Diffusion for Materials Generation. *ICLR2024*.\n\nWe will include a discussion above of these works in our paper. \nWe hope our response addresses the reviewer’s question.\n\n****\n\n**$\\bullet$ Response to weakness 5**:   \n\n\nWe sincerely thank you for your valuable comments.\n\nWe conducted experiments using iComFormer [1] as the backbone, which outperforms eComFormer. Below, we provided the evaluation of JARVIS dataset. Notably, our solution outperforms or at least is on par with the baselines (Crystalframer). The preliminary results are as follows.\n\n### Table: Property prediction results on the JARVIS dataset\n| Method                                               | Form. energy (eV/atom) | Total energy (eV/atom) | Bandgap (OPT) (eV) | Bandgap (MBJ) (eV) | E hull (eV) |\n|-----|------|-----|-----|-------|------|\n| Crystalframer| 0.0263    | 0.0279| 0.117| 0.242 | 0.047|\n| eComFormer | 0.0284  | 0.0315  | 0.124 | 0.283 | 0.044|\n| — w/ SO(3)-equivariant Gram-Schmidt local frame  | 0.0285 | 0.0296| 0.115| 0.271 | 0.043|\n| **— w/ Quaternion-based SPFrame (ours)**  | 0.0261  | 0.0276 | 0.107  | **0.239** | **0.042**  |\n| iComFormer  | 0.0272 | 0.0288  | 0.122 | 0.261  | 0.047  |\n| — w/ SO(3)-equivariant Gram-Schmidt local frame| runing |runing   | 0.112  |  0.255   | 0.045  |\n| **— w/ Quaternion-based SPFrame (ours)**   | **0.0250**  | **0.0259** | **0.106** | 0.251  | **0.042**  |\n\nWe hope our response addresses your concerns."}]

Author rebuttal:

Meta-review: The paper introduces a methodology for crystal property prediction based on local and global frames. The proposed methodology constructs invariant local frames as well as an equivariant global frame to ensure SO(3) invariance. The paper has merits that have been identified by the reviewers, including the importance of the problem addressed and the fact that SPFrame can be easily integrated with existing models. During the discussion period, the authors provided clarifications and addressed several of the reviewers' concerns. Overall, the contribution is well motivated, practically useful, and of interest to the community of crystal structure modeling.
Decision: Accept (poster)