TetraGT: Tetrahedral Geometry-Driven Explicit Token Interactions with Graph Transformer for Molecular Representation Learning
Authors: Jinjia Feng
Zhewei Wei
Taifeng Wang
Zongyang Qiu
Venue: ICLR 2026 Poster (2026)
Keywords: Molecular Representation Learning
Graph Transformer
Molecular Geometry Pretraining
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: The Tetrahedral Graph Transformer (TetraGT) directly models molecular geometric tokens with effective interactions through tetrahedral inequalities, enabling enhanced molecular representations and superior accuracy in molecular property prediction.

Abstract: Molecular representations that fully capture geometric parameters such as bond angles and torsion angles are crucial for accurately predicting important molecular properties including enzyme catalytic activity, drug bioactivity, and molecular spectral characteristics, as demonstrated by extensive studies.
However, current molecular graph representation learning approaches represent molecular geometric parameters only indirectly through combinations of atoms and bonds, neglecting the spatial relationships and interactions between these higher-order geometric structures.
In this paper, we propose \textbf{TetraGT} (\textbf{Tetra}hedral \textbf{G}eometry-Driven Explicit \textbf{T}oken Interactions with Graph Transformer), a novel architecture that directly models molecular geometric parameters.
Based on the spatial solid geometry theory of face angle and dihedral angle inequality, TetraGT explicitly represents bond angles and torsion angles as structured tokens for the first time, directly reflecting their intrinsic role in determining the molecular conformational stability and properties. 
Through our designed spatial tetrahedral attention mechanism, TetraGT achieves highly selective direct communication between structural tokens.
Experimental results demonstrate that TetraGT achieves superior performance on the PCQM4Mv2 and OC20 IS2RE benchmarks. 
We also apply our pre-trained TetraGT model to downstream tasks including QM9, PDBBind, Peptides and LIT-PCBA, demonstrating that TetraGT delivers excellent results in transfer learning scenarios and shows scalability with increasing molecular size.

Published as a conference paper at ICLR 2026
TETRAGT: TETRAHEDRAL GEOMETRY-DRIVEN EXPLICIT TOKEN INTERACTIONS WITH GRAPH TRANSFORMER FOR MOLECULAR REPRESENTATION LEARNING
Jinjia Feng' Zhewei Wei!* Taifeng Wang?
Zongyang Qiu?
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China 2BioMap Research, Beijing, China
fenng, zhewe ) @ruc. prruc.edu edu. en, tfwangster&gmail.. com, riuzongyang23euutloo com
ABSTRACT
Molecular representations that fully capture geometric parameters such as bond angles and torsion angles are crucial for accurately predicting important molecular properties including enzyme catalytic activity, drug bioactivity, and molecular spectral characteristics, as demonstrated by extensive studies. However, current molecular graph representation learning approaches represent molecular geometric parameters only indirectly through combinations of atoms and bonds, neglecting the spatial relationships and interactions between these higher-order geometric structures. In this paper, propose TetraGT (Tetrahedral Explicit Token Interactions with Graph Transformer). a novel architecture that di- rectly models molecular geometric parameters. Based the spatial solid geometry theory of face angle and dihedral angle inequality, TetraGT explicitly represents bond angles and torsion angles as structured tokens for the first time, directly reflecting their intrinsic role in determining the molecular conformational stability and properties. Through our designed spatial tetrahedral attention mechanism, TetraGT achieves highly selective direct communication between structural tokens. Experimental results demonstrate that TetraGT achieves superior performance on the PCQM4Mv2 and OC20 IS2RE benchmarks. We also apply our pre-trained TetraGi model to downstream tasks including QM9, PDBBind, Peptides and LIT-PCBA, demonstrating that TetraGT delivers excellent results in transfer learn- ing scenarios and shows scalability with increasing molecular size. Our code is available at htthtps: /tttps://githuun www.auuw.ooooooooooooaae..eeeeee
Setometty-Driveer
INTRODUCTION
Accurate prediction of molecular properties is essential for advancements in drug discovery and biochemical research. Properties such as enzyme catalytic activity (Lopes et al. 2009). drug bioactiv- ity (Kessler 1982). molecular spectral characteristies (Wald) 1968), and reaction stereoselectivity (Hashimoto & Maruoka 2015) are intrinsically determined by the ftree-dimensional structural arrangements of molecules. Studies have consistently shown that molecular geometric parame- ters particulllll bond angles and torsion angles- critically
crrtia-cy influence these properties by determining molecular conformational stability and reactivity (Lovering et al. 2009).
Following the success of Transformer /Vaswami 2017) in various domains, Graph Transformers (GTs) (Ying et al., 2021: Feng et al. (022) have emerged as powerful approaches learning molecular representation. Recent GT's (Zhou et al. 2023, Stärk et al.) 2022) Hussain et al.) 2022). which incorporate 3D structural information, represent molecules primarily through node tokens (atoms) and sometimes edge tokens (bonds), relying global attention mechanisms to facilitate information
"Zheeei Wei is the corresponding author.

---

Published as a conference paper at ICLR 2026
exchange across the molecular graph. Inspired by the remarkable success of AlphaFold (Jumper et al. 2021) in protein structure prediction, methods such as UniMol+ (Lu et al. 2023) and TGT (Hussain et al. 2024) have introduced triangle inequality coonuaiityycotttrinned interatomic distance prediction. Such advancements have demonstrated that the incorporation of geometric constraints can significantly enhance the prediction performance of molecular properties.
Despite these improvements, current approaches encounter three primary challenges in representing molecular geometry comprehensively. (1) Lack of Local Chirality: Chirality describes the "handed- ness" of molecular spatial arrangements. Molecules with different local chirality may yield similar or even identical distance matrices, creating ambiguity in molecular representation. (2) Implicit Modeling of Geometric Structures: Recent models such as QuinNet (Wang et al., 2024c) and
ViSNet [Wang et al. 2024b) that introduce four- or five-atom interactions to enhance expressiveness, still encode higher-order geometric information only implicitly through combinatorial operations between atom-level tokens, causing deviations geometric parameters to propagate and accumulate through indirect representations. (3) Structural Interdependency Neglect: Existing approaches fail to account for the interdependent relationships between molecular geometric parameters, relations that collectively determine the overall molecular conformation and subsequently its functional properties.
To address these challenges, we propose TetraGT, a novel Graph Transformer architecture that explicitly models geometric parameters structured tokens. TetraGT directly represents bond angles and torsion angles as structural tokens the model architecture, reducing the accumulation of prediction errors. TetraGT introduces spatial tetrahedral attention mechanism that promotes effective communication between geometric-paranne parameeter tokens while ensuring the satisfaction of tetrahedral geometric constraints, enabling the generation of globally consistent and physically realistic molecular conformations. In addition, we introduce a directed cycle angle loss that enables stable prediction of angles over the full range (0, 2 and leverages directionality for explicit local chirality discrimination. Furthermore, based on hierarchical virtual node aggregation architecture, our method effectively alleviates information bottlenecks caused by excessive compression of information from different orders. By learning fundamental geometric principles rather than memorizing specific conformational patterns, TetraGT improves the model's generalization ability across diverse molecular structures and can accurately predict geometries from scratch, without requiring initial estimates of 3D coordinates. Our main contributions are summarized as follows:
To the best of our knowledge, we the first to explicitly model molecular geometric parameters (bond angles and torsion angles) as structured tokens rather than indirectly deriving them from pairwise atomic relationships. This direct representation significantly reduces accumulated errors and enables more accurate capture of molecular conformational characteristics.
We introduce three synergistic architectural imoovttioss-spaleee tetrahedral attention mechanism, directed cycle angle loss, and hierarchical virtual node aggregation rrchitecture-thee collec- tively enhance TetraGT's expressive power by enforcing geometric consistency, enabling chirality discrimination, and capturing multi-scale structural information.
TetraGT achieves superior performance across multiple benchmarks, establishing new state-of- the-art results on quantum chemistry datasets (PCQM4Mv2 and OC20 IS2RE) and demonstrating remarkable effectiveness in diverse transfer learning scenarios including molecular property predic- tion (QM9 and Peptides), binding affinity prediction (PDBBind), and drug discovery (L.T-PCBA).
RELATED WORK
2.1 ANGLE PREDICTION MOLECULAR CONFORMATION OPTIMIZATION
The incorporation of angular constraints, including bond angles and torsion angles, in molecular conformations has been progressively applied in recent works. Early methods (Ganea et al., 202T] Rai et al. 2022) introduced torsion angle constraints in miree-dimensional conformation genera- tion. Building on these foundations, several diffusion-based and autoregressive approaches have emerged (Jing et al. 2022; Zhang et al. 2023). For example, DiffPack (Zhang et al. 2024) learned the joint distribution of side chain torsion angles through diffusion and denoising and AUTODIFF (Li et al.) 2024a) designed conformational motifs molecular assembly strategy to mitigate issues with skewed angles. Other methods attempt to capture angular information more explicitly through equivariance and positional encoding that incorporates angular information, such as LEFTNet (Du
2

---

Published as a conference paper at ICLR 2026
et al. 2024). Va (Aykent & Xia 2024) and Geoformer (Wang et al. 2024a). However, these methods still predict angles implicitl from atomic coordinates or encode them as auxiliary features rather than modeling them explicit structural entities, leading accumulated errors as deviations compound through indirect representations. More critically, none of these works address the funda- with different chirality can produce similar angle and distance distributions, creating inherent ambiguity (Lin et al. (026). Our approach fundamentally differs by treating geometric parameters as structured tokens, with bond angles and torsion angles represented as first-class entities. We introduce a directed cycle angle loss that predicts angles in the full range of (0, 2T). incorporating directionality to explicitly capture local chirality, making TetraGT the first work to achieve chirality-aware molecular representation through angular modeling.
mental challenge of local chirality cetriiiintiioomollees
2.2 PREDICTIVE MOLECULAR STRUCTURAL PRE-TRAINING
AlphaFold (Jumper ct al. 202 employs Transformer architecture for predictive structural pre- training on protein datasets. small molecule structural pre-training, hybrid approaches such GraphTrans (Wu al.
2021). GSA (Rashedi al., 2009), GROVER (Rong al. (2020), and GPS (Rampášek et al.
2022a) combine Transformers with Graph Neural Networks for en- hanced expressiveness. Pure GT models directly take structural elements as token inputs, where Graphormer (Ying et al. 2021 Shi et al. 2022) and EGT (Hussain et al. 2022) are the two most rep- resentative approaches. Graphormer-type models including Unimol (Zhou et al. 2023), GEM-2 (Liu et al. 2022a), and Transformer-M (Luo et al. 2022) primarily use atoms as tokens, implicitly encoding bonds and spatial structures through positional encoding and attention bias. In contrast, EGT-based models treat edge embeddings as tokens with global attention for node-edge information exchange. Recent advances have incorporated triangular inequality constraints, as seen in GPS++ (Masters)
et al. 2022). Unimol+ (Lu et al.] 2023). and TGT (Hussain et al., [024), improving geometric consistency at the edge level. Despite these advances, current methods remain limited to the inter- actions between node-level and edge-level tokens, failing to capture structural interdependencies among higher-order geometric structures (bond angles and torsion angles). This results in isolated modeling of bond angles and torsion angles, significantly compromising the physical plausibility of conformations. Building upon explicit modeling of angles as tokens, TetraGiT introduces a spatial tetrahedral attention mechanism that ensures these parameters collectively satisfy inequalities derived from tetrahedral geometry, enabling effective communication between interdependent struc- tures- extendxn triangle inequality principles from edges higher-order elements. Meanwhile, our hierarchical virtual node architecture aggregates information across different structural orders, allowing the model learn multi-scal interdependencies.
3 METHOD
3.1 TETRAGT ARCHITECTURE
The Definition and Preliminaries
where The TetraGT model utilizes atomic features (X E Raxd=, where n is the number of atoms and dz is the atom feature dimension), edge features (E <Raxnxd, where de is the edge feature dimension), and 3D conformational information including the complete distance matrix (D E TR=x"), all bond angles (B "R" n, is the number of bond angles), and torsion angles (T E RR*,, n, is the number of torsion angles) within the molecule to predict molecular properties y and update 3D conformational information using learnable parameters 0. The model has L blocks, where the 1-th block outputs node embeddings h (9), edge embeddings (!)), bond angle embeddings (()), and dihedral angle embeddings {()) To ensure the physical validity of predicted molecular geometries, we consider the tetrahedron as a fundamental local structure that satisfies basic geometric constraints governing the spatial arrangement of atoms. Here, "tetrahedra" refers the geometric 3D simplex formed by any four non-coplanar atoms, rather than a chemical tetrahedral (sp3) center.
Specifically, such tetrahedra must satisfy the following angle constraints:
Lemma (Tetrahedral Angle Constraints). Let ji.j, k, 1} be four non-cot coplanar points in IR3 forming (non-degenerate)) tetrahedron. In particular, all face angles and dihedral angles introduced below
3

---

Published as a conference paper at ICLR 2026
Projection
b NOM
(LD
byas
Disi
Soflaus
Indox Marii
Expand
Edge PFN
X. Jayers
Angle FFN
Tomsiun FFN
10.0
bys
Toppa Intention
Tarahulni Intemaiim
C
Intemation
Scale
(c)
Dus
Milti-lvel Amsotinn
TeinG Layer &
9
Layer Norm
b/
Egu
16.0
(a) Multi-level Attention Module
(b) TetraGT Network Architecture
(d) Figure I: (a) Multi-level Attention Module. Index Match denotes the selection of corresponding edge embeddings based on the indices of nodes. (b) TetraGT Network Architecture. (c) Tetrahedral interaction for bond angles. (d) Tetrahedral interaction for torsion angles.
lie in (0, 2T). Let byks. bisi, biky denote the face angles at vertex k in faces ijk, ikl,jje respectively, and let t taki denote the dihedral angles between faces ijk and ikl, faces ijk and jkl, and faces ikl and jkl respectively. Then:
tisks,
(a) Face Angles: For any permutation {01, 02 0z} {byker brss buih. we have 01 + 02 0g and 0g < 2n.
01 + 02
(b) Dihedral Angles: For any permutation {δr, Φ2 Ø3} {layke, tysks, tads h. we have Φ1 + 62 Φs-
(c) Relationship between Face and Dihedral Angles at vertex k:
cos(tişıa) cos(byus) cco(saa) cos(brxj)
sin(bus) sin(bra;)
with similar formulas for jika and tıkl by cyclic permutation of the indices. The non-coplanarity assumption ensures that the denominators the above expressions are non-zero.
The Initialization of Substructures. Atom representations are composed of the atom's 's inherent properties, while edge representations are formed by chemical bond properties, the types atoms at both ends, and bond length. As illustrated in Figure [c) and (d), TetraGT comprehensively models neighboring face angles (such as bikj. bykj. and kyr between atom triplets and dihedral angles (tyga, j. etc.) between atom quadruplets. This approach enables face angles like buy.. by and byrk to aggregate information directly without being bottlenecked by nodes k and j. with the same principle applying to dihedral angles. Inspired by Lemma 1, initialize dihedral angle tokens using atom and edge representations along with face angle information. This approach naturally incorporates tetrahedral geometric constraints into our model.
Multi-level Attention Module. In each layer, the Multi-level Attention mechanism leverages output representations from the previous layer facilitate continued interaction and refinement of these geometric features, First, we compute the node and edge embeddings through the attention mechanism shown in Figure a). Subsequently, the bond angle embedding is obtained by using the indices of the two edges forming the angle to locate the corresponding positions and summing the embeddings. Similarly, for dihedral angle updates, we use the indices of three consecutive edges that form the torsion angle to locate and sum the corresponding torsion angle embeddings. This approach allows for a hierarchical update of representations from atoms to chemical bonds, then to bond angles, and finally to torsion angles. The updates of atom and edge representations in the Multi-level Attention Module are as follows:
different structural levels the graph, progressing
R = softmax (e! o( Ww! yh 360-Dww(lh (iw(l.))
A0-11 (.M) (RR- Dwaa /Van+ + + (2)
where dy is the head dimension, W wilkh) ((())) E xdx,, w(l.e) w Rd,xds, The representation of bond angles and torsion angles is achieved by adding the corresponding edge
(L,h)
g
4

---

Published as a conference paper at ICLR 2026
representations to their respective indices:
Σ (ab) e ((1)), (14)) + bisk - "
(3)
Σ (ab) E (67), (7)), (k0)) figat w.
(4)
where W Rddddd W W(L.) RRdddss Both bond angles and torsion angles utilize the edge representations from the current layer for aggregation, allowing for an efficient use of atomic and edge representations from the previous layer.
Tetrahedral Interaction Module. We carefully designed this approach for two primary reasons. First, naively modeling all possible triplet and quadruplet interactions has unacceptable computational costs, especially when the number of atoms in molecules scales. Second, arbitrary substructures often lack physical significance in molecular modeling. TetraGT leverages tetrahedral geometric theory to selectively model meaningful higher-order interactions in tetrahedral structures. To further improve efficiency, we introduce a local sampling strategy that restricts attention to the w nearest neighbors, reducing theoretical complexity from O(N3) to O(wN2).
For a central bond angle (i, k), the facial interaction with neighboring bond angles sharing vertex is computed as follows:
[b((ba))) - (luss) b' b'(ba))) ogg (brRa)
Ojki
dign (E)
Σ
aiyue softmaxi
(5)
Vd
where (J) denotes the set of w nearest neighbors of j at vertex k, the value vector viky is derived from learnable projection of the bond angle embedding bikj. and olzka is the attention weight assigned to the bond angle j) by the bond angle (i. As shown in Figure [c), design enables efficient and direct information exchange between neighboring face angles like bywi- biss. and buy at vertex k while maintaining computational tractability through strategic local sampling. Meanwhile, the query vector of and key vector p' are derived from the bond angle embeddings. The bias term b' and gating term g' are scalars derived from the bond angle embedding busi incorporating geometric constraints from Lemma ((a) to facilitate physically valid interactions.
Similar to face angles, for a torsion angle (i,j,k, k, (), the dihedral interaction with neighboring torsion angles computed as follows:
((((t1)))) (taus) + +b² (ttras))) (gg(()))))
Σ
Qaska sklys
asyht = softmaxi
(6)
Vd
where No. represents the w nearest neighboring torsion angles, and the value vector Viki is derived from the torsion angle embedding. As illustrated in Figure dd), this mechanism enables efficient direct information exchange between dihedral angles sharing common faces. The fixed atoms (e.g., J, k in Figure d)) naturally form the base bond angle, ensuring that interacting dihedral angles are not composed of completely disconnected atoms. This carefully designed local sampling strategy preserves structural validity while effectively managing the computational complexity inherent to higher-order interactions (detailed computational analysis in Appendix B. Furthermore, the bias and gating terms incorporate constraints from Lemma I (bb), informed by the face-dihedne relationship established in Lemma 1(c), maintain geometrically consistent representations.
Following these specialized attention mechanisms, the representations are updated as:
b" (1) (1-1) FFN(O,
(()) = (1-1) FFN(O))))
(7)
Directed Cycle Angle Loss (DCA loss). TetraGi extends molecular geometry prediction to include bond and torsion angles. When chirality changes, least one angle shifts from 2n in a fixed reference direction. Distance-only prediction methods struggle with this distinction, as both angular values satisfy identical distance matrices. This challenge becomes particularly acute at molecular terminals, where chirality-indued Previous approaches often restricted angles the 0-и range, failing to capture chiral variations. Recognizing the cyclic nature of angle prediction, TetraGT implements a directed circular binning loss:
distance variations become nearly imperceptible.
N
LDCA = min
Σ
qs log(p:))
Σ log(P(++ mod n)).
(8)
5

---

Published as a conference paper at ICLR 2026
Pretrain Stage
Pretrain Targel Prop.
Tam
Coord. Noise
Aed
Directed Cycle
TaskPred
Distance Loss
Optional
True 3D Conformer
Conffred (Fous Fandlune)
TaskPred (ge--rraoo)
Molecular
Confored Stage
Finetune Stage
Finelune
Conformer
Graph
Target Prop.
Figure 2: The three stages of TetraGT training.
By extending the angle range to (0,2m) with counterclockwise as the primary direction, this approach accommodates all chirality scenarios and appropriately handles boundary cases, avoiding excessive penalties for angles that are conceptually close but numerically distant (c.g., 359 vs 19).
Hierarchical Virtual Node. Recent studies (Li al. 2024b Xing et al. 2024) show that virtual
nodes in graph data significantly reduce information bottlenecks, but previous molecular property prediction approaches either compressed all atomic information (losing critical structural details) or used only atomic-level virtual nodes (inadequately representing 3D interactions). To address these limitations, TetraGT implements novel hierarchical virtual nodes architecture. Fach substructure type (atoms, edges, bond angles, torsion angles) has dedicated virtual node that interacts with tokens of its own type through appropriate mechanisms: FFN for atoms, triplet interaction for edges, and tetrahedral interaction for bond and torsion angles. For property prediction, molecule-level virtual node connects to these four substructure virtual nodes as the final output representation.
3.2 MODEL TRAINING
The training procedure of TetraGT includes three stages for the molecular property prediction task. First, in the conformation prediction stage, conformation predictor is trained to predict accurate molecular conformations from 2D molecular graphs. During the pretraining stage, task predictor employed to predict molecular properties from the pre-training dataset. This predictor also receives noisy conformational structures as input and denoise conformational structures. In the fine-tuning stage, the frozen, pre-trained conformation predictor and task predictor are fine-tuned on downstream datasets.
Conformer Prediction Stage. We train the TetraGT conformation predictor predict all pairwise interatomic distances, bond angles, and torsion angles within a molecule. The conformation predictor takes 2D molecular graphs as input (Optionally, coordinates) and outputs all pairwise interatomic distances, bond angles, and torsion angles. Angles invariant translation and rotation. Inspired by TGT, we predict binned angles instead of continuous values, as torsion angle structures are typically less stable than chemical bonds and more susceptible rapid changes due molecular energy fluctuations. The TetraGT employs cross-entropy loss for pairwise atomic distances and the Directed Cycle Angle Loss for angles.
initial distance estimate, e.g., from RDKit.
Pre-training Stage. In the pre-training phase, TetraGT trains the TetraGT task predictor on noisy ground truth 3D conformations. This approach ensures that the task predictor is robust to noise in both input distances and angles, enabling it to adapt to approximate conformations output by the conformation predictor, which still contain noise. We maintain predictions for pairwise interatomic distances, bond angles, and torsion angles. We use distance prediction loss and angle prediction loss as auxiliary tasks that encourage different order substructure representations to denoise the 3D structure, combined with the primary tasks from the pre- pre-training dataset, to jointly train TetraGT's task predictor in multi-tass learning framework. Furthermore, TetraGT employs hierarchical substructure virtual nodes for joint prediction in molecular property prediction, facilitating the association between substructures and molecular properties.

---

Published as a conference paper at ICLR 2026
Table 1: LIT-PCBA results. The top 1st and 2nd results highlighted.
Table 2: Results PCOM4My2 valid set.
Model
param.
Finserrntrvi ct al 2022
16. 1M
173.5
Model
Avg. Test ROC-AUCT
GCN
Kipf Welling
20M
137.9
GINEVZ Brossard
1020
116.7
NaiveBayes
GIN-VN All 20T
Ciillee 2017
6.7M
108.3
SVM Heuus 1998
73.4
Deeper
RandomFores Forest (remman 2001
62.0
TakenGT Kim ch Aupa
48.5M
91.0
XGBeost
BGT (Hastoin em 61A4
39.3M
86.9
GRPE
GCN Kipp & Welling
2016
72.3
Grapherrer (Tigg
Shi
GAT Velickovic
75.2
GraphCPP (Cx 20
13.8M
K5.2
FP GNN
Car cl
75.9
GEM-2( 20-4-22-DDIIIT Miiiiie 2022a
32.1M
79.3
FGT Hussain al.
2012
78.9
Transformer (Lao al. 2022
69M
77.2
GEM (Fany al. 2022
78.4
TGT((RDKii) Hussan et 2027
203M
67.1
EGT+RDKi (Hlusan el
2024
81.2
TetraGT-6 layer
60M
69.3
TGT (uussain il. 2024
81.5
layer
TetraTT224 layer
215M
67.1
TemraTT
82.4
TetraGT:24 layent RDKi)
215M
65.9
Table 3: Performance on OC20 IS2RE validation set.
Energy MAE (meV))
EwT (%))
Model
OOD
OOD
OOD
AVG.
OOD
OOD
OOD
AVG.
Ads.
Cal.
Both
Ads.
Cat.
Both
(Schitt 2017
2.65
DimeNetes Gasterger
2020
563.6
712.7
561.2
649.2
621.7
4.25
2.48
4.40
2.56
3.42
GemNet-T Gistergr at 2021
556.1
734.2
565.9
696.4
638.2
4.51
2.24
4.37
2.38
3.38
SphereNet 20226
563.2
668.2
559.0
619.0
602.4
4.56
2.70
4.59
2.70
3.64
GNS (Godwan al.
540.0
650.0
550.0
590.0
582.5
GNS-NN (L (Goowil
470.0
510.0
480.0
460.0
480.0
Graphorme-3DD
432.9
0
444.1
498.0
EquiFormer Liso Smate
422,2
542.0
423.1
475.4
465.7
7.23
3.77
7.13
4.10
5.56
EguiFormer+Ni Liao
Smidl
415.6
497.6
416.5
434.4
441.0
7.47
4.64
7.19
4.84
6.04
DRFomme WWing
418.7
486.3
432.1
433.2
442.5
8.39
5.42
8.12
5.44
6.84
Uni Mol+ La el
2023
379.5
452.6
401.1
402.1
408.8
11.1
6.71
9.90
6.68
8.61
381.3
TetraT
375.3
440.1
382.8
392.7
397.7
11.9
7.28
11.69
6.90
9.14
Fine-tune Stage. In the fine-tuning phase, TetraGT employs frozen, pre-trained conformation predictor to efficiently generate sigcipreeisoon 3D structural features from the 2D molecule graph. During this process, the conformation predictor specifically operates in stochastic mode with active dropout (Hussain al., 2024). Subsequently, all the predicted bond angles, torsion angles, and interatomic pair distances serve input to the task predictor. The fine-tuning process simultaneously combines the primary objective of the downstream dataset's task with auxiliary optimization functions for distance and angle prediction. Specifically, distance matrix, bond angles, and torsion angles as input, requiring the model to predict the same substructures in the ground truth conformations, well the target objectives of the current dataset.
utilize the model-generatee
interatomic pair
4 EXPERIMENTS
The experimental section aims to validate the effectiveness of our proposed model and methods in addressing existing challenges. We first demonstrate the performance and scalability of TetraGiT on large-scale quantum chemistry datasets, PCQM4Mv2 (Hu et al.) (022) and OC20 IS2RE (Chanussot) ct al., 2021). We then evaluate the transfer learning capabilities of the TetraGT model both the conformer prediction and pre-training stages. We conduct ablation studies on the key components and aggregated angle representation of TetraGT, and analyze its efficiency and scalability across different molecular sizes. We also conducted quantitative analysis and visualization of conformer chirality prediction and conformer prediction accuracy, with details provided in Appendix and D Full experimental details and configurations can be found in Appendix F
7

---

Published as a conference paper at ICLR 2026
Table 4: Results (MAE()) on the QM9 dataset.
Method
/
[
G
Ar
7PVE:
C.
Un
U
1f
R²
GruphMVPP 1 Liu et al.
0.031
0.070
28.5
26.3
46.9
1.63
0.033
GEM (Pang 2072
0.034
0.081
33.8
27.7
52.1
1.73
0.035
3D Intomax (Stakk ala 2022
0.034
0.075
29.8
25.7
48.8
1.67
0.033
3D-MGP Jiao
0.020
0.057
21.3
18.2
37.1
1.38
0.026
Schnet
Schöir ct
2017
0.033
0.235
41.0
34.0
63.0
1.7
0.033
14
19
14
73
PhyxNe1 Like Metwly
2019
0.053
0.062
32.9
24.7
42.5
1.39
0.028
8.15
8.34
8.42
765
Commerait
Aiddersoo of Sl 2019
0.038
0.085
34.0
38.0
61.0
2.03
0.026
21
961
Net
Custerger
21200
0.030
0.044
24.6
19.5
32.6
1.21
0.023
6.32
6.28
6.53
7.56
331
PaiNN Schölr
0,012
0.045
27.6
20.4
5.83
5.98
7.35
66
RGNN
Salmai of mi M
0.029
0.071
29.0
25.0
48.0
1.55
0.031
11
12
12
106
NoisyNide
0.025
0.052
20.4
18.6
28.6
1.16
0.025
7.30
7.57
7.43
8.30
700
SphereNet
el ml 207440
0.053
18.9
31,1
0.024
6.33
7.78
268
ComEENer Wang mi MUS41
0.025
0.045
23.1
19.8
32.4
1.20
0.024
6.59
6.82
6.86
7.98
2.59
SEGNN
0.023
0.060
42.0
0.031
660
EQGAT Le al 2024
0.011
0.053
20.0
16.0
32.0
2.00
0.024
25
24
382
LEFTNe
2024
0.011
0.039
18:
39
1.19
0.022
5
66
SaVeNet Xyken Xia
0.0085
0.021
SDy33-T Tuchs 2020
0.051
0.142
35.0
33.0
$3.0
0.052
TorchMD-NNr Thokk Do Fabritiis 2022
0.011
0.059
20.3
17.5
36.1
0.026
6.15
6.38
6.16
7.62
33
Equiformer [ 1bo Smicr
0.011
0.046
15.0
14.0
30.0
1.26
0.023
6.59
6.74
6.63
7.63
251
TransformecMT (1.16
0.037
0.041
27.4
75
TGT (Hussain ml 2114
0.025
0.040
9.9
9.7
17.4
1.18
0.020
Equiformery V2
2024
0.010
0.050
13
29
1.47
0.023
6.17
6.49
6.22
7.57
186
EquiformerV 2243K Tar
2024
0.009
0.039
12.2
11.4
24.2
1.21
0.020
4.28
4.24
182
Geoformer (Wang et 21241
0.010
0.040
18.4
15.4
33.8
1.28
0.022
4.43
4.41
4.39
6.13
27.5
TenaG
0.017
0.032
8.7
15.6
1.11
0.019
4,92
5.11
4.36
6.07
35
4.1 LARGE-SCALE QUANTUM CHEMICAL PREDICTION
PCQM4Mv2. PCQM4Mv2, part of the OGB-LSC graph property prediction challenge, contains over 3.7 million molecules. The dataset task is to predict the HOMO-LUMO gap. The performance of the distance predictor is tuned on a random 5% subset of the training data, which we refer to as validation- 3d. Experimental results, expressed as Mean Absolute Error (MAE) in meV, presented in Table 2 We observe that the 24-layer TetraGT model achieves the best performance on the PCQM4Mv2 dataset, surpassing all baseline models and outperforming the previous state-of-the-ar TGT model by 1.1 meV, demonstrating the effectiveness our proposed approach. Notably, even without RDkit conformers input, the 24-layer TetraGiT model relying solely 2D) molecular graphs achieves comparable performance to the previous state-of-the-an TGT that utilized RDkit conformers. The gap between the 12-layer and 24-layer TetraGT suggests that effectively encoding higher-order substructures on graphs requires deeper model architectures and larger model capacities.
Open Catalyst 2020 IS2RE. The Open Catalyst 2020 Challenge aims to predict the adsorption energy of molecules on catalyst surfaces. We conduct experiments on the IS2RE (Initial Structure to Relaxed Energy) task. The IS2RE dataset provides initial Density Functional Theory(DFT) structures of crystals and adsorbates. Following TGT'ss experimental configuration, we crop/sample based on the distance to adsorbate atoms, limiting the number of atoms to a maximum 64. IS2RE task results are presented in Table 3 expressed as MAE (in mcV) and Energy within Threshold (EwT) at 20 Ve The table demonstrates that TetraGT achieves state-of-the-arr (SOTA) performance across all subsets of the IS2RE evaluation dataset, both in terms of absolute values and effective proportion, without significantly increasing computational resources. This fimmly establishes TetraGT as the best-performing direct method on the OC20 IS2RE task.
4.2 TRANSFER LEARNING
Our model learns two complementary types of knowledge on the PCQM4My2 dataset: the conformer predictor acquires geometric information by predicting conformations, and the task predictor learns quantum chemical propertics by predicting the HOMO-LUMO gap. We evaluate how effectively these two forms of knowledge learned by TetraGT transfer to diverse downstream tasks.
3D Downstream Tasks, For tasks with available 3D structural information, we directly fine-tune the task predictor using precise 3D conformational data during inference. On QM9, TetraGT achieves ofate-of-the-art results in 5 out 12 tasks as demonstrated in Table 4 surpassing TGT on all targets and significantly outperforming other models in HOMO (Err). LUMO (Er.), and
8

---

Published as a conference paper at ICLR 2026
Table 5: Results on PDBBind core set (version Table 6: Results Peptides-func and Peptides- 2016). The evaluation metries include Pearson's struct datasets. correlation coefficient (R), Mean Absolute Error (MAE), Root-Mean Squared Error (RMSE), and
Model
Peptides-func
Peptides-strect
Avg. Precision/So)
MAE]
Standard Deviation (SD)
Krouzer
2041
0.2545, auth
Mithol
R1
MARy
RMISRI
SDj
Grap Kotreeee
30226
p4
0,280% soit
Osion
Basteeter
8.
1:5lamm
om
1.90
DRew Compatii 21
MGT Con-Ec PUP-3
2725
68. 17774
71.50 71.500a4
0.2536 2015
0.2453 men
GNN.D
0.736am
Li92
1471amm0
Graph 9: 1 2725
0.244%
DMPNN Tannere BaM
8.7%
1888anm
149
1480mm
GRIT Zain 6:16)
60 88
0 0:2469k
SGCN 1i ROME
8.685aa15
1250aac
1.583g00
1.88p00x
GRED DILLIDI Cell P11
71,33um
0.2155
6.78
1.18
1434ume
TGG
KIP
66.79
0.2485
wace
1.17aos0
1399oams
GPNA Frin
(9.55
0.2454,
SIGN
Tramhranrd bard 2022
0.83
0.940pmm
1.250paa
1207wam
GSSC (innn
2024
70
0 g.3499
Tanki
1.38Apam
TetraGT
72.86.o5
0.3421
Table 7: Distance and angle prediction performance of different angle interaction mechanisms and train- ing times on PCQM4Mv2 validation-3D set.
Table 8: Ablation Study on PCQM4Mv2.
Tetrabe.
Directed
Hisrarch.
Mode Distrbuttion
Interac.
Cycle
Virual
(poisusees PAnsie)
MAEL (mc
Medale
Loss
Node
No
Axial
Full
Tetrahe. Att.
71.0
4th and 5th- Order
Att.
Att.
70.5
1:1
70.2
Dist. Cross-Ent.())
1,204
1.164
1.179
1,125
2:1
1:2
69.5
70.7
Angle Cross (nl.(() Time/Epoch(L)
1.310
1,307
1,231
4:1
68.8
1.00
1.36
1.43
1.12
8:1
70.1
HOMO-LUMO gap (Ae) predictions- that are most closely aligned with our pre-training objectives PCQM4Mv2. On the remaining QM9 properties, TetraGT typically attains second-best or highly competitive performance. This diverse pattern is consistent with the alignment between our pre-training supervision (HOMO-LUMO gap prediction and distance/angle denoising) and the underlying physics of different QM9 properties: energy- and orbital-related quantities benefit most directly, whereas properties that depend more on long-range polarization or global shape only indirectly optimized due to their similarity angle to the pre-training task. A more detailed analysis of these desk-dependent behaviors is provided in Appendix [G.]] The PDBBind (version 2016) (Wang al., (2004) results reveal TetraGT's superior binding affinity prediction capabilities, achieving state-of-the-art performance with R = 0.852 and MAE= 0.909, substantially outperforming other baselines (see T). Similarly impressive results emerge from Peptides-strune struct (Dwivedi al. 2022), where TetraGT achieves the lowest MAE of 0.2421 across 11 structural regression tasks, as detailed in Table 6 These consistent improvements across diverse 3D tasks demonstrate that TetraGT effectively facilitates knowledge transfer beyond the specific pre-training dataset.
2D Downstream Tasks. For datasets lacking 3D coordinates, we employ TetraGT's pre-trained conformer predictor as a frozen feature extractor to provide structural information. Table 6 shows TetraGT achieving 72.86% Average Precision on Peptides-func (Dwivedi al.) (022). a 10-way functional classification task, surpassing all baseline methods by significant margins. The LIT- PCBA TTann-guuynn et al. 2020) drug discovery benchmark further validates this approach, with TetraGT achieving superior average ROC-AUC across 7 protein interaction prediction tasks compared to other pre-trained models, as reported in Table T The consistent improvements across both 2D) datasets indicate that molecules with only 2D representations can also obtain effective and valuable information from the 3D data-prerr conformer predictor of TetraGT. Detailed dataset descriptions and additional experimental analyses are provided in Appendix G
4.3 ABLATION STUDY
the impact of different angle interaction methods on the prediction of the interatomic distance, the prediction of the angle in conformations, and the training time. Table presents an ablation study of our three main optimization designs and the ratio of distance to angle loss for the 12-layer TetraGT model. The results show that all three components positively contribute to performance, with the tetrahedral interaction module providing the most significant gains, while the Directed Cycle Angle Loss and Hierarchical Virtual Nodes further improve optimization stability and multi-level feature aggregation. Lastly, we experimented with different ratios of distance loss to angle loss and found that the model performs best when the ratio is 1:4.

---

Published as a conference paper at ICLR 2026
4.4 EFFICIENCY AND SCALABILITY ANALYSIS
TetraGT's strong performance on PDBBind and Peptides datasets, which contain protein-ligand complexes and peptides with up to hundreds of atoms, already demonstrates the model's capability to handle large molecular systems. To further assess scalability with respect to molecular size, we additionally evaluate TetraGT on the OC20 IS2RE dataset; detailed experimental settings and analysis are provided in Appendix H In terms of empirical efficiency, across PCQM4Mv2 and OC20 IS2RE, 6/12/24-layer TetraGT models consistently achieve lower MAE than UniMol+ and TGT under comparable shorter training and inference times, while its pretraining cost on OC20 is more than three times lower than UniMol+ (33 vs. 112 A100 GPU days). These results indicate that the tetrahedral geometry-driven modules substantially enhance accuracy without incurring prohibitive computational overhead; more fine-grained efficiency analysis are given in Appendix
CONCLUSION
In this work, introduce the TetraGT architecture, which directly models molecular geometric parameters (such as bond angles and torsion angles) and enables effective interactions between higher- order structures through tetrahedral attention, significantly enhancing the accuracy of molecular geometry modeling and local chirality expression. In future research, we plan to investigate dynamic representations of molecular geometric parameters in spatial stereochemistry, enabling more effective and rational geometric constraints for structural predictions.
ACKNOWLEDGMENTS
Part of this work was done during an internship at BioMap. The work was partially done at Gaoling School of Artificial Intelligence, Beijing Key Laboratory of Research on Large Models and Intelligent Governance, Engineering Research Center of Gext-Generation Intelligent Search and Recommendation, MOE, and Pazhou Laboratory (Huangpu), Guangzhou, Guangdong 510555, China. This research was supported in part by National Science and Technology Major Project (2022ZD01 14802), by National Natural Science Foundation of China (No. U2241212, No. 92470123 28, No. 62376275). We also wish to acknowledge the support provided by the fund for building world- class universities (disciplines) of Renmin University of China, by Engincering Research Center of Next-Generation Intelligent Search and Recommendation, Ministry of Education, by Intelligent Social Governance Interdisciplinary Platform, Major Innovation & Planning Interdisciplinary Platform for the "Double-Firs Class" Initiative, Public Policy and Decision-making Research Lab, and Public Computing Cloud, Renmin University of China.
6 ETHICS STATEMENT
All data used in this study are publicly available and do not contain personally identifiable information. The research was conducted accordance with the ethical guidelines for computational research, ensuring that all methodologies and procedures followed appropriate ethical standards. The authors declare that they have no conflicts of interest related this study.
REPRODUCIBILITY STATEMENT
All experimental code will be made publicly available upon paper acceptance. Detailed hyperpa- rameters and experimental configurations are provided in the Appendix F All experiments were conducted using the same hardware configuration described in Appendix
8
LLM USAGE
Large language models (LLMs) were used for refining sentence structure, improving grammatical accuracy, and enhancing the clarity of the manuscript text. A supporting role played by the LLMs in the manuscript's language polishing, but no scientific content, data analysis, or experimental design was generated by the L.Mss.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Brandon Anderson, Truong Son Hy, and Risi Kondor. Cormorant: Covariant molecular neural
networks. Advances in neural information processing systems, 32, 2019.
Simon Axelrod and Rafael Gomez-Bombarelli. Geom, energy-annotated molecular conformations for property prediction and molecular generation. Scientific Data, 18)) 2022.
Sarp Aykent and Tian Xia. Savenet: a scalable vector network for enhanced molecular representation learning. Advances in Neural Information Processing Systems, 36, 2024.
Pedro Ballester and John BO Mitchell. A machine learning approach to predicting protein- binding affinity with applications to molecular docking. Bioinformatics, 26(9):1 169-1175, 2010.
Johannes Brandstetter, Rob Hesselink, Elise van der Pol, Erik Bekkers, and Max Welling. Geometric and physical quantities improve (3) equivariant message passing. In International Conference on Learning Representations, 2022.
Leo Breiman. Random forests. Machine learning, 45:5-32, 2001.
Rémy Brossard, Oriel Frigo, and David Dehaene. Graph convolutions that can finally model local structure. arXiv preprint arXiv:2011.15066 2020.
Shaofei Cai, Liang Li, Xinzhe Han, Jiebo Luo, Zheng-Jun Zha, and Qingming Huang. Automatic relation-aware graph network proliferation. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 10863-10873, 2022.
Lowik Chanussot, Abhishek Das, Siddharth Goyal, Thibaut Lavril, Muhammed Shuaibi, Morgane Riviere, Kevin Tran, Javier Heras- Domingo, Caleb Ho, Weihua Hu, et al. Open catalyst 2020 (oc20) dataset and community challenges. Acs Catalysis, 11(10))6059-600 60762 2021.
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp. 785-794, 2016.
Yun Young Choi, Sun Woo Park, Minho Lee, and Youngho Woo. Topology-informed graph trans- former. arXiv preprint arXiv:2402.02005, 2024.
Yuhui Ding, Antonio Orvieto, Bobby He, and Thomas Hofmann. Recurrent distance filtering for graph representation learning. arXiv preprint arxiv:2312.01553, 2023.
Yuanqi Du, Limei Wang, Dieqiao Feng. Guifeng Wang, Shuiwang Ji, Carla P Gomes, Zhi-Ming Ma, et al. A new perspective on building efficient and expressive 3d equivariant graph neural networks. Advances in Neural Information Processing Systems, 36, 2024.
Vijay Prakash Dwivedi, Ladislav Rampášek, Michael Galkin, Ali Parviz, Guy Wolf, Anh Tuan Luu, and Dominique Beaini. Long range graph benchmark. Advances in Neural Informaiinn Processing 2022.
Systems,
Xiaomin Fang, Lihang Liu, Jieqiong Lei, Donglong He, Shanzhuo Zhang, Jingho Zhou, Fan Wang, Hua Wu, and Haifeng Wang. Geometry-enhanced molecular representation learning for property prediction. Nature Machine Intelligence, 4(2): 2022.
Jinjia Feng, Zhen Wang, Yaliang Li, Bolin Ding, Zhewei Wei, and Hongteng Xu. Mgmae: molecular representation learning by reconstructing heterogencous graphs with high mask ratio. In Pro- ceedings of the 31st ACM International Conference on Information Knowledge Management, pp. 509 519, 2022.
Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling, Se trastffsfform:::: 3d roto- translation equivariant attention networks. Advances in neural information processing systems, 33: 1970-1981, 2020.
Octavian Ganea, Lagnajit Pattanaik, Connor Coley, Regina Barzilay, Klavs Jensen, William Green, and Tommi Jaakkola. Geomol: Torsional geometric generation of molecular 3d conformer ensembles. Advances in Neural Information Processing Systems, 34:13757--1766, 2021.
11

---

Published as a conference paper at ICLR 2026
Johannes Gasteiger, Shankari Giri, Johannes T Margraf, and Stephan Günnemann. Fast and uncertainty-awalt aware directional message passing for non-equilibrium molecules. arXiv preprint mrXfv:2011.14115. 2020.
Johannes Gasteiger, Florian Becker, and Stephan Günnemann. Gemnet: Universal directional graph neural networks for molecules. Advances in Neural Information Processing Systems, 34: 6790-6802, 2021.
Lei Geng, Yue Feng, Yaxi Niu, Fang Zhang, and Huaqing Yin. Mgt: Machine learning accelerates performance prediction of alloy catalytic materials. Journal of Chemical Information and Modeling, 65(1))31-40, 2024.
Justin Gilmer, Samuel Schoenholz, Patrick Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference on machine learming, pp. 1263-1272, PMLR, 2017.
Jonathan Godwin, Michael Schaarschmidt, Alexander L Gaunt, Alvaro Sanchez-Gonzalez Yulia Rubanova, Petar Veličković, James Kirkpatrick, and Peter Battaglia. Simple gnn regularisation for 3d molecular property prediction and beyond. In International Conference Learning Representations, a.
Jonathan Godwin, Michael Schaarschmidt, Alexander Gaunt, Alvaro Sanchez-Gonzaler Yulia Rubanova, Petar Veličković, James Kirkpatrick, and Peter Battaglia. Simple gnn regularisation for 3d molecular property prediction and beyond. International Conference on Learning Representations, b.
Benjamin Gutteridge, Xiaowen Dong, Michael M Bronstein, and Francesco Di Giovanni. Drew: Dynamically rewired message passing with delay. In International Conference Machine Learning, pp. 12252-12267. PMLR, 2023.
Thomas Halgren. Merck molecular force field. basis, form, scope, parameterization, and performance of mmff94. Journal of computational chemistry, 17(5-6):490-515 1996.
Takuya Hashimoto and Keiji Maruoka. Recent advances of catalytic asymmetric 1, 3-dipolar cycloadditions. Chemical reviews, 115(11):5366 5412, 2015.
Xiaoxin He, Bryan Hooi, Thomas Laurent, Adam Perold, Yann LeCun, and Xavier Bresson. A generalization of vit/mlp-mixer graphs. In International conference on machine learning, pp. 12724-12745. PMLR, 2023.
Marti A. Hearst, Susan Dumais, Edgar Osuna, John Platt, and Bernhard Scholkopf. Support vector machines. IEEE Intelligent Systems and their applications, 13(4):18-22 1998.
Weihua Hu, Matthias Fey, Hongyu Ren, Maho Nakata, Yuxiao Dong, and Jure I Leskovec. Ogb-Isc: A large-scale challenge for machine learning on graphs. In Thirty-fifth Conference Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2022.
Yinan Huang, Siqi Miao, and Pan Li. What can we learn from state space models for machine learning on graphs? arXiv preprint arXiv:2406.058155 2024.
Md Shamim Hussain, Mohammed Zaki, and Dharmashankar Subramanian. Global self-attention as replacement for graph convolution. In Proceedings of the 28th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pp. 655-665, 2022.
Md Shamim Hussain, Mohammed J Zaki, and Dharmashankar Subramanian. Triplet interaction improves graph transformers: Accurate molecular graph learning with triplet graph transformers. In Forty-first International Conference on Machine Learning, 2024.
Rui Jiao, Jiaqi Han, Wenbing Huang, Yu Rong, and Yang Liu. motivated equivariant pre- training for 3d molecular graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 8096-81 0104, 2023.
12

---

Published as a conference paper at ICLR 2026
Bowen Jing, Gabriele Corso, Jeffrey Chang, Regina Barzilay, and Tommi Jaakkola. Torsional diffusion for molecular conformer generation. Advances in Neural Information Processing Systems, 35:24240-2444 24253, 2022.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger,
Kathryn Tunyasuvumakool, Russ Bates, Augustin Zidek. Anna Potapenko, al. Highly accurate protein structure prediction with alphafold. nature, 596(7873):583-588 2021.
Horst Kessler. Conformation and biological activity of cyclic peptides. Angewandte Chemie International Edition in English, 21(7):512-523, 1982.
Jinwoo Kim, Dat Nguyen, Sconwoo Min, Sungjun Cho, Moontae Lee, Honglak Lee, and Seunghoon Hong. Pure transformers are powerful graph learners. Advances in Neural Information Processing Systems, 35:14582 -4592-45955 2022.
Thomas Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv: 1609002077 2016.
Walter Kohn, Axel D Becke, and Robert G Parr. Density functional theory of electronic structure. 1996.
The journal of physical chemistry, (00(31):129744110
Devin Kreuzer, Dominique Beaini, Will Hamilton, Vincent Létourneau, and Prudencio Tossou. Rethinking graph transformers with spectral attention. Advances in Neural Information Processing Systems, 1222111221629 2021.
Greg Landrum. Rdkit documentation. Release, 1(1-79):4, 2013.
Tuan Le, Frank Noé, and Djork-Arné Clevert. Equivariant graph attention networks for molecular property prediction. arXiv preprint arXiv:2202.0989), 2022.
Guohao Li, Chenxin Xiong, Ali Thabet, and Bernard Ghanem. Deepergen: All you need to train deeper gens. arXiv preprint arXiv:2006.0777 07739, 2020.
Shuangli Li, Jingbo Zhou, Tong Xu, Liang Huang, Fan Wang, Haoyi Xiong, Weili Huang, Dejing Dou, and Hui Xiong. Structure-aw aware interactive graph neural networks for the prediction of protein-ligand binding affinity. In Proceedings of the 27th ACM SIGKDD conference knowledge discovery & data mining, pp. 975-985, 2021.
Xinze Li, Penglei Wang, Tianfan Fu, Wenhao Gao, Chengtao Li, Leilei Shi, and Junhong Liu. Autodiff: Autoregressive diffusion modeling for structure-based drug design. arXiv preprint arXhv:2404.02003. 2024a.
Xuan Li, Zhanke Zhou, Jiangchao Yao, Yu Rong, Lu Zhang, and Bo Han. Neural atoms: Propagating long-range interaction molecular graphs through efficient communication channel. In The Twelfth International Conference Learning Representations, 2024b.
Yi-Lun Liao and Tess Smidt. Equiformer: Equivariant graph attention transformer for 3d atomistic graphs. In The Eleventh International Conference on Learning Representations.
Yi-Lun Liao, Brandon M Wood, Abhishek Das, and Tess Smidt. Equiformerv2: Improved equivariant transformer for scaling to higher-degree representations. In The Twelfth International Conference on Learning Representations, 2024.
Jaechang Lim, Seongok Ryu, Kyubyong Park, Yo Joong Choc, Jiyeon Ham, and Woo Youn Kim. Predicting drug-target interaction using novel graph neural network with 3d structure-ebeedded graph representation. Journal of chemical information and modeling, 59(9):3981-399) -338888 2019.
Ning Lin, Jiacheng Cen, Anyi Li, Wenbing Huang, and Hao Sun. Reducing symmetry increase in equivariant neural networks. In The Fourteenth International Conference on Learning Representa- tions, 2026.
Lihang Liu, Donglong He, Xiaomin Fang, Shanzhuo Zhang, Fan Wang. Jingzhou He, and Hua Wu. Gem-2: Next generation molecular property prediction network with many-body and full-range interaction modeling. arXiv preprint arXiv:2208.0580 055663, 2022a.
13

---

Published as a conference paper at ICLR 2026
Shengchao Liu, Hanchen Wang, Weiyang Liu, Joan Lasenby, Hongyu Guo. and Jian Tang. Pre- training molecular graph representation with 3d geometry. In ICLR 2022 Workshop on Geometrical and Topological Representation Learning.
Yi Liu, Limei Wang, Meng Liu, Yuchao Lin, Xuan Zhang, Bora Oztekin, and Shuiwang Ji. Spherical message passing for 3d molecular graphs. In International Conference on Learning Representations (ICLR), 2022b.
Pedro EM Lopes, Benoit Roux, and Alexander D MacKerell Jr. Molecular modeling and dynamics studies with explicit inclusion of electronic polarizability; theory and applications. Theoretical Chemistry Accounts, 124(1):1 1) 1-28, 2009.
Frank Lovering, Jack Bikker, and Christine Humblet. Escape from flatland: increasing saturation as approach to improving clinical success. Journal of medicinal chemistry, 52(21):6652--6 6756. 2009.
Shuqi Lu, Zhifeng Gao, Di He, Linfeng Zhang, and Guolin Ke. Highly accurate quantum chemical property prediction with uni-mol++. arXiv preprint arXiv:2303. 10.3033.6900 /6982, 2023.
Shengjie Luo, Tianlang Chen, Yixian Xu, Shuxin Zheng. Tie- Yan Liu, Liwei Wang, and Di He. One transformer can understand both 2d & 3d molecular data. The Eleventh International Conference Learning Representations, 2022.
Liheng Ma, Chen Lin, Derek Lim, Adriana Homero-Soriang Puneet K Dokamia, Mark Coates, Philip Torr, and Ser-Nam Lim. Graph inductive biases in transformers without message passing. In International Conference on Machine Learning, pp. 23321-233311 PMLR, 2023.
Dominic Masters, Josef Dean, Kerstin Klaser, Zhiyi Li, Sam Maddrell-Mander, Adam Sanders, Hatem Helal, Deniz Beker, Ladislav Rampášek, and Dominique Beaini. Gps++: An optimised hybrid mpnn/transformer for molecular property prediction. arXiv preprint arXiv:2212.02229, 2022.
Maylis Orio, Dimitrios A Pantazis, and Frank Neese. Density functional theory. Photosynthesis research, 102:443-4533 453, 2009.
Wonpyo Park, Woong-Gi Chang, Donggeon Lee, Juntae Kim, al. Grpe: Relative positional encoding for graph transformer. In ICLR2022 Machine Learning for Drug Discovery.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, nigh-performance deep learning library. Advances in neural information processing systems, 32, 2019.
Brajesh K Rai, Vishnu Sresht, Qingyi Yang, Ray Unwalla, Meihua Tu, Alan M Mathiowetz, and Gregory A Bakken. Torsionnet: A deep neural network to rapidly predict mmall-molecule torsional energy profiles with the accuracy of quantum mechanics. Journal of Chemical Information and Modeling, 62(4))785-800, 2022.
Ladislav Rampášek, Michael Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu, Guy Wolf, and Do- minique Beaini. Recipe for general, powerful, scalable graph transformer. Advances in Neural Information Processing Systemx, 35: 15511-1154555 2022a.
Ladislav Rampášek, Michael Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu, Guy Wolf, and Do- minique Beaini. Recipe for general, powerful, scalable graph transformer. Advances in Neural Information Processing Systems, 35: 1551111555,, 2022h.
Esmat Rashedi, Hossein Mewaaaaadii-oourt and Saeid Saryazdi. Gsa: gravitational search algorithm. Information sciences, 179( 179(13):223--2248 2009.
Yu Rong, Yatao Bian, Tingyang Xu, Weiyang Xie, Ying Wei, Wenbing Huang, and Junzhou Huang. Self-supervised graph transformer on large-scale molecular data. Advances in neural information processing systems, 33:12559-12571, 2020.
14

---

Published as a conference paper at ICLR 2026
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural networks. In International conference on machine learning, pp. 9323-93323 PMLR, 2021.
Kristof Schütt, Pieter-Jan Kindermans, Huziel Enoc Sauceda Felix, Stefan Chmiela, Alexandre Tkatchenko, and Blaus-Robert Müller. Schnet: continuous- convolutional neural network for modeling quantum interactions. Advances in neural information processing systems, 30, 2017.
Kristof Schütt, Oliver Unke, and Michael Gastegger. Equivariant message passing for the prediction of tensorial properties and molecular spectra. In International Conference on Machine Learning, pp. 9377-9388. PMLR, 2021.
Liushuai Shi, Le Wang. Chengjiang Long, Sanping Zhou, Mo Zhou, Zhenxing Niu, and Gang Hua. Sgen: Sparse graph convolution network for pedestrian trajectory prediction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.
2021.
Yu Shi, Shuxin Zheng, Guolin Ke, Yifei Shen, Jiacheng You, Jiyan He, Shengjie Luo, Chang Liu, Di He, and Tie- Yan Liu. Benchmarking graphormer on large-scale molecular modeling datasets. arXiv preprint arXiv:2203.0m 04810, 2022.
Justin S Smith, Roman Zubatyuk, Benjamin Nebgen, Nicholas Lubbers, Kipton Barros, Adrian E Roitberg, Olexandr Isayev, and Sergei Tretiak. The ani- lecx and ani-1x data sets, coupled-cluster and density functional theory properties for molecules. Scientific data, 7(1):134, 2020.
Ying Song, Shuangjia Zheng, Zhangming Niu, Zhang-Huu Fu, Yutong Lu, and Yuedong Yang. Communicative representation learning on attributed molecular graphs. In 29th International Joint Conference on Artificial Intelligence and the 17th Pucific Rim International Conference on Artifi- cial Intelligence JJCAI-PRICAI2020), International Joint Conferences on Artificial Intelligence Organization, 2020.
Hannes Stärk, Dominique Beaini, Gabriele Corso, Prudencio Tossou, Christian Dallago, Stephan Günnemann, and Pietro Liò. 3d infomax improves gnns for molecular property prediction. In International Conference Machine Learning, pp. 20479-20502. PMLR, 2022.
Philipp Thölke and Gianni De Fabritiis. Torchmd-net: Equivariant transformers for neural net- work based molecular potentials. In ICLR 2022-70th International Conference on Learning Representations, International Conference Learning Representations, ICLR, 2022.
Viet-Khoa Tran-Nguyen, Célien Jacquemard, and Didier Rognan. Lit-pcba: an unbiased data set for machine learning and virtual sereening. Journal of chemical information and modeling, 60(9): 4263 4273,4273,10 2020.
Oliver T Unke and Markus Meuwly. Physnet: A neural network for predicting energies, forces, dipole moments, and partial charges. Journal of chemical theory and computation, 15(6):3678-360 3693, 2019.
A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
Petar Velickovie, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio, et al. Graph attention networks. stat, 1000220)::00485500 2017.
George Wald. Molecular basis of visual excitation. Science, 162(3850):230-23) 1968.
Bowen Wang, Chen Liang, Jiaze Wang, Furui Liu, Shaogang Hao, Dong Li, Jianye Hao, Guangyong Chen, Xiaolong Zou, and Pheng-Ann Heng. Dr-label: Improving gnn models for catalysis systems by label deconstruction and reconstruction. arXiv preprint arXiv:2303.02875, 2023.
Limei Wang, Yi Liu, Yuchao Lin, Haoran Liu, and Shuiwang Ji. Comenet: Towards complete and efficient message passing for 3d molecular graphs. Advances in Neural Information Processing Systems, 35:650-664, 2022.
Renxiao Wang, Xueliang Fang, Yipin Lu, and Shaomeng Wang. The pdbbind database: Collection of binding affinities for protein- ligand complexes with known dire-dimmnsiooal structures. Journal of medicinal chemistry, 47(12):2777-2980 2004.
15

---

Published as a conference paper at ICLR 2026
Yusong Wang, Shaoning Li, Tong Wang, Bin Shao, Nanning Zheng, and Tie- Yan Liu. Geometric transformer with interatomic positional encoding. Advances in Neural Information Processing Systems, 36, 2024a.
Yusong Wang. Tong Wang. Shaoning Li, Xinheng He, Mingyu Li, Zun Wang, Nanning Zheng, Bin Shao, and Yan Liu. Enhancing geometric representations for molecules with equivariant vector-scalar interactive message passing. Nature Communications, 15(I):313, 2024b.
Zun Wang, Guoqing Liu, Yichi Zhou, Tong Wang, and Bin Shao. Efficiently incorporating quintuple interactions into geometric deep learning force fields, Advances in Neural Information Processing Systems, 36, 2024c.
Geoffrey Webb, Eamonn Keogh, and Risto Mikkkulanenn. Naive bayes. Encyclopedia of machine learning, 15(1):713-714, 2010.
Zhanghao Wu, Paras Jain, Matthew Wright, Azalia Mirhoseini, Joseph E Gonzalez, and Ion Stoica. Representing long-range context for graph neural networks with global attention. Advances in Neural Information Processing Systems, 34: 14266-11133 2021.
Yujie Xing, Xiao Wang, Yibo Li, Hai Huang, and Chuan Shi. Less is more: on the over-globalizing problem in graph transformers. In Forty-first International Conference on Machine Learning. 2024.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? arXiv preprint 1rXiv:1810.00826 2018.
Kevin Yang, Kyle Swanson, Wengong Jin, Connor Coley, Philipp Eiden, Hua Gao, Angll Guzman- Perez, Timothy Hopper, Brian Kelley, Miriam Mathea, et al. Analyzing learned molecular representations for property prediction. Journal of chemical information and modeling, 59(8): 3370-3388, 2019.
Tianmeng Yang. Yujing Wang, Zhihan Yue, Yaming Yang, Yunhai Tong, and Jing Bai. Graph pointer neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pp.
2022.
Chengxuan Ying. Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie- Yan Liu. Do transformers really perform badly for graph representation? Advances in neural information processing systems, 34:28877-28888, 2021.
Yangtian Zhang, Zuobai Zhang, Bozitao Zhong, Sanchit Misra, and Jian Tang. Diffpack: A torsional diffusion model for autoregressive protein side-chain packing. Advances in Neural Information Processing Systems, 36, 2024.
Zimei Zhang, Gang Wang, Rui Li, Lin Ni, RunZe Zhang, Kaiyang Cheng, Qun Ren, Xiangtai Kong, Shengkun Ni, Xiaochu Tong, al. Tora3d: an autoregressive torsion angle prediction model for molecular 3d conformation generation. Journal of Cheminformatics, 15(1):57, 2023.
Liangzhen Zheng, Jingrong Fan, and Yuguang Mu. Onionnet: a multiple-layer intermolecular-contaan based convolutional neural network for piotein-ligand binding affinity prediction. ACS omega, (14):15956-1596) 2019.
Gengmo Zhou, Zhifeng Gao, Qiankun Ding, Hang Zheng, Hongteng Xu, Zhewei Wei, Linfeng Zhang, and Guolin Ke. Uni-mol: A universal 3d molecular representation learning framework. In The Eleventh International Conference on Learning Representations, 2023.
A BROADER IMPACTS
Through our paper, we demonstrate that TetraGT can better predict molecular properties and generate precise conformations in drug development, showing its powerful applicability in practical drug discovery tasks. However, given its effectiveness, TetraGT could potentially be misused to generate harmful molecular conformations and predict toxic properties. This risk can be mitigated by com- prehensively considering toxicity and other side effects in the properties of downstream tasks, or by screening out harmful molecular conformations during the generation process.
16

---

Published as a conference paper at ICLR 2026
DFT
TGT
TetraGT
Figure 3: The ability to identify local chirality. The first row depicts DFT conformations. The second and third rows show the corresponding molecular conformations from TGT distance predictor and TetraGT conformations predictor. TetraGT can accurately generate molecules with local chirality identical the target conformation, whereas TGT conformations, relying solely on distance matrices, exhibit deviations. Red arrows indicate atoms representing the centers of local chirality in the molecules.
B DENSITY FUNCTIONAL THEORY FOR MOLECULAR CONFORMATION PREDICTION
Density Functional Theory (DFT) (Kohn et al. 1996 Orio et al., 2009) is first-principles compu- tational method based on quantum mechanics that plays crucial role in molecular conformation generation and property prediction. DFT describes many-electron systems through electron density rather than wave functions, significantly reducing computational complexity. Its theoretical founda- tion rests on the Hohenberg- Kohn theorem, which proves that all properties state can be uniquely determined by the electron density. In practical applications, the complex many- electron problem is transformed into more tractable single-electron problems through the Kohn-Sham equations. In molecular conformation generation, DFT can obtain precise dimee-iimnsional confor- mations of molecules by solving electronic structure equations. This process includes optimizing molecular geometry, calculating bond lengths, bond angles, and dihedral angles, determining the lowest energy conformation, and predicting electron distribution within molecules. The molecular conformations generated by DFT possess high accuracy and are often used as benchmarks for the evaluation of other conformation generation methods. This high precision stems from its rigorous quantum mechanical theoretical foundation, which can accurately describe electronic effects, chemi- cal bonding properties, and intramolecular interactions in molecules. However, DFT calculations also have limitations, such as high computational cost and difficulty in handling large molccular systems. In modern molecular design, DFT often complements machine learning methods (Schütt et al. 2017] Axelrod & Gomez-Bone Bombarell 2022 Smith et al., 2020). Machine learning models can quickly predict molecular properties and intiill conformations, while DFT is used generate high-precision reference conformations and validate results. This combination leverages the advantages of both methods: the efficiency of machine learning and the high accuracy of DFT. With improvements in computational power and algorithms, DFT's applications in molecular science research will continue expand. providing crucial support for drug design, materials development, and other fields.
system's 's ground
C QUANTITATIVE ANALYSIS OF CHIRALITY PREDICTION
The conformation predictor outputs binned distances and angles under local chirality constraints, providing essential structural information for downstream task predictors. To quantitatively evaluate
17

---

Published as a conference paper at ICLR 2026
TetraGT'ss improvement over TGT in handling local chirality, conducted systematic evaluation on the PCQM4M training set, which contains 3,803,453 molecules, including 1,772,922 molecules with chiral centers (46.61%). The evaluation methodology compares model-predicted 3D conformers with high-precision DFT-calculated conformers, using angular deviations around chiral centers as the assessment criterion, with deviation threshold of x/6. Experimental results demonstrate TetraGi's superiority over the baseline TGT model across three key metrics in Table 9 In terms of bond angle MAE, TetraGT achieves 0.205 rad, 16.6% reduction compared to TGT's 0.246 rad. For torsion angle MAE, TetraGT reaches 0.312 rad, significantly lower than TGT's 0.597 rad by 47.7%. Regarding chirality prediction accuracy, TetraGT attains 76.7%, substantially outperforming TGT's 32.5% with 136% improvement. These quantitative results strongly validate TetraGT's excellence modeling chiral structures, particularly complex torsion angle prediction and overall chirality determination tasks. The substantial improvements across all metrics demonstrate the effectiveness of TetraGT's direct angle modeling approach capturing local molecular geometry.
Table 9: Comparison of TetraGT and TGT performance on chirality prediction
Model
Bond Angles MAE (rad) Torsion Angles MAE (rad)
Chirality Pred (%)
TGT TetraGT
0.246
0.597
32.5
0.205
0.312
76.7
D THE ACCURACY OF CONFORMATION PREDICTOR IN ANGLES AND DISTANCES
To demonstrate the accuracy of TetraGT in geometric conformation prediction, we convert distances and angles to continuous unbounded values. Following the strategy employed in TGT (Hussain et al. 2024). we train two small refinement networks for distances and angles respectively. These networks accept clipped and binned values as input and output continuous, unbounded values. We train these networks using MAE loss and employ random inference to obtain the median of the output distances. We compare the accuracy of individual pairwise distances and angles on the validation-3D split of the PCQM4My2 dataset (i.c., data unseen during training), based on MAE, RMSE (Root Mean Square Error), and percentage errors within different thresholds shown in Table TO and Table IT Our findings indicate that in terms of distances, our TetraGT predictor outperforms TGT across all metrics. Regarding angles, TetraGT significantly surpasses both RDKit and TGT in bond angle prediction and substantially leads in torsion angle prediction. This suggests that through angle constraints, TetraGT's conformation predictor can more accurately predict the underlying structure of molecules compared to the distance predictor in TGT.
Table 10: Accuracy of pairwise distances terms MAET, RMSEJ and percent error within threshold (EwTt).
Model
MAE (A)
RMSE (Å)
EwT-0.2Á(%%)
AwT-0.1A(S%)
EwT--0.5A((%)
FwT-0.01 Á(%)
RDKit
0.248
0.541
73.33
66.65
56.90
26.79
TGT Refiner
0.152
0.378
80.53
75.68
70.80
54.54
TetraGT'+ + Refiner
0.127
0.322
87.32
79.15
75.31
Table 11: Accuracy of bond angles and torsion angles in terms of MAEJ, RMSEJ and percent error within a threshold (EwT1).
Bond Angles
Torsion Angles
Model
MAE (rad)
RMSE (rad)
EwT-n/16 rad (%)
MAF (rad)
RMSE (rad)
FwT-x/ 116 (S)
RDKit
0.239
0.575
71.43
0.694
1.145
33.62
TGT + Refiner
0.225
0.431
76.26
0.563
0.713
41.89
TetraCTT Refiner
0.185
0.362
83.50
0.306
0.467
62.31
18

---

Published as a conference paper at ICLR 2026
E EFFICIENCY ANALYSIS OF TETRAGT VERSUS BASELINE MODELS
E. PCQM4MV2
Table 12 presents a comprehensive comparison of TetraGT against state-of-the-art molecular pre- training methods, Unimol+ and TGT, across different model scales, showing parameter counts, com- putational complexity, experimental performance on the PCQM4Mv2 dataset, and training/Inference times. Based experimental results, we comprehensively analyze TetraGT's method from both efficiency and effectiveness perspectíves. Regarding computational complexity, where N represents the number of atoms and w denotes the local sampling window size, TetraGT with local sampling achieves O complexity for standard atom and pair embedding interactions through restricting attention to w nearest neighbors, compared to the original ((N3) complexity. In typical molecules, the number of bond angles ranges from 1.5N to 2N, and torsion angles from N to 2N. With local sampling applied to these higher-order interactions, the additional computational complexity becomes O(wN), yielding an overall complexity of Oww + O(wN) N?) On the large-scale PCQM4Mv2 dataset, TetraGT demonstrates an excellent balance between performance and computa- tional efficiency. We systematically analyzed the trade-off between model scale and performance. Results show that 6-layer TetraGT (60M parameters) achieves an MAE of 69.4 me comparable to 18 18-layer Unimol+ (77M parameters) 69.3 meV, while significantly reducing training time (ap- proximately 14 days versus 40 days using A 1100 GPU). As model layers increase, 24-layer TetraGT (215M parameters) reduces MAE to 66.2 meV, significantly outperforming 24-layer TGT (203M parameters, 67.1 meV MAE). Notably. 12-layer TetraGT (127M parameters) maintains competi- tive performance (68.1 meV MAE) while reducing training time from 38 20 GPU days, with corresponding inference time reduction. While the efficiency improvement from local sampling is less pronounced on PCQM4Mv2 due its relatively small molecular sizes (average 14 atoms) and TetraGT's s additional higher-orde order structural interactions, this strategy is crucial for scalability enables TetraGT to successfully process large biomolecular datasets like PDBbind and Peptides, which would be computationally intractable without local sampling. These results indicate that TetraGT architecture is competitive even at smaller scales and can better leverage its structural modeling advantages parameter count increases.
E.2 OPEN CATALYST 2020 IS2RE
Table 13 presents comprehensive evaluation of TetraGT against both pre-trained and non- trn-pre-trained methods on the OC20 dataset, focusing on computational efficiency and model performance. Based on experimental results, analyze TetraGT's capabilities from multiple perspectives. Regarding computational efficiency. TetraGT demonstrates competitive inference and fine-tuning times com- pared non-pre-training methods. Specifically, TetraGT'ss fine-tuning duration (220 minutes) aligns well with established models such as DimeNet++ (230 minutes), GemNet-T (200 minutes), and SphereNet (290 minutes). While ComENet exhibits faster training speed (20 minutes), TetraGT achieves substantially superior performance metrics, with energy MAE of 397.7 meV versus 588. 8 VeV and FwT of 9. 14% versus 3.56%, validating the effectiveness of our pre-training strategy. In comparison with other pre-trained methods, TetraGT shows remarkable efficiency improvements while maintaining performance advantages. Compared to TGT, despite incorporating additional angu- lar information and direct angle modeling mechanisms, TetraGT maintains similar training efficiency (approximately 34 days versus 32 days using A 100 GPU) while achieving superior performance. Notably, compared to Uni-Mol+ TetraGT achieves better performance metrics while significantly reducing pre- pre-training time (34 days versus 1 112 days using A100 GPU), demonstrating an optimal balance between computational efficiency and model effectiveness.
F EXPERIMENTAL DETAILS
The model is implemented using the PyTorch (Paszke et al. 2019) library. We perform mixed- precision training on 2 nodes, each equipped with 8 NVIDIA Tesla A100 GPUs (80GB RAM/GPU) and 16-core 2.6GHz Intel Xeon CPUs (320GB RAM per node). The hyperparameters used for each dataset are presented in Table 14] For PCQM4Mv2 and OC20 list the hyperparameters for both the conformation and the task predictor models and both training and finetuning. For QM9, we only list the hyperparameters for finetuning. For MOLPCBA, LIT-PCBA, and MOLHIV only show
19

---

Published as a conference paper at ICLR 2026
Table 12: Comparison of performance and efficiency metrics PCQM4Mv2
Model
param.
Complexity
layers
MAE (meV)
Training Time
Inference Time
Unimol+
27.7M
O(
6
71.4
~12 A100 GPU day
19 A GPU min
Unimal+
52.4M
O(N3)
12
69.6
<42 A100 GPU day
<32 A A10D GPU min
Unimol+
77M
(
18
69.3
<u40 A100 GPU day
~56 V100 GPU min
TGT
116M
((N3)
12
70.9
~18 A100 GPU day
~27 A A100 GPU min
TGT
203M
O(Na)
24
67.1
~32 A100 GPU day
400 A GPU min
TetraGT
60M
OwwN2)
6
69.3
10 A100 GPU day
16 A GPU min
TetraGT
127M
O(wN2)
12
68.1
11 A100 GPU day
~29 A100 GPU min
TetraGT
215M
O(wN2)
24
65.9
~3 A100 GPU day
~33 A100 GPU min
Table 13: Comparison of performance and efficiency metrics on OC20
Model
Pretraining Time
Train Time
Inference Time
Avg. Energy MAE (meV)
Avg. FwT (%)
CGCNN
18 min
min
658.5
2.82
SchNet
10 min
min
666.0
2.65
DimeNett+
230 min
4 min
621.7
3.42
GemNet-T
min
min
638.2
3,38
SphereNet
290
min
602.3
3.64
ComENet
20 min
min
588.8
3.56
Unimnl+
112 A 100 GPU days
340 min
8 min
408.8
8.61
TGT
32 A100 GPU days
200 min
5 min
403.0
8.82
TetraGT
33 A100 GPU days
220 min
5 min
397.7
9.14
the hyperparameters for training from scratch. The missing hyperparameters do not apply to the corresponding dataset or model. For QM9 no secondary distance and angle denoising objective is used. For LIT-PCBA, 0 triplet interaction heads indicate that an EGT is used without any triplet interaction module.
To provide the conformation predictor with initial 3D information, utilize RDKit (Landrum, 2013)
to extract 3D coordinates and apply MM Force Field Optimization (Halgren 1996). Due to the absence of Ground Truth 3D coordinates in the PCQM4M<2 validation set, randomly divide the training set into train-3D and validation-31 splits, with the latter containing 5% of the training data. Hyperparameters of the conformation predictor fine-tuned by monitoring the average cross-entropy loss of binned distance and angle prediction on the validation-3D split, which is found be good indicator of downstream performance. The input noise level is adjusted by evaluating the finetuned performance on the validation set. We get the best results by using an average of 50 sample predictions during stochastic inference. Other training configurations not mentioned are based on / TGT (ussain et al. 2024) sttps://githinn /gggttbttt..ggo//g c shamim htt.:////..aaiiimmssssaaae/eeeea
(MIT license).
G ADDITIONAL RESULTS AND ANALYSES
G.1
QM9
We present the comprehensive evaluation results on the QM9 dataset across all 12 prediction tasks in Table Overall, TetraGT demonstrates strong and consistent predictive capabilities across diverse molecular properties. In particular, it achieves ttate-of-the-an performance on several energy-related metrics, including HOMO energy (EH: 8.5), LUMO energy (Ez: 8.7), energy gap (As: 15.6), and heat capacity (C.: 0.019). On the remaining targets where does not rank first, TetraGT typically attains second-best or competitive performance close to the top of the benchmark (e.g., C,: 0.020, matching TGT's performance), indicating that its overall performance level is comparable to the strongest existing methods rather than being specialized to a narrow subset of tasks. For optical and quantum properties such as and ZPVE, TetraGT remains highly competitive, and for thermodynamic quantities (U,, U, H, G) and geometric features (R?), it surpasses previous pre- trainer approaches including Transformer-M.
20

---

Published as a conference paper at ICLR 2026
Table 14: Hyperparameters for each dataset.
PCQM4MY2
OC20
QM9
PDBbind Task Pred.
LIT-PCBA Task Pred.
Peptides*
Hyperparameters
Conf. Pred.
Task Pred.
Conf: Pred.
Task Pred.
Task Pred.
Task Pred.
Sampl. Window
Layers
24
24
24
14
24
24
8
24
Node Embed. Dim
768
768
768
768
768
768
1024
768
Edge Embed. Dim
256
256
256
512
256
256
256
256
Angle Emhed. Dim
128
128
128
256
128
128
128
128
# Altn. Heads
64
64
64
64
64
Triplet Heads
16
16
16
Node FFN Dim.
768
768
1536
768
768
768
2048
768
Edge FFN Dim.
256
256
512
512
256
256
512
256
Angle
128
256
Max. Hops Fine.
32
32
32
32
32
32
Activation Input Dist. Eac.
GELU Fourier
GELU Fourier
GELU
GELU
GELU
RBF
RBF
RBF
RBF
RBF
RBF
Source Dropout
0.3
0.3
0.3
0.3
0.3
0.3
0.3
0,3
Triplet Dropout
0.0
0.0
Path Drupout
0.2
0.2
0.2
0.1
0.2
0.1
0.1
0.1
Dropout
0,1
Edge Activ. Dropout
0.1
0.1
0.1
0.1
0.1
0.1
0.1
0,]
Angle Activ. Dropout
Input 3D Noise
0.2
0.6
0.0
0.0
Input Noise Smooth.
0.0
Optimizer
Adam
Adam
Adam
Adam
Adam
Adam
Adam
Adam
Batch Size
1024
2048
256
256
1024
128
Max.
0.001
0.001
10
Min. I.R
10-6
10-6
0.001
10-6
10-7
10-
Warmup Steps
30000
20000
8000
16000
600
10000
Total Training Steps
60000
350000
30000
100000
1200
30000
Grad. Clip. Norm
5.0
5.0
5.0
5.0
5.0
5.0
2.0
5.0
Conf. Loss Weight
3.0
0.0
0.0
0.1
0.1
Angle Bins
256
512
256
512
512
512
# Dist. Bins
256
512
256
512
512
512
Dist. Bins Range
16
8
8
FT Batch Size
2048
1024
2048
64
128
FT Warmup Steps
3000
1000
2000
FT Total Steps
50000
12000
150000
20000
30000
FT Max. 1.R
2 x 10
10
10-4
2 x 10
10
LR
6
10 6
10
10" 6
FT Conf. Loss Weight
0.1
2.0
0.1
0.1
0.1
*The hyperparameters in the Peptides column represent both Peptides-fune (2D) and Peptides- (3D). They share model hyperparameters, with the training hyperparameters in the second and third sections applying to Peptides-fune func (2D), and the fine-tuning hy hyperparameters the last section applying to Peptides-struct (3D).
The heterogeneous performance across different QM9 properties can be better understood by exam- ining the alignment between our pre-training objectives and the underlying physics of each target.
TetraGT is treitrained PCQM4Mv2 with dual objective that (i) predicts the HOMO-LUMO gap and (ii) denoises inter-atomic distances and angular variables. This setup primarily encourages the model to learn holistic description of the electronic esseture--specciallyy the frontier orbitals- together with geometrically consistent 3D representation. As result, properties in QM9 whose behavior tightly coupled to frontier orbital energies and global electronic configurations (namely и. E., and As, as well as some energy-related thermodynamic quantities) benefit most directly from the pre-trained representations, and indeed TetraGT exhibits the largest gains on these targets. By contrast, properties such as the dipole moment μ4, polarizability ZPVE. or R² depend more heavily on long-rane range charge distribution, polarization effects, and global shape fluctuations, which are only indirectly constrained by our current pre-training objective. In this sense, the supervision signal derived from the HOMO-LUMO gap induces natural bias towards metrics with similar distributions and physical dependencies, leading to excellent but not universally dominant performance all 12 tasks.
We also note that architectural design choices further shape this behavior when compared to highly specialized equivariant energy models. Methods such as EquiformerV2+NN adopt strict SE(3)- equivariant architectures tailored to continuous 3D coordinates and their derivatives (forces), which provides a very strong inductive bias for small- molll-mo energy and force prediction. In contrast, TetraGT designed general-purpose geometric representation learner: it introduces explicit
21

---

Published as a conference paper at ICLR 2026
Table 15: LIT-PCBA results in terms of ROC-AUCT (%).
ALDHI FENI
GBA
KAT2A
MAPKI PKM2
VDR
Average
No. active
7,168
369
166
194
308
546
884
No. inactive
137,965
355,402
296,052
348,548
62,629
245,523
355,388
NaiveBayes (ebb
2010
69.3
87.6
70,9
65.9
68.6
68.4
80.4
73.0
SVM (Hearst et al. 1998
76.0
87.7
77.8
61.2
66.5
75.3
69.7
73.4
RandomForee Breeman
2001
74.1
65.7
59,9
53.7
57.9
58.1
64.4
62.0
XGBoost (Chen & Guestrin
2016
75.0
88.8
83.0
50.0
59.3
73.7
78.2
72.6
GCN (Kipf & Welling
2016
73.0
89.7
73.5
62.1
66.8
63.6
77.3
72.3
GAT Velickovic 2022)
2017
73.9
88.8
77.6
66.2
69.7
72.4
78.0
75.2
FP-GNN (Cai al.
76.6
88.9
75.1
63.2
77.1
73,2
77.4
75.9
EGT (Hussain
2022
78.7
92.9(t
75.4(4)
72.8c)
75.3g)
76.5(2)
80.7g2
78.9
GEM (Fang ct al.
2022)
77.2())
91.4m
82.1g)
74.0g)
71.0g)
74.6,z
78.5()
78.4
GEM-2 (Lau el al.
2022a)
80.2
94.5
85.6p)
76.3())
73.3
78.2(0.4)
82.3
81.5
EGT+RDKI (Hussmm et al.
2024
80.2
95.2
84.5(4)
74.3()
73.5m
78.0ko.)
82.8,03)
81.2
TGT Hussain al. 2024
80.6
95.5
84.4.3)
74.6,2)
74.3,07)
78.4
82.9(a3)
81.5
TetraGT
81.2
95.7 85.6(3)
75.8g)
16.1
79.1.u.4;
83.4(0.) 82.4
bond-angle and torsion-angle tokens, employs tetrahedral interaction mechanisms encode local geometry and chirality, and uses hierarchical virtual nodes to aggregate information from atoms, bonds, angles, and torsions. This design is intended to support transfer across a wide range of tasks and scales (PCQM4Mv2, OC20 IS2RE, PDBBind, Peptides-struct, Peptides-func, LIT-PCBA), rather than being exhaustively tuned for each individual scalar property in QM9. For fairness and generality, we use unified pre-training objective and architecture across benchmarks, without introducing task-specific loss functions or architectural variants for different QM9 targets. Inevitably, this entails a trade-off between broad generalization and task-specific optimization: TetraGi attains state-of-the-art or ncar-state-of- -f-the- performance on most properties, especially those aligned with its pre-training supervision, while some highly specialized equivariant models can still be marginally better subset of QM9 metrics.
In Table 4 categorize compared methods into three distinct groups for clarity. The first group comprises pre-trained GNN methods, including GraphMVP, GEM, 3D Infomax, and 3D-MGP, The second group consists of directly trained GNN methods, spanning from SchNet through Sa VeNt. The third group encompasses Transformer-based methods from for this family, we do not distinguish between pre-trained and non-pre-trained variants because they are all trained on large-scale datasets, and our focus is on contrasting their architectural inductive biases (cquivariant energy models vs. our angle-driven tetrahedral representation) rather than the source of supervision alone.
through TetraGT;
G.2 COMPREHENSIVE ANALYSIS ON PDBBIND CORE SET
The PDBBind core set (version 2016) is rigorously curated benchmark containing 290 protein-ligant complexes selected from the PDBBind refined set based on stringent quality criteria. Each complex includes high-resolution crystal structures (typically < 2.5 Å) with experimentally determined binding affinities (Kd, Ki. or IC50 values) that span approximately 10 log units from picomolar to millimolar range. The dataset provides complete 3D coordinates for both protein and ligand molecules, along with binding affinity values converted to pKd/pKi (-log 10 of dissociationn/ihiiition constant in molar units) for consistency. The task is to predict the binding affinity value given the 3D structure of the protein-liget ligand complex, making it fundamental benchmark for structure-basee drug design. The set is specifically designed to test generalization capability, with proteins clustered by 90% sequence similarity and only one representative selected from each cluster, ensuring diverse protein families and binding modes are represented.
TetraGT achieves a Pearson correlation coefficient 0.85240.017 on this benchmark, representing a significant advancement over previous state-of-the-art methods. The improvement Transformer-
22

---

Published as a conference paper at ICLR 2026
M (R=0.830=0.011) which previously held the best performance, demonstrates that our tetrahedral geometric constraints provide more effective inductive bias than standard transformer architectures for molecular binding prediction. More importantly, the reduction in mean absolute error from 0.940 to 0.909 meV indicates TetraGT"ss enhanced capability in making accurate quantitative predictions, which is crucial for practical applications in lead optimization where small differences in binding affinity can determine therapeutic efficacy.
When compared to classical scoring functions, TetraGT shows even more dramatic improvements. RF-Score, which relies on carefully engineered features based on atom-pair distance counts, achieves R-0.789, while OnionNet, using multiple layers of intermolecular contacts, reaches R=0.768. The substantial gap between these methods and TetraGT illustrates the power of learned representations that can capture complex geometric patterns beyond predefined feature sets. Among GNN-based approaches, the progression from early methods like GNN-DTI (R=0.736) to more sophisticated architectures like SIGN (R=0.797) shows steady improvement, yet TetraGT's performance leap suggests that explicitly modeling tetrahedral geometry provides crucial structural information that general graph convolutions miss.
The stability of TetraGT's predictions, evidenced by the lowest standard deviation across all metrics (SD=1.181 101=000100,, is particularly noteworthy. This consistency across diverse protein families and ligand chemotypes indicates that the model has learned robust representations that generalize well beyond the training distribution, a critical requirement for virtual screening applications where novel chemical scaffolds must be evaluated reliably.
G.3 DETAILED ANALYSIS ON PEPTIDES BENCHMARKS
The Peptides-furg funne and Peptides-struet datasets from the Long Range Graph Benchmark (LRGB) comprise 15,535 peptides ranging from to 50 amino acids in length, derived from the SATPdb database of therapeutic peptides. Each peptide is represented as a molecular graph where nodes correspond to atoms and edges represent chemical bonds, with node features encoding atom types (9 categories) and edge features encoding bond types (3 categories). The datasets share the same peptide molecules but differ fundamentally in their tasks and available information. Peptides- func is a multi-labe classification task predicting 10 binary functional properties: antimicrobial, antibacterial, antiviral, antifungal, anticancer, anti- anti-HIV, antimalarial, antiparasitic, antimycobacttrrial and cell peletrrenggg capabilities. Importantly, this dataset provides only 2D molecular graphs without experimental 3D) coordinates, requiring methods to infer structural information from connectivity alone. In contrast, Peptides-struee struct is a regression task predicting 11 continuous structural properties computed from peptide 3D conformations: three principal components mass inertia, three principal components of valence inertia (hydrogen distribution), three geometric axis lengths, sphericity, and plane distance. This dataset includes the actual 3D coordinates, allowing direct utilization of spatial information. Both datasets are split into train (70%), validation (15%), and test (15%) sets with stratified sampling to ensure balanced property distributions.
For Peptides-fune, which lacks experimental 3D) coordinates, TetraGT's approach of using a pre- trained conformer predictor to generate approximate 3D features proves highly effective, achieving 72.86+0.399 .3%% Average Precision. This surpasses the previous best DRew model (71.50+0.449% by meaningful margin, particularly impressive given the multi-label nature of the task where peptides can simultaneously exhibit multiple therapeutic properties. The improvement over GraphGPS (65.3440.91 %) and MGT (68.17#0.64 17+0.64%) further demonstrates that TetraGT's geometric message passing, even with predicted conformations, captures functionally relevant structural motifs more effectively than standard graph transformers. The ability to accurately predict diverse functional properties from molecular structure alone has important implications for therapeutic peptide discovery, where rapid screening of large peptide librarics is essential.
For Peptides-struct, where actual 3D structural information is available, TetraGT achieves an MAE of 0.2421 outperforming all baseline methods including Graph ViT/MLP-Mixer (0. 001110.00 which previously held the best performance, The consistent improvement across all 11 structural properties suggests that TetraGT's 's tetrahedral representations naturally align with the geometric nature of these prediction targets. The model's 's ability to accurately predict properties like inertia components and geometric dimensions indicates deep understanding of 3D molecular
23

---

Published as a conference paper at ICLR 2026
geometry, essential for applications requiring precise structural modeling such peptide docking conformational analysis.
The performance gap between methods is particularly revealing when comparing approaches with different architectural philosophics. Traditional graph neural networks like SAN (Peptides-fume 64.39%, Peptides-struct: 0.2545) struggle with both tasks, likely due to limited expressivity in captur- ing long-range dependencies common in peptide structures. More recent architectures incorporating attention mechanisms show improvement, with GRIT achieving 69. 88.88% on functional prediction and 0.2460 on structural regression. However, these gains remain incremental compared to TetraGT's substantial improvements, suggesting that generic architectural enhancements alone are insufficient without appropriate geometric inductive biases.
An interesting observation emerges when comparing performance patterns across the datasets. Methods that excel on Peptides-fune func don't necessarily maintain their relative performance on Peptides- struct. For instance, DRew shows strong functional classification (71.50%) but moderate structural prediction (0.2536), while Graph VTT/MLP-Mixet shows the opposite pattern. TetraGT is unique in achieving top performance on both tasks, indicating its representations capture both functional and structural aspects effectively. This dual capability is particularly valuable for drug discovery applications where understanding both molecular function and conformation is essential.
G.4 LIT-PCBA
We also show a breakdown of the LIT-PCBA results for the individual protein targets in Table [5] Notice that, TetraGT outperforms other models in ALDH1, FEN1, PKM2, VDR and GBA. Despite the low number of positive samples, TetraGT ranked second among all models on KAT2A and MAPK1, surpassing TGT (Hussain et al.) (024) on all proteins target. We can analyze why TetraGT shows slightly lower performance on KAT2A, and MAPK1 compared to some other methods. KAT2A and MAPKI both have limited active samples (194 and 308 respectively) with significant class imbalance. The performance differences are relatively small for KAT2A, TetraGT achieves 75.8% compared to GEM-2's 76.3%, and for MAPK1, TetraGT's 76.1% is the best performers. These marginal differences might be attributed to the specific structural characteristics of these proteins and the extreme class imbalance their datasets, which could potentially benefit from more specialized handling of imbalanced data during model training.
In Tables and 15
we present three groups methods. The first group consists of traditional machine learning methods (NaiveBayes, SVM. RandomForest, XGBoost). The second group consists of directly trained GNNs (GCN, GAT, FP-GNN), The third group consists of pre-trained deep learning methods from EGT through TGT.
G.5 PCQM4Mv2
In Table 2 we organize methods into three groups. The first group represents earlier methods, ranging from MLP-Fingerprini through GPS++. The second group includes current state-of-the-art methods (Transformer-M. M, Uni-Mol+, TGT) that incorporate 3D conformation perturbation and denoising prediction. The final group consists solely of our proposed TetraGT method.
G.6 OC20
For T methods are divided into two main categories. The first group encompasses GNN methods from SchNet through GNS+NN, while the second group includes Transformer-based methods from Graphormer-3D through TGT.
H SCALABILITY ANALYSIS
Our comprehensive evaluation demonstrates TetraGT's robust performance across molecular systems of vastly different scales, from small organic molecules in PCQM4Mv2 (mean: 15 atoms) and MolHIV/MoIPCB. (mean: 26 atoms) to large pootein-ligame complexes in PDBBind (500-1000 atoms in binding sites) and complex catalytic systems in OC20 (approximately 80 atoms). TetraGT
24

---

Published as a conference paper at ICLR 2026
achieves of-te-oftuee art performance consistently across all these benchmarks, indicating excellent scalability without architecture-speciff limitations.
To systematically evaluate scalability, conducted detailed experiments the OC20 IS2RE dataset by grouping molecules into size segments and analyzing performance variations. Table I6 presents Energy MAE across different molecular size ranges, demonstrating remarkably stable performance with MAE values fluctuating within a narrow band of 22 meV (354.7-377.0 meV) despite the molecular size nearly tripling from smallest largest categories. This consistency confirms that tetrahedral message passing does not suffer from size-dependent degradation commonly observed in graph neural networks.
pressntss the
Table 16: Performance analysis of TetraGT model on OC20 IS2RE dataset across various molecular sizes.
Atom num
Energy MAE (me
36-45
363.7
46-55
376.1
56-65
354.7
66-75
377.0
76-85
371.2
86-95
368.9
96-105
364.5
The absence of monotonic error trends with increasing molecular size indicates that TetraGT ef- fectively captures both local and global geometric patterns regardless of system scale. The slight variations in MAE across different ranges likely reflect inherent complexity differences in molecular compositions rather than size- dependenndnt model limitations. Notably, the optimal performance 56- 65 atoms aligns well with typical organic adsorbate sizes in catalysis applications, while maintaining competitive accuracy for both smaller and larger systems.
While TetraGT'ss architecture poses no inherent theoretical limitations on molecular size, practi- cal applications face two primary constraints. The first stems from GPU memory capacity when processing 3D conformer data, which defines maximum processable system size. This constraint becomes particularly relevant for large biomolecular systems requiring extensive memory allocation. The second challenge arises from computational complexity scaling, as the number of higher-ordee structures (bond angles and torsion angles) grows quadratically with system size, potentially causing attention mechanisms suffer from averaging effects across expanding interaction spaces.
The demonstrated scalability across diverse molecular scales, combined with consistent performance stability, positions TetraGT model's ability to maintain high accuracy from small drug-like molecules to large biomolecular complexes without size-specific modifications validates the tetrahedral message passing design as an effective balance between computational efficiency and representational power. Future exploration of the proposed optimization strategies may further extend TetraGT's applicability even larger biochemical systems while maintaining practical computational requirements.
versatile tool for broad molecular modeling applications. The
I ADDITIONAL DETAILS ABOUT RELATED WORKS
Molecular Property Prediction The remarkable performance of message-passing GNNs in pre- dicting molecular properties has inspired new generation of geometric and physics-aware neu- ral networks, which maintain invariance or equivariance under 3D rotational and translational transformations. Early developments in this direction include SchNet (Schütt et al. 2017) and DimeNet (Gasteiger et al. 2020)), which pioneered the use of distance-based convolution approaches. The field further evolved with the introduction of spherical methodologies, as exemplified by Gem- Net (Gasteiger et al. 2021). SphereNet (Liu et al. 2022b), ComENet (Wang et al., 2022). LEFT- Net (Du et al., 2024). and SAVENet (Aykent & Xia 2024). each incorporating various forms of angular information. This architectural evolution ultimately led to more sophisticated equivari- ant transformer designs, including Equiformer (Liao Smidt), EquiformerV2 (Liao et al., 2024). TDrchMD-Net (Thölke & De Fabritiis 2022), and Geoformer (Wang al. 2024a). which generalized
25

---

Published as a conference paper at ICLR 2026
the concept of equivariant aggregation. While these advances have significantly improved molec- ular representation learning, our work proposes a fundamentally different paradigm for modeling higher-order structures. Recent models like QuinNet (Wang et al. 2024c) and ViSNet (Wang et al.) 2024b) have introduced four or five-atom interactions to enhance model expressiveness and accuracy. However, these methods primarily focus local representations of atomic nodes and chemical bonds, capturing higher-order features implicitly through combinatorial operations between atom-level tokens. In contrast, our approach transforms higher-order graph structures into independent token representations, enabling direct learning and representation of structural patterns in molecules. This innovation is particularly crucial for model interpretability and effective utilization of expert prior knowledge. From an information propagation perspective, traditional methods require higher-order structural information (such as four-body and five-body interactions) to propagate gradually along the graph topology, creating significant information bottlenecks. As demonstrated in TGT research, even information exchange between adjacent embeddings faces restrictions. Our method addresses these limitations through direct structural token representation, not only avoiding these bottlenecks but also enabling efficient access and utilization of key higher-order information by all graph nodes, thereby providing a more effective framework for learning molecular structural information.
J SENSITIVITY ANALYSIS OF THE WINDOW SIZE w
In this section, we provide a systematic analysis of the local sampling window size w. In TetraGT, the tetrahedral interaction uses local window select neighboring angle/torsion tokens that are allowed to participate in attention. If we is too small, in principle this may lead to "under-coverage" of important geometric couplings, potentially degrading both conformer prediction and property prediction. To examine this, we conduct experiments on the OC20 IS2RE dataset (in-distribution validation split), varying w (5, 10, 15, 20), and study both overall performance and its dependence on molecular size.
We first evaluate, for different w, the conformer prediction errors (MAE of pairwise distances, bond angles, and torsion angles), the property prediction error (Energy MAE), and the relative training time per epoch (normalized such that the setting w 10 corresponds to 1.0). The results are summarized in Table 17 We observe that increasing w from 5 to 10 leads to clear improvements in both geometric and energy prediction: for example, Energy MAE decreases from 382.4 me to 373.2 meV, while bond-angle MAE decreases from 0.297 rad to 0.228 rad and torsion-angle MAE from 0.387 rad to 0.316 rad. However, once w 2 10, the additional gains become very limited: increasing w from 10 to 15 20 only reduces Energy MAE by about 0.6-1.1 meV, whereas the training time per epoch increases from 1.0 to 1.52 and 2.83, respectively. This indicates that, beyond a moderate threshold (e-g., w 10), TetraGT is quite robust the choice of w; we do not observe noticeable accuracy degradation that could be attributed to local under-coverage, while overly large w mainly increases computational cost with rapidly diminishing returns.
Table 17: Effect of different window sizes w on conformer prediction and energy prediction (OC20 IS2RE, in-distribution validation).
Distance MAE (A)
Bond-angle MAE (rad)
Torlion-augle MAE (rad)
Energy MAE (meV)
Rel. time / epoch
5
0.191
0.297
0.387
382.4
0.76
10
0.168
0,228
0,316
373.2
1.00
15
0.163
0.221
0.313
372.6
1.52
20 0.157
0.217
0.309
372.1
2.83
To further investigate whether w needs to scale with molecular size, we stratify the OC20 IS2RE
in-distribution validation set by the number of atoms N into four ranges: N < 45, 45 < N < 65. 65 < N < 85, and 85 < N < 105. For each range, we report the Energy MAE under different w, as shown in Table 18 Across all size ranges, increasing w from 5 to 10 consistently yields a noticeable reduction in Energy MAE (typically around 12-16 V). In contrast, further increasing w from 10 to 15 or 20 brings only very minor improvements the order of 1-2 Ve in each size bin. Even for the largest molecules (85 < N < 105), the difference between using a moderate fixed window (w = 10) and larger windows is very small, and we do not observe any regime where large molecules require substantially larger w to maintain accuracy. This suggests that, at the scale of the OC20 benchmark, a fixed moderate window size already captures the most important local geometric couplings for energy
26

---

Published as a conference paper at ICLR 2026
prediction, and we find empirical evidence that needs grow explicitly with the total atom count N.
Table 18: Energy MAE vs. window size and molecule size (OC20 IS2RE, in-distribution validation; unit: meV).
N S
45
45 <
N
<
65
65 <
N S 85
85
N S 105
378.6
381.1
384.6
380.3
10
363.5
371.2
376.5
367.1
15
362.1
368.8
375.1
366.0
20
361.6
368.9
374.7
365.4
From modeling perspective, the tetrahedral interaction in TetraGT is designed to capture local geometric couplings within physically meaningful four-atom motifs. For any given bond angle torsion angle, the most relevant interacting angles/torsions are typically confined chemical neighborhood (within few bonds). The local coordination number of atoms is bounded by valence, which naturally imposes an upper bound the number of physically meaningful neighboring angles/torsions. Consequently, moderate window size (e.g.. = 10-15) is already sufficient in practice to cover the vast majority of such neighbors, while longer-range geometric dependencies can be propagated through stacked layers of angle and torsion tokens, rather than relying on single attention layer with very large w. Taken together, the empirical results and this geometric intuition suggest that TetraGT is robust to the choice of within reasonable range; choosing moderate w effectively avoids local geometric under-coverage while striking a favorable trade-off between accuracy and computational efficiency.
limited
27


Reviews:
[{"review_id": "wRpu6sYbmp", "reviewer": "Reviewer_bqeD", "summary": "This paper proposes TetraGT, a graph Transformer framework for molecular representation learning. The key idea is to treat not only atoms and chemical bonds as tokens, but also higher-order geometric structures, specifically bond angles and torsion angles, as explicit tokens that can directly interact through an attention mechanism constrained by tetrahedral geometric consistency, rather than relying on traditional models to infer such information implicitly from pairwise distances. The paper further introduces a chirality-aware angular learning objective, the Directed Cycle Angle Loss, which models angles as directed periodic variables over (0,2π); this is intended to distinguish local chirality and avoid discontinuities around cases that occur in standard angle regression or classification. In addition, the model employs a hierarchical virtual node design that aggregates information separately at the atom, bond, angle, and torsion levels before passing it to a global node, aiming to alleviate the information bottleneck of a single global aggregator. The authors argue that these components together enable more faithful modeling of stereochemistry, local chirality, and conformational stability, thereby improving representation quality and transferability.", "questions": "1. The method treats local groups of atoms as “tetrahedral units” and uses these units as the fundamental template for constrained attention and geometric consistency. However, the paper does not demonstrate how well this assumption holds in other common chemical settings (e.g., conjugated rings, metal coordination sites) where local geometry is not tetrahedral. It is unclear whether this inductive bias could introduce systematic errors in such non-tetrahedral regimes, or whether the model adapts automatically?\n\n2. The tetrahedral geometric inequalities are injected as attention biases/gates, and the authors argue this both reduces complexity from 𝑂(𝑁3) to approximately 𝑂(𝑤𝑁2) and enforces physical consistency. However, the paper does not provide a formal characterization of this mechanism. In particular, it does not report metrics such as the fraction of geometrically invalid local angle/torsion configurations before vs. after training, nor does it prove that predicted angles/torsions are always physically realizable. It remains unclear whether this acts as a true constraint or mainly as an inductive bias.\n\n3. The method relies on a local sampling window of size 𝑤 to decide which angle/torsion tokens are allowed to interact. If 𝑤 fails to cover an important substructure, or only partially covers it, the model could miss critical geometric couplings. The paper does not present sensitivity studies on different 𝑤 values or analyze the impact of under-coverage on accuracy, nor does it clarify how 𝑤 should scale with molecular size.\n\n4. Training still requires on the order of tens of A100 GPU-days, comparable to other top-performing large models, which is a high barrier for many labs. The authors assert that the approach is scalable to larger systems, but they do not analyze how resource usage grows when moving to substantially larger molecules or protein–ligand complexes. In particular, it is not clear whether the number of tokens and pairwise interactions will scale roughly linearly or blow up faster, so the scalability claim is not yet quantitatively supported. \n\n5. With respect to the overall pipeline: in the multi-stage training process, could you clarify the relationship between the “conformation predictor” and the “task predictor”? It would also be helpful to more explicitly describe the full loss composition across stages — is it primarily pairwise atomic distance regression together with the Directed Cycle Angle Loss for angles, or are there additional property-prediction heads contributing to the objective? \n\n6. (Minor question) In the abstract, Figure 1, and the conclusion, the authors use the alternate name “TDGT” for the proposed architecture but do not provide a proper definition.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. The paper directly addresses three known gaps: explicit modeling of chirality, explicit treatment of higher-order geometry (bond and torsion angles), and explicit handling of geometric consistency constraints. \n\n2. The proposed tetrahedral interaction module constrains attention among angle/torsion tokens to local tetrahedral units and encodes geometric consistency, aiming to retain physical validity while avoiding naive O(N^3)high-order connectivity. \n\n3. The Directed Cycle Angle Loss treats angles as directed periodic variables over (0,2π), which is intended to distinguish local chirality and avoid instability at angular wrap-around. Most prior models do not handle it directly. \n\n4. The model promotes atoms, bonds, bond angles, and torsions to first-class tokens and aggregates them via hierarchical virtual nodes, rather than relying on a single global node. This is meant to reduce information bottlenecks. \n\n5. The method is evaluated on diverse and competitive benchmarks (PCQM4Mv2, OC20 IS2RE, QM9, PDBBind, Peptides, LIT-PCBA) and reports state-of-the-art or near state-of-the-art results against strong baselines, suggesting transferability beyond a single task", "weaknesses": "1. The method treats local groups of atoms as “tetrahedral units” and uses these units as the fundamental template for constrained attention and geometric consistency. However, the paper does not demonstrate how well this assumption holds in other common chemical settings (e.g., conjugated rings, metal coordination sites) where local geometry is not tetrahedral. It is unclear whether this inductive bias could introduce systematic errors in such non-tetrahedral regimes, or whether the model adapts automatically?\n\n2. The tetrahedral geometric inequalities are injected as attention biases/gates, and the authors argue this both reduces complexity from 𝑂(𝑁3) to approximately 𝑂(𝑤𝑁2) and enforces physical consistency. However, the paper does not provide a formal characterization of this mechanism. In particular, it does not report metrics such as the fraction of geometrically invalid local angle/torsion configurations before vs. after training, nor does it prove that predicted angles/torsions are always physically realizable. It remains unclear whether this acts as a true constraint or mainly as an inductive bias.\n\n3. The method relies on a local sampling window of size 𝑤 to decide which angle/torsion tokens are allowed to interact. If 𝑤 fails to cover an important substructure, or only partially covers it, the model could miss critical geometric couplings. The paper does not present sensitivity studies on different 𝑤 values or analyze the impact of under-coverage on accuracy, nor does it clarify how 𝑤 should scale with molecular size.\n\n4. Training still requires on the order of tens of A100 GPU-days, comparable to other top-performing large models, which is a high barrier for many labs. The authors assert that the approach is scalable to larger systems, but they do not analyze how resource usage grows when moving to substantially larger molecules or protein–ligand complexes. In particular, it is not clear whether the number of tokens and pairwise interactions will scale roughly linearly or blow up faster, so the scalability claim is not yet quantitatively supported. \n\n5. In the abstract, Figure 1, and the conclusion, the authors use the alternate name “TDGT” for the proposed architecture but do not provide a proper definition.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BoxFSBDIBM", "reviewer": "Reviewer_nYFx", "summary": "This paper proposes TetraGT, an attention-based model designed to accurately predict molecular properties from both 2D molecular graphs and 3D conformations. Existing methods often fail to directly model key molecular geometric parameters, such as bond angles and torsion angles, leading to limitations in capturing high-order structural relationships and local molecular chirality. TetraGT addresses these shortcomings through four core innovations: (1) The direct modeling of geometric parameters as structured tokens to prevent error accumulation from indirect atomic and bond representations; (2) A spatial tetrahedral attention mechanism, informed by tetrahedral geometry theory, to facilitate information exchange between these parameters; (3) An improved directed cycle angle loss function to handle geometric parameters and identify local chirality; (4) A hierarchical virtual node aggregation architecture that captures sub-structural information for a comprehensive molecular representation.\nExperimental results demonstrate that TetraGT achieves superior overall performance on upstream datasets such as PCQM4Mv2 and OC20 IS2RE compared to models like Uni-Mol+. It also shows strong generalizability, outperforming models like EquiformerV2+NN on downstream tasks including QM9 and PDBBind.", "questions": "1. A systematic explanation should be provided in the main text to account for the performance gap on the specific QM9 metrics where TetraGT does not achieve optimal results.\n2. The \"Related Work\" section should be relocated to follow the \"Introduction\" and precede the \"Method\" section. Content in the introduction that details core challenges and innovations (e.g., the rationale for the spatial tetrahedral attention mechanism) should be integrated into this restructured \"Related Work\" section to better contextualize the research.\n3.  The key contributions within the \"Introduction\" section should be presented as a clear, itemized list to enhance readability and provide a straightforward overview of the paper's novel elements.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The work is innovative in its direct representation of molecular geometry as structured tokens and the introduction of a spatial tetrahedral attention mechanism and a directed cycle angle loss function.\n2. The model design is comprehensive and methodologically sound, effectively integrating geometric parameter interaction, local chirality identification, and graph sub-structure aggregation.\n3. The model's performance and generalizability are rigorously validated across multiple upstream and downstream benchmark tasks.\n4. This research provides a novel paradigm for molecular property prediction, with considerable potential for application in drug design and materials discovery.", "weaknesses": "1. The analysis of experimental results is insufficient. While Table 4 shows that TetraGT achieves state-of-the-art performance on 5 out of 12 metrics on the QM9 dataset, it is outperformed on the remaining 7 metrics by models like EquiformerV2+NN. The manuscript lacks a systematic discussion to explain this performance disparity.\n2. The organization of the related work section is non-standard. Placing the \"Related Work\" section after the \"Experiments\" and immediately before the \"Conclusion\" deviates from conventional academic structure.\n3. The introduction is poorly articulated. The summary of key contributions in the \"Introduction\" section is not itemized, which hinders readability and fails to clearly preview the paper's innovations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0N3XLfhrfL", "reviewer": "Reviewer_P24Z", "summary": "TetraGT is a molecular representation model that explicitly incorporates 3D geometric parameters (bond angles and torsion angles) into graph transformer architectures. TetraGT treats angles and dihedrals as structured tokens and models their interactions based on tetrahedral geometry constraints. The method defines mathematical relationships among face and dihedral angles to ensure physically valid spatial configurations and introduces a multi-level attention mechanism that hierarchically updates representations from atoms to bonds, bond angles, and torsion angles. A specialized “tetrahedral interaction module” enables efficient communication among geometrically related triplets and quadruplets while reducing complexity via local sampling. Additionally, TetraGT introduces a directed circular angle loss to handle periodicity and chirality in angle prediction and employs hierarchical virtual nodes to integrate multi-level structural information for final molecular property prediction.", "questions": "There have been recent higher-order geometric message passing and equivariant Transformers (e.g., GemNet, Equiformer, TFN, SE(3)-Transformers, MACE) that already model angular and dihedral information. Does this explicit token-based formulation yield substantial conceptual or empirical advantage beyond existing geometric message passing frameworks?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Explicit “angle and torsion tokens” + “tetrahedral attention” are new combinations not seen in previous 3D graph Transformers.\n- The physical constraints (Lemma 1) add rigor, not just architectural novelty.\n- The directed cycle angle loss (DCA loss) for handling 2π-periodicity is conceptually sound and addresses chirality.\n- The experiments show strong performance gains over baselines across multiple benchmarks, but clearer comparisons with recent equivariant models and ablations on key components would better substantiate the claimed improvements.", "weaknesses": "- While the model’s use of tetrahedral geometry constraints is mathematically grounded, the repeated use of the term “tetrahedral” may be confusing in a chemical context, since most molecular sites are not tetrahedral in bonding geometry. The paper should clarify that “tetrahedral” here refers to the geometric configuration of any four non-coplanar atoms (a 3D simplex), rather than chemically tetrahedral centers (sp³ atoms).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xbR0m3P1Eb", "reviewer": "Reviewer_uB9E", "summary": "This work focuses on traditional molecular modeling, aiming to learn effective representations from 3D molecular configurations and then fine-tune them for specific downstream tasks (e.g., property prediction). The authors identify that previous methods often fail to incorporate higher-order geometric structures. To address this, they enhance Graph Transformer (GT) architectures by introducing bond angle and torsion angle information as tokens and designing a spatial tetrahedral attention mechanism. Additional techniques include a specially designed “directed cycle angle loss” and a hierarchical virtual node for molecule-level information aggregation. Experimental results demonstrate strong performance.", "questions": "1. In Table. 12&13, the training and inference times of UniMol+ and TGT are missing. Since inference time is typically more important in practical applications, could you clarify why these results were omitted?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. **Strong benchmark results.** The proposed method achieves excellent performance on several benchmarks (LIT-PCBA, PCQM4MV2, OC20 IS2RE, QM9), reaching or surpassing state-of-the-art results in nearly all cases. These results effectively demonstrate the transfer learning capability of the approach across diverse downstream tasks.\n2. **Novel tetrahedral constraints.** The introduction of Tetrahedral Constraints and the Tetrahedral Interaction Module shows a novel perspective on modeling geometric relationships. The ablation study further confirms that this module contributes meaningfully to overall performance.\n3. **Chirality structure analysis.** The authors conduct a statistical analysis of chirality structures (46.61% of the total 3M molecules). Table 9 compares errors with TGT, supporting the rationale behind their more complex design choices (e.g., attention mechanism and loss functions). The ablation study on the Directed Cycle Loss also validates the effectiveness of this component.\n4. **Hierarchical virtual node.** The hierarchical virtual node design provides a certain degree of novelty and contributes to molecular-level information aggregation, though the innovation is not particularly substantial compared to other components.", "weaknesses": "1. **Efficiency concern.** As a work focused on model architecture design, the paper lacks analysis or discussion of model efficiency in the main text. Although Appendix E presents promising results, a more comprehensive discussion of efficiency in the main body would strengthen the paper’s overall contribution and practical relevance.\n2. **Discussion of prior works on higher-order geometric structures.** Works such as [1,2] have also considered interactions among higher-order geometric structures, albeit in **implicit** ways (in contrast to the token-based approach proposed in this paper), as mentioned and discussed in Appendix J. Although the *“Implicit Modeling of Geometric Structures”* section in the Introduction provides some relevant statements, it would be helpful to clearly specify **which prior works** do not explicitly model higher-order geometric structures and **how** these structures are indirectly represented through combinations of atomic positions or pairwise relationships. Including this clarification in the main Introduction would better emphasize the motivation and novelty of the proposed approach.\n\n[1] Wang Z, Liu G, Zhou Y, et al. Efficiently incorporating quintuple interactions into geometric deep learning force fields[J]. Advances in Neural Information Processing Systems, 2023, 36: 77043-77055.\n\n[2] Wang Y, Wang T, Li S, et al. Enhancing geometric representations for molecules with equivariant vector-scalar interactive message passing[J]. Nature Communications, 2024, 15(1): 313.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)