Relational Graph Transformer
Authors: Vijay Prakash Dwivedi
Sri Jaladi
Yangyi Shen
Federico Lopez
Charilaos I. Kanatsoulis
Rishi Puri
Matthias Fey
Jure Leskovec
Venue: ICLR 2026 Poster (2026)
Keywords: graph
graph transformer
relational deep learning
Primary area: learning on graphs and other geometries & topologies
TL;DR: We propose the Relational Graph Transformer (RelGT), the first graph transformer architecture designed for deep learning relational databases.

Abstract: Relational Deep Learning (RDL) is a promising approach for building state-of-the-art predictive models on multi-table relational data by representing it as a heterogeneous temporal graph. However, commonly used Graph Neural Network models suffer from fundamental limitations in capturing complex structural patterns and long-range dependencies that are inherent in relational data. While Graph Transformers have emerged as powerful alternatives to GNNs on general graphs, applying them to relational entity graphs presents unique challenges: (i) Traditional positional encodings fail to generalize to massive, heterogeneous graphs; (ii) existing architectures cannot model the temporal dynamics and schema constraints of relational data; (iii) existing tokenization schemes lose critical structural information. Here we introduce the Relational Graph Transformer (RelGT), the first graph transformer architecture designed specifically for relational tables. RelGT employs a novel multi-element tokenization strategy that decomposes each node into five components (features, type, hop distance, time, and local structure), enabling efficient encoding of heterogeneity, temporality, and topology without expensive precomputation. Our architecture combines local attention over sampled subgraphs with global attention to learnable centroids, incorporating both local and database-wide representations. Across 21 tasks from the RelBench benchmark, RelGT consistently matches or outperforms GNN baselines by up to 18%, establishing Graph Transformers as a powerful architecture for Relational Deep Learning.

Published as a conference paper at ICLR 2026
RELATIONAL GRAPH TRANSFORMER
Vijay Prakash Dwivedi Sri Jaladi'
Yangyi Shen'
Federico López?
Charilaos I. Kanatsoulis' Rishi Puri Stanford University, AK,moAA, SNVIDIA { (ddwiedii, jure (cs. stacsssstafooreedu
Matthias Fey"
Jure Leskovec
ABSTRACT
Relational Deep Learning (RDL) is a promising approach for building state-of- the-art predictive models on multi-table relational data by representing as a heterogeneous temporal graph. However, commonly used Graph Neural Network models suffer from fundamental limitations in capturing complex structural pat- terns and long-range dependencies that are inherent in relational data, While Graph Transformers have emerged as powerful alternatives to GNNs on general graphs, applying them to relational entity graphs presents unique challenges: (i) Traditional positional encodings fail to generalize massive, heterogeneous graphs; (ii) exist- ing architectures cannot model the temporal dynamics and schema constraints of relational data; (iii) existing tokenization schemes lose critical structural informa- tion. Here introduce the Relational Graph Transformer (RELGT), the first graph transformer architecture designed specifically for relational tables. RELGT em- ploys novel multi-element tokenization strategy that decomposes each node into five components (features, type, hop distance, time, and local structure), enabling efficient encoding of heterogeneity, temporality, and topology without expensive precomputation. Our architecture combines local attention over sampled subgraphs with global attention learnable centroids, incorporating both local and database- wide representations. Across 21 tasks from the RelBench benchmark, RELGT consistently matches outperforms GNN baselines by up to 18%, establishing Graph Transformers as powerful architecture for Relational Deep Learning"
INTRODUCTION
Real-world enterprise data, such as financial transactions, supply chain data, e-commerce records, product catalogs, customer interactions, and electronic health records, are predominantly stored in relational databases (Codd) 1970). These databases typically consist of multiple tables, each dedicated different entity types, interconnected through primary-foreign key links. This abstraction underpins large quantities of complex, dynamically updated data that scale with business volume, storing potentially immense, unexploited knowledge (Fey al. 2024). However, extracting predictive patterns from such data has traditionally depended on manual feature engineering within complex machine learning pipelines, requiring the transformation of multi-table records into flat feature vectors suitable for models like deep neural networks and decision trees (Chen & Guestrin 2016).
Relational Deep Learning. To enable end-to-end deep learning. relational databases can be repre- sented as relational entity graphs (Fey al. 2024), where nodes correspond to entities and edges capture primary-foreign key relationships. This graph-based representation allows Graph Neural Networks (GNNs) learn abstract features directly from the underlying data structure, effectively modeling complex dependencies for various downstream prediction tasks. With this setup, which is termed Relational Deep Learning (RDL), GNNs reduce or eliminate the need for manual feature engineering and often lead to better performance (Robinson et al. 2024) at a fraction of the traditional model development cost.
Existing gaps. Despite their effectiveness, standard message pastagggggg GNN architectures (Gilmer al., 2017 Kipl & Welling 2016 Hamilton al. 2017 Velickovic al. 2017) have notable
limitations, such as insufficient structural expressiveness (Xu et al. 2019 Morris et al. 2019 Loukas

---

Published as a conference paper at ICLR 2026
xI lipers
Seed Node
Local Tokens mm
Global Centroks
user churn
item
Local Context
Allention
Relational Entity Graph
Tokenization
Transformer Network
Prediction Heads
Overview of the RELGT architecture. First, the input relational entity graph (REG) is converted into tokens where each training seed node (such as the customer node in this example) gets a fixed number of neighboring nodes, which are encoded with a multi-element tokenization strategy. These tokens are then passed through Transformer network that builds both local and global representations, which are then fed to downstream prediction layers.
Figure 1:
2019) and restricted long-range modeling capabilittis (Alon & Yahav 2020). For example, consider an e-commerce database with three tables: customers, transactions, and products, which can be represented as a relational entity graph as in Figure hops away from each other, connected only through shared customers. 'his creates an information bottleneck: transaction-Uu crnssacciioot--ttrssaactttt to-transaction patterns require multiple layers of message passing, while product relationships remain entirely indirect in shallow networks. Furthermore, products would never directly interact in a two-layer GNN (Robinson et al. 2024). as their messages must pass through both transaction and customer, highlighting the inherent structural constraints of GNN architectures that restrict capturing long-range dependencies.
standard GNN, transactions are always two
Graph Transformers (GTs) have emerged as more expressive models for graph learning, utilizing self-attention in the full graph to increase the range of information flow and additionally, incorporating positional and structural encodings (PEs/SEs) to better capture graph topology (Dwivedi & Bresson 2021: Ying et al. 2021 Rampášek et al. 2022). These advances have produced strong results across domains (uiler al. 2023). including foundation models for molecular graphs (Sypetkowski et al.) 2024). However, many GT designs are limited to non-temporal, homogeneous, and small-scale graphs, assumptions that do not hold for relational entity graphs (REGs) (Fey al. (024), which are typically (i) heterogeneous, with different tables representing distinct node types; (ii) temporal, with entities often associated with timestamps and requiring careful handling to prevent data leakage; (iii) large-scale, containing millions or records across multiple interconnected tables. In particular, existing PEs often require precomputation, depend on graph size, and typically do not scale well to large, heterogeneous, dynamic graphs (Cantürk al. 2023 Kanatsoulis et al. (2025). For instance, node2vec (Grover& & Leskovec) 2016). while more efficient than Laplacian or random walk PEs, become prohibitively expensive and impractical to compute
massive graphs (Postávaru et al. 2020). These limitations, along with the inability to capture the multi-dimensional complexity of relational structures, render current GTs inadequate for relational databases.
Present work. We introduce the Relational Graph Transformer (RELGT), the first Graph Transformer specifically designed for relational entity graphs. RELGT addresses key gaps in existing methods by enabling effective graph representation learning within the RDL framework. It is a unified model that explicitly captures the temporality, heterogeneity, and structural complexity inherent to relational graphs. We summarize the architecture as follows (Figure
We develop a multi-element tokenization scheme that converts each node into structurally enriched tokens. By sampling fixed-size subgraphs as local context windows and encoding each node's features, type, hop distance, time, and local structure, RELGT captures fine-grained graph properties without expensive precomputation at the subgraph or graph level.
Tokenization:
We develop a transformer network that combines local and global representations, adapting existing GT architectures (Rampášek et al. 2022). The model extracts features from the local tokens while simultaneously attending to learnable global tokens that act as soft centroids, effectively balaneing fine-gre -en-erer structural modeling with database-widd patterns.
Attention:
We showcase RELGT's effectiveness through comprehensive evaluation on 21 tasks from RelBench (Robinson et al., 2024).. RELGT consistently outperforms GNN baselines, with gains of up to 18% establishing transformers a powerful architecture for relational deep learning. Compared to HGT, a strong GT baseline for heterogeneous graphs, RELGT achieves better results without added computational cost, even when HGT uses Laplacian eigenvectors for PF.
Validation:
2

---

Published as a conference paper at ICLR 2026
2 BACKGROUND
2.1 RELATIONAL DEEP LEARNING
Relational Deep Learning is an end-to-end learning framework that converts relational databases into graph structures, enabling direct use of GNNs for representation learning (Fey et al. 2024).
Definitions. Formally, we can define a relational database as the tuple (T.R) RR comprising a collection of tables T (Ti... ........ Tn connected through table relationships R S TxT. A link (Tney: They:TPkey) E R denotes a foreign key in one table referencing a primary key in another. Each table contains entities (rows) {v1,..., Uny with each entity typically consisting of: (1) unique identifier (primary key). (2) references to other entities (foreign keys), (3) entity-specific attributes, and (4) timestamp information indicating when the entity was created modified. The structure relational databases inherently forms a graph representation, called as relational entity graphs (REGs). An REG is formally defined as a heterogeneous temporal graph G (V, E, Φ. 1, 7), where nodes V represent entities from the database tables, edges E represent primary-foreign key relationships, maps nodes to their respective types based on source tables, s assigns relation types to edges, and т captures the temporal dimension through timestamps (Fey et al. 2024
Challenges. Relational entity graphs exhibit three distinctive properties that set them apart from conventional graph data. First, their structure is fundamentally schema-defineds with topology shaped by primary-foreign keys rather than arbitrary connections, creating specific patterns of information flow that require specialized modeling approaches. Second, they incorporate temporal dynamics, as relational databases track events and interactions over time, necessitating techniques like aware neighbor sampling prevent future information from leaking into past predictions. Third, they display multi-type heterogeneity, as different tables correspond to different entity types with diverse attribute schemas and data modalities, presenting challenges in creating unified representations that effectively integrate information across diverse node and edge types (Schlichtkrull et al.) 2018 Wang al. 2019). These characteristics create both challenges and opportunities for GNN architectures, requiring models that can simultaneously address temporal evolution, heterogeneous information, and aaaeemma-constrinnge structures while processing potentially massive multi-table datasets.
2.2 RDL METHODS
The baseline GNN approach introduced by (Robinson et al. 2024) for RDL uses a heterogeneous GraphSAGE (Hamilton et al. (2117) model with temporal-aware neighbor sampling, which demon- strates significant improvements compared to traditional tabular methods like LightGBM (Ke et al.) 2017) acToss most of the tasks in the RelBench benchmark. This baseline architecture leverages Py Frame's modal feature encoders (Hu et al.) (202)) to transform diverse entity attributes into initial feature embeddings that serve imput to the GNN. Several specialized architectures have been developed to address specific challenges in relational entity graphs. ReIGNN (Chen et al. 2025) introduces composite pessage-passingg with atomic routes to facilitate direct information exchange between neighbors of bridge and hub nodes, commonly found in relational structures. Similarly, ContexIGNN (Yuan al. 2024) employs hybrid approach, combining pair-wise and two-tower representations, specifically y optimized for recommendation tasks in RelBench.
Beyond pure GNN approaches, sttrivall-uggment and hy brid tabular- GNN methods (Lachi et al. 2024) have also demonstrated comparable or superior performance to the standard GNN baseline, while showing the use of L.MMs (Grattafiori al., 2024) and inference speedups, respectively. These approaches confirm the effectiveness of graph, tabular, and LLLM-base' methods for downstream predictions in RDL. However, these methods typically optimize specific aspects of the problem, failing to incorporate broader advances from GTs in general.
generation techniques (Wydmuch et al. 2024)
2.3 GRAPH TRANSFORMERS
Graph Transformers extend the self-attention mechanism from sequence modeling VVawwani et al.
2017) to graph-structured data, offering powerful alternatives to traditional GNNs (Dwivedi & Bresson 2)21). These models typically restrict attention local neighborhoods, functioning as message-passee passing networks with attention biteentio--based aggregation (Joshil 2020) [Bronstein et al. 2021 while positional encodings developed based Laplacian eigenvectors (Dwivedi al. 2020)
3

---

Published as a conference paper at ICLR 2026
Subsequent Graph Transformers incorporate global attention mechanisms, allowing all nodes to
attend to one another (Ying et al. [202]] Mialon et | Kreuzer et al. (221). This moves beyond
the local neighborhood limitations of standard GNNs (Alon & Yahav 2020), albeit at the cost of significantly increased computational complexity.
Modern GT architectures have improved the aforementioned early works by creating effective structural encodings and ensuring scalability to medium and large-scale graphs. For structural expressiveness of the node tokens, several positional and structural encoding methods have been developed (Dwivedi et al., 2022] Cantürk et al. 2023 Lim et al. [0222 Huang et al. Kanatsoulis et al., 2025) to inject the imput graph topology. For scalability. various strategies have emerged
including hierarchical clustering that coarsens graphs (Zhang et al. [2022] ZZhu et al.] (023), sparse
attention mechanisms that reduce computational cost (Rampášek et 20.2 [Shirzad [
al. 2023).
and neighborhood sampling techniques for processing massive graphs (Zhao et al. 2021] Chen
al. 2022] Dwivedi et al., 2023). Models like GraphGPS (Rampášek et al. 2022) combine these advances through hybrid local-global designs that maintain Transformers' global context advantages while ensuring practical efficiency when scaling to medium and large graph datasets. However, these approaches exhibit several key limitations: they are largely confined static graphs, and lack mechanisms to handle multiple node and edge types. While specialized Transformers for heterogeneous graphs exist (Hu et al., 2020; Mao al. 2023 Zhu et al.) 2023] Zhai et al., 2024), integrating them, alongside other aforementioned methods, into the RDL. pipeline remains challenging. This is primarily because adapting PEs under precomputation constraints is difficult, compounded by the complexity of modeling large-scale, temporal, and heterogeneous relational entity graphs (REGs).
2.4 ADDRESSING CHALLENGES
While heterogeneous graph transformers (Hu al. 2020) and temporal graph methods exist, no prior GT architecture effectively handles relational entity graphs where heterogeneity, temporality, and rich entity attributes co-occur within schema-definee database structures. Heterogeneous Temporal Graph Transformers like HTGformer (Wang] (2025) process heterogeneity and temporality through separate, iterative modules without graph positional encodings, a component now considered essential in modern GTss (Rampášek et al., 2022 Müller et al., (203)), and do not address the multimodal attributes or schema constraints inherent to relational databases (Hu et al. 2024). Existing subgraph- based GTs (Zhao et al. 202 Chen et al. 2022) focus on scalability for homogeneous graphs without mechanisms for heterogeneity or temporal dynamics.
We address these limitations by recognizing that relational entity graphs require a rethinking on how their multimodal attributes and comprchensive graph structure are jointly processed. We systematically decompose the information coming from the REGs into specialized components that can be independently learned and composed, as we describe in detail in Section 3 This principled design enables GTs to handle the unique combination of heterogeneity, temporality, and schema- defined structures in relational databases without expensive global precomputation.
3 RELGT: RELATIONAL GRAPH TRANSFORMER
3.1 TOKENIZATION
Traditional Transformers in NLP represent text through tokens with two primary elements: (i) token identifiers (or features) that denotes the token from a vocabulary set and (ii) positional encodings that represent sequential structure (Vaswani et al., (017)). For example, token can correspond to word and its positional encoding can correspond to order in the input sentence. Similarly, Graph Transformers generally adapt this two-element representation to graphs, where nodes are tokens with features, and graph positional encodings provide structural information. Although this two-element approach works well for homogeneous static graphs, it becomes computationally inefficient when trying to encode multiple aspects of graph structural information for REGs.
In particular, capturing heterogeneity, temporality, and schema-defined structure (as defined in Section 2.1) through a single positional encoding scheme would either require complex, multi-stage encoding or result in significant information loss about the rich relational context. For instance, if we were to extend existing PEs for REGs, several practical challenges emerge: (i) standard Laplacian random walk-based PEs would need significant modification differentiate between multiple
4

---

Published as a conference paper at ICLR 2026
relaional enclty graph
tokens to Transfomee Network
encoders applied to each loken w/ their graph structural information
Enooowr
Iyse
Enccoler
sampling
node feature
node type
hop distance
time
subgraph
seed nodes
local subgraph seed node
local tokens
soed nodes with local neighbors
The tokenization procedure. A temporal-aware subgraph sampling step extracts a fixed set of local tokens for each training seed node, denoted by the node in black. Each token incorporates its respective graph structure information, which are element-wise transformed to a common embedding space and combined form the effective token representation to be fed to the Transformer network.
Figure 2:
node types (e.g., customers products transactions), (ii) these encodings lack mechanisms to incorporate temporal dynamics critical for recent purchases more relevant than older ones), and (iii) the scale of relational databases makes global PE computation in REGs prohibitively expensive. With millions of records across tables, precomputation would only be feasible on small subgraphs, resulting incomplete structural context.
predictions (e.g., capturing that a user's
3.1.1 PROPOSED APPROACH
RELGT overcomes these limitations through a multi-element token representation approach, without any computational overhead concerning the dependency on the number of nodes in the input REG. Rather than trying to compress all structural information into a single positional encoding, we decompose the token representation into distinct elements that explicitly model different aspects of relational data. This decoupled design allows each component to capture a specific characteristic of REGs: node features represent entity attributes, node types encode table-based heterogeneity, hop distance preserves relative distances among nodes in a local context, time encodings capture temporal dynamics, and GNN-based positional encodings preserve local graph structure.
Sampling and token elements. The tokenization process in RELGT converts a REG G (V,E, o. th, 7) into sets of tokens suitable for processing by the Transformer network. Specifi- cally, as shown in Fig 2 for each training seed node Uj E V. we first sample fixed set of K neighboring nodes Uj from within 2 hops of the local neighborhood using temporal-aware sampling ensuring that only nodes with timestamps 7 (1;) < r(es) are included to prevent temporal leakage.
Each token in this set represented by 5-tuple: (Mey o(1)). p(r,, 2)), T(;)) - T(ua). GNN-PE..)) where, (i) node feature (Ix,)) denotes the raw features derived from entity attributes in the database, (ii) node type (Ø(1,)) is a categorical identifier corresponding to the entity's s originating table, (iii) relative hop distance (p(u,, Uy)) captures the structural distance between the seed node and the neighbor node vy. (iv) relative time (T(2,) T(Us)) represents the temporal difference between the neighbor and seed node, and (v) finally, subgraph based PE (GNN-PE,,, provides a graph PE for each node within the sampled subgraph, generated by applying a lightweight GNN to the subgraph's adjacency matrix with random node feature initialization (Sato al. 2021 Kanatsoulis et al., (025).
Encoders. Each element in the 5-tuple is processed by a specialized encoder before being combined into the final token representation, illustrated in Figure
1. Node Feature Encoder. The node features Tey. representing the columnar attributes of the node table row in database), encoded into d-dimensional
my in REG (which corresponds embedding. Each modality, such as numerical, categorical, multi-categorical text, and image data, is encoded separately using moodality-speciffe encoders following (Hu al. 2024). and the resulting
"When fewer than K neighbors available within 2 hops, use randomly selected nodes as fallback takens maintain the fixed size K, ensuring consistent computational complexity regardless of local structure.
5

---

Published as a conference paper at ICLR 2026
representations then aggregated into unified d-dimensional embedding.
fear (22) MultiModalEEncodrr(zyyy E Rd
where MultiModalEncoder(() unified feature encoder adapted from (Hu et al. 2024).
2. Node Type Encoder. The node type encoding steps converts each table-specific entity type (v²))
to d-dimensional representation, incorporating the heterogeneous information from the input data.
hyype (1y) = Wiype -onhht(((())) R
where (1y) is the node type of Uy. Wype E Rd is the learnable weight matrix, [7] is the number of node types, and onehot(-) is the one-hot encoding function.
3. Hop Encoder. The relative hop distance p(u;; 12), , that captures the structural proximity between the seed node U, and neighbor node vj. is encoded into d-dimensional embedding as:
Abup (U vj) hop on v2)) R
with p(u;, U2) being the relative hop distance between seed node e; and neighbor node ey, and Whop E Id gddxhu the learnable matrix mapping hop distances (up to hmax).
Time Encoder. The time encoder linearly transforms the time difference 7(U,) - T(u;) between neighbor node 1 and the seed node Uy:
Atme Stime(1s, (ti, 02) Wume (T(2)) - ru.)) R
where (u3) ((e) is the relative time difference, and Wrime RR are learnable parameters.
Subgraph PE Encoder. Finally, for capturing local graph structure that can otherwise not be represented by other token elements, apply light-weight GNN to the subgraph. This GNN encoder effectively preserves important structural relationships, such as complex cycles and quasi- cliques between entities (Kanatsoulis & Ribeiro 2024), as well as parent-child relationships (e.g.. product node within the local subgraph corresponding to specific transactions). and can be written as:
(5) where GNN(-.)) is light-weight GNN applied to the local subgraph yielding the encoding for node By Aseeal E R is the adjacency matrix of the sampled subgraph of K nodes, and Zrandom I K ddui randomly initialized node features for the GNN (with d as the initial feature dimension).
h1)( = GNN (Aloall Zrandom) Rd
One key advantage of using random node features in this GNN encoder is that it breaks structural symmetries between the subgraph topology and node attributes, thereby increasing the expressiveness of GNN layers (Sato et al. 2021). However, a fixed random initialization would destroy permutation equivariance, critical property for generalization. To address this, resample Zrandom independently at every training step. This 'stochastic initialization' approach can be viewed as a relaxed version of the learnable PE method described in Kanatsoulis et al. (2025), thus approximately preserving permutation equivariance while retaining the expressivity gains afforded by the randomization.
At last, the effective token representation is formed by combining all encoded elements:
Makeen (v²) = O [/reat (1;) (oype())) || (U;, m3) brime (Vi, 0y) (u;)
(6)
where || denotes the concatenation of the individual encoder outputs, and O E [gaddd is a learnable matrix mix the embeddings. This multi-element approach provides a comprehensive token representation that explicitly captures node features, type information, structural position, temporal dynamics, and local topology without requiring expensive computation on the graph structure.
3.2 TRANSFORMER NETWORK
The Transformer network in RELGT, shown in Fig. processes the tokenized relational entity graph using a combination of local and global attention mecham mechanisms, following the successful designs used in modern GTs (Rampášek et al. 2022 Wu al. 2023 Kong al. 2023 Dwivedi et al. 2023).
Local module. The local attention mechanism allows each seed node to attend to its K local tokens selected during tokenization, capturing the fnn-grained relationships defined by the database schema.

---

Published as a conference paper at ICLR 2026
ktyery
Attention
Local Features
Global
all- all-pair attention
Features
Conmoid
gooling
Input Nodes
Output
Transformer Network
Nodes
Figure 3: The Transformer network which processes the input tokens by first building local represen- tations using the local tokens, then incorporating global context by attending centroids that dynamically updated during training. The final node representations combine both local structural details and global database context, enabling effective prediction across downstream tasks.
This mechanism is different from GNN used in RDL (Robinson al. 2024 in two key aspects: self-attention is used as the message- -aasggn scheme and the attention is al all-pair, i.., all nodes the local K set attend to each other. This is implemented using L layer Transformer (Vaswani et al..
2017) and provides a broader structural coverage compared to baseline GNN (Robinson et al. 2024)) practical application of this improvement can be seen in the e-commerce example introduced in Sec.
I where the proposed full-attention mechanism can directly connect seemingly unrelated products by identifying relationships through shared transactions enables the model to capture subtle associations, such as customers frequently purchasing unexpected combinations of items. The local node representation ho (1/ is obtained as:
customer behaviors. This capability
Mocal U1) = (Fol(FFAAAtttntini
{u, (1)))=)))z)
where, L denotes the layers, FFN and Attention are standard components in Transformer (Vaswani et al. 2017). and Pool denotes the aggregation of {u, ) and using a learnable linear combination.
Global module. The global attention mechanism enables each seed node attend to a of B global tokens representing centroids of all nodes in the graph, conceptually and is adapted from prior works (Kong et al. 2023) Dwivedi et al. 2023). These centroids are updated during training using an Exponential Moving Average (EMA) K-Means algorithm applied to seed node features in each mini-batch, providing broader contextual view beyond the local neighborhood. The global representation is formulated as:
h hglobal (v:) Attention(y,, {c.)
(8)
The final output representation of each node is obtained by combining local and global embeddings:
houppu (e:) = FFN (h)a(a)aa))s))) Aglobal (v)))
with FFN being a feed forward network. The components of the Transformer in stages follow standard instantiations with normalization and residual connections.
For downstream prediction, the combined representation of the seed node is passed through a task- specific prediction head. The model is trained end-to-end using suitable task specific loss functions. By leveraging multi-element token representations within hybrid local-global Transformer ar- chitecture, REI.GT effectively addresses the challenges of heterogeneity, temporal dynamics, and schema-defined structures inherent in relational entity graphs.
EXPERIMENTS
RELGT evaluated on the recently introduced RDL Benchmark (RelBench) (Robinson et al. 2024). RelBench consists of 7 datasets from diverse relational database domains, including e-commerce, clinical records, social networks, and sports, among others. These datasets are curated from their respective source domains and consist wide range of sizes, from 1.3K 5.4M records in the training set for the prediction tasks, with a total of 47M training records. For each dataset, multiple predictive tasks defined, such predicting user's engagement with advertisement within the next four
7

---

Published as a conference paper at ICLR 2026
Table I: Test set results on the entity regression and classification tasks in RelBench. Best values are in bold. RDL: HeteroGNN baseline (Robinson et al., 2024), HGT: Heterogeneous GT (Hu et al. 2020), PE: Laplacian PE. Relative gains are expressed as percentage improvement over RDL baseline.
(a) MAE for entity regression. Lower is better
(b) AUC for entity classification. Higher is better.
Postanet
Yask
RDH.
HGT
IGT -PE
RelGT
(curs)
5 Rei Gaie
Dutast
Task
4.003
4.2301
4,921
X8170
Goin
2.11
ne1-F1
de daávee din
dsáves -LAp
0.220
0.2077
0.7075
02117
0.667
0.853
0.7557
4.45
0.650
06676
0/450
0.0830
164
0.660
0.632
0.6455
0.6678
0.85
Pel-pponc
0.758
112635
0.3611
0.2502
2.79
user-repeat
0.768
0.64%
06536
-3.64
scudy-adverse
Lanz-Agnorw
0.162
0.3247
0.3161
0.8157
.006
0:400
0.1263
aludyy
0.6600
0.5857
0.5521
0.086]
rel -00370n
Ste8-Jow
LUME 3..9
90163
14313
55,9985
15:4200
55.899
15.8633
14.3665
48.9221
2.26
0.32
neer-charn
Iree-churn
0.7842
038381
0.7197
0:663
0.619
0.7801
018255
0.7110
-004
post
6,965
0,0079
0.9000
0/6644
0.ME3
0.056
0.0641
0,0639
8:8566
4.29
user-baigee
0,996
0,9008
0.3166
-3.94
U267-churn
0.6988
0.6695
0.6399
0.8927
-037
days or determining whether a clinical trial will achieve its primary outcome within the next year. In total, RelBench has 30 tasks across the 7 datasets, covering entity classification, entity regression, and recommendation. For our evaluation, we focus on 21 tasks on entity classification and regression
4.1 SETUP AND BASELINES
We implement RELGT within the RDL pipeline (Robinson et al. 2024) by replacing the original GNN component, while preserving the learning mechanisms, database loaders, and task evaluators. The model has between 10-20M parameters, and we use learning rate of 1e 4. For tasks with fewer than IM training nodes, we tune the number of layers E 1, 4, and use dropout rates of 0.3, 0.4, 0.5. For tasks with more than IM training nodes, we fix the number of layers to L = 4 due to compute budgets. For the sampling during the token preparation stage, we use K = 300 local neighbors and set B 4096 the number of tokens for global centroids. For smaller datasets (under one million training nodes), we use a batch size of 256 to ensure sufficient training steps. For larger datasets, we use batch size of 1024. We do not perform exhaustive hyperparameter tuning; rather, our goal is to showcase the benefits of using RELGT in place of GNNs within the RDL framework. As shown in our ablation of the multi-element tokenization and global module in RELGT (Tab. ( and context size (Fig. 4. careful tuning may further improve performance across different tasks.
In addition the HeteroGNN baseline used in RDL. we report results for two variants of the Heterogeneous Graph Transformer (HGT) (Hu et al. 2020) to highlight the advantages of RELGT over existing GT models. Notably, many GTs, such as GraphGPS (Rampášck al., 2002), , are not directly applicable to heterogeneous graphs. Therefore, we adopt HGT and an enhanced version, HGT+PE, which incorporates Laplacian PE. These positional encodings are computed on the sampled subgraphs rather than the entire graph. Additional details are included in Appendix A.5
4.2 RESULTS AND DISCUSSION
RELGT improves over GNN in RDL. The experimental results in Tables La and lb demonsttt that RELGT consistently matches or outperforms the standard GNN baseline used in RDL (Robinson et al.,
2024) across multiple datasets and tasks. Using a +1 % threshold to assess comparable performance, RELGT shows: (i) clear improvements (more than a 1 % relative gain) on 10 tasks. (ii) comparable results (within +1%) on 9 tasks, and (iii) competitive but lower performance (more than 1% relative loss) on 2 tasks. We observe the largest improvements in rel-trial site-success (18.43%), rel-avito ad-ctz (15.85%), and rel-fl driver-top3 (10.56%), while on rel-stace ack user-badge, RELGT performs below the RDL baseline by a margin of -3.94%. For all other tasks, RELGG consistently improves or matches the performance of the baseline GNN. We attribute the overall performance improvement to two key factors: (i) the broader structural coverage enabled by RELGT's attention mechanisms as described in Section 3.2 and (ii) the fine-grained encodings employed in tokenization scheme, which are further studied as follows and presented in Table 2
We exclude recommendation tasks in this work since they involve specific considerations, such as identifying target nodes (You al. 202T) or using pair-wise learning architectures (Yuan et al. 2024b. Details in Sec. A.1
8

---

Published as a conference paper at ICLR 2026
Table 2: Relative drop (%) in performance in RELGT after removing model component. Negative scores suggest the component is critical in RELGT, and vice-versa. Full results in Table 9
No Global Module
No GNN
No Node Type
No Hop Distance
No Relative Time
Dataset
Task
PE
rel-avito
ad-ctr
6.00
-1.44
-7.14
-3.33
-9.14
rel-avito
user-clicks
7.85
-15.55
5.01
5.77
8.37
rel-avito
user-visits
0.35
2.38
0.11
0.39
0.75
xel-event
user-ignore
1.30
0.12
-0.11
0.66
-0.09
xel-trial
study-out
-2.14
-1.72
3.74
-0.43
2.48
rel-trial
site-success
19.01
9.17
2.88
21.49
0.71
rel-amazon
user-chura
0.64
-0.78
0.16
0.06
-2.00
xel-hm
item-sales
9.33
17.35
12.69
0.93
777.24
Average
3.87
5.95
1.75
2.19
9.91
Subgraph GNN PE is critical in RELGT. In Table 2 highlight the importance of several components in RELGT by conducting ablation studies. We remove one component at a time while preserving all others, and report the relative performance drop compared the full RELGT model. Our results show that removing the subgraph GNN (PE), which encodes local subgraph structure (Section 3.T). D. leads to consistent performance degradation across all tasks. This component proves critical for disambiguating parent-child relationships when full-attention is applied, thanks to the random node features initialization (Sato et al. 2021 Kanatsoulis et al. (2025). For instance, without the GNN (PE), products belonging to specific transactions (Figure) cannot be effectively captured, even when other encodings remain.
Global module can bring gains depending the task. In the same Table removing the global attention to the learnable centroids (Section(3.)) reveal task- dependent patterns that align with the findings reported in (Kong al.) 2023] Dwivedi al., 2023). For some tasks, such as rel -trrial a site-succese removing the attention to the centroids tokens leads to a substantial performance drop (-19.08%) indicating that the global database-wide context provides crucial information beyond the local neighborhood. However, for certain tasks such as rel- avvito user-clien cks, removing the global module actually improves performance (7.79% relative gain), suggesting that for some prediction targets, local information is sufficient, and the global context might introduce noise. These mixed results highlight the complementary nature of local and global information in relational graphs, with the latter being optional depending on the task.
results of
Ablation of other encodings. The remaining ablations Table reveal mixed results across different components. While removing explicit fin-grained encodings (node type, hop distance, and relative time) degrades performance on some tasks, it improves performance on others. For tasks with specific temporal dependencies (detailed in Sec. A.I. our current temporal encodings may inadvertently introduce noise. Similarly, for node type and hop distance encodings, their information might already be partially captured by other model components. Despite these variations, the full RELGT model still shows consistently superior results when averaged across all tasks. However, our findings suggest that REIGT's scores could be further enhanced by careful tuning of these encoding components based on their task-specific importance. In particular, additional gains can be achieved by incorporating more efffective temporal encodings (Clauset& & Eagle| 2012 Huang al. 2024 Jiang & Pu 2023)
HGT, a GT baseline, underperforms with significant overhead. As shown in Tables la and lb HGT (Hu al. 2020) underperforms compared to the HeteroGNN baseline of RDL (Robinson
et al. 2024) across most tasks, with only two exceptions: rel-trial study-advezse and rel-event user-i gnore. Notably. the integration of Laplacian PEs in HGT improves perfor- mance in 8 (of 21) tasks. Moreover, as illustrated in Figure 4 the computational overhead required for precomputing the Laplacian PEs substantially increases per-epoch runtime across various tasks. These empirical findings clearly reveal the difficulties of directly applying existing GT architectures to relational entity graphs, emphasizing the importance and need for our contributions with RELGT.
Local context size K. In our main RELGT experiments, we set the local context size at 300 nodes (Section 3.1, however, we study its variability in Figure 4 for context sizes K E {100, 300, 500). Although K 300 generally produces the best results, optimal values vary across specific tasks. For instance, rel- rel-avito ad-ctr benefits from a larger context size, whereas zel-tran relltriaa study-out.come achieves better performance with smaller context window. These findings

---

Published as a conference paper at ICLR 2026
Runtime Comparison: Acdioon Line 12
Ablation Context
HGT (aadinn
120
2.32
2.0k
520
2,244
1,60
Performanc
0.0
V25
AGP
diice
+380
$5:00
char
Figure 4: Left: Epoch runtime comparison of HGT Hu et al. 2020) and HGT+PE, with Laplacian PE (see Figure for all tasks). The red portion shows the additional time consumed by the precomputation of Laplacian PE against the base HGT time (blue). Right: Ablation for different K values as the local context size in RELGT. Results using K 300 serve as the baseline (100% performance), with K 100 and K
500 runs measured as % of performance relative to K 300.
suggest that RELGT's performance could be further enhanced by task-specific tuning of the context size, allowing for better model expressivity based on the structural characteristics of each dataset.
5 CONCLUSION
In this work, we introduce the first Graph Transformer designed specifically for relational entity graphs: the Relational Graph Transformer. It addresses key challenges faced by existing models, such as incorporating heterogeneity, temporality, and comprehensive structural modeling within a unified GT framework. RELGT represents nodes as multi-element tokens enriched with fine-grained graph context and combines local attention over sampled subgraph tokens with global attention to learnable centroids, enabling effective representation learning on relational data. Experiments on the RelBench benchmark show that RELGT consistently outperforms GNN and GT baselines across multiple tasks. Moreover, our analysis highlights the critical role of subgraph- lightweight and effective alternative to traditional graph positional encodings. This work establishes RELGT as powerful architecture for relational deep learning and opens new avenues for advancing and scaling such architectures toward foundation models tailored for relational data.
positional encodings as
ACKNOWLEDGMENTS
We thank Eric Chen, Shenyang Huang and Fang Wu for their helpful feedbacks and members of the Stanford SNAP group for their suggestions during the project. We also gratefully acknowledge the support of NSF under Nos. OAC-1835598 (CINES), CCF-1918940 (Expeditions), (IHBEM), 24S-2403318 (III); Stanford Data Applications Initiative, Wu Tsai Neurosciences Institute, Stanford Institute for Human-Centered AI, Chan Zuckerberg Initiative, Amazon, Genentech, Hitachi, and SAP. The content solely the responsibility of the authors and does not necessarily represent the official views of the funding entities.
REFERENCES
Uri Alon and Eran Yahav. On the bottleneck of graph neural networks and its practical implications. arXiv preprint NXX::2006.00205,, 2020.
Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković, Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint 11X1v:2104.1347,, 2021.
Semih Cantürk, Renming Liu, Olivier Appointe-Ganne Vincent Létourneau, Guy Wolf, Dominique Beaini, and Ladislav Rampášek. Graph positional and structural encoder. arXiv preprint 200.::2307.01007 2023.
Jinsong Chen, Kaiyuan Gao, Gaichao Li, and Kun He. Nagphormer: A tokenized graph transformer for node classification in large graphs. In The Eleventh International Conference on Learning Representations, 2022.
10

---

Published as a conference paper at ICLR 2026
Tianlang Chen, Charilaos Kanatsoulis, and Jure Leskovec. Relgnn: Composite message passing for relational deep learning. In Forty-second Interaational Conference on Machine Learning, 2025.
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd aom sigkdd international conference on knowledge discovery and data mining, pp. 785-794, 2016.
Aaron Clauset and Nathan Eagle. Persistence and periodicity in dynamic proximity network. arXiv preprint arXiv: 1211.733, 2012.
Edgar F Codd. A relational model of data for large shared data banks. Communications of the ACM, 13(6):377-377 1970.
Vijay Prakash Dwivedi and Xavier Bresson. A generalization of transformer networks to graphs. AAA/ Workshop on Deep Learning on Graphs: Methods and Applications, 2021.
Vijay Prakash Dwivedi, Chaitanya K Joshi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Benchmarking graph neural networks. arXiv:2003.00982, 2020.
Vijay Prakash Dwivedi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Graph neural networks with learnable structural and positional representations. In International Conference on Learning Representations, 2022.
Vijay Prakash Dwivedi, Yozen Liu, Anh Tuan Luu, Xavier Bresson, Neil Shah, and Tong Zhao. Graph transformers for large graphs. arXiv preprint arXiv:2312.11109, 2023.
Matthias Fey and Jan Eric Lenssen. Fast graph representation learning with pytorch geometric. arXiv preprint arXiv: 1903.02488 2019.
Matthias Fey, Weihua Hu, Kexin Huang. Jan Eric Lenssen, Rishabh Ranjan, Joshua Robinson, Rex Ying, Jiaxuan You, and Jure Leskovec. Position: Relational deep learning-grape graphh representation learning on relational databases. In Forty-first International Conference on Machine Learning, 2024.
Justin Gilmer, Samuel Schoenholz, Patrick Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference machine learning, pp. 1263-1272. PMLR, 2017
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama herd of models. arXiv preprint arXiv:2407.21788. 2024.
Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 855 864, 2016.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
Weihua Hu, Yiwen Yuan, Zecheng Zhang, Akihiro Nitta, Kaidi Cao, Vid Kocijan, Jinu Sunil, Jure Leskovec, and Matthias Fey. Pytorch frame: A modular framework for multi-modal tabular learning. arXiv preprint 4rXiv:2404.00777, 2024.
Ziniu Hu, Yuxiao Dong, Kuansan Wang, and Yizhou Sun. Heterogencous graph transformer. In Proceedings of the web conference 2020, pp. 2704-2710, 2020.
Shenyang Huang, Farimah Poursafaei, Reihaneh Rabbany, Guillaume Rabusseau, and Emanuele Rossi. Utg: Towards unified view of snapshot and event based models for temporal graphs. arXiv preprint arXiv:2407. 40..22299..2,,, 2024.
Yinan Huang, William Lu, Joshua Robinson, Yu Yang. Muhan Zhang, Stefanie Jegelka, and Pan Li. On the stability of expressive positional encodings for graphs. In The Twelfth International Conference Learning Representations.
11

---

Published as a conference paper at ICLR 2026
Xiangjian Jiang and Yanyi Pu. Exploring time granularity on temporal graphs for dynamic link prediction in real-world networks. arXiv preprint arXiv:2311.1225,, 2023.
Chaitanya Joshi. Transformers are graph neural networks. The Gradient, 12:17, 2020.
Charilaos Kanatsoulis and Alejandro Ribeiro. Counting graph substructures with graph neural networks. In The twelfth international conference on learning representations, 2024.
Charilaos Kanatsoulis, Evelyn Choi, Stephanie Jegelka, Jure Leskovec, and Alejandro Ribeiro. Learning efficient positional encodings with graph neural networks. In The Thirteenth International Conference on Learning Representations, 2025.
Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie- Yan Liu. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems, 30, 2017.
Thomas Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprini arXiv: oeee::.09.00000 2016.
Kezhi Kong, Jiuhai Chen, John Kirchenbauer, Renkun Ni, C Bayan Bruss, and Tom Goldstein. Goat: A global transformer on large-scale graphs. In International Conference on Machine Learning. 2023.
Devin Kreuzer, Dominique Beaini, Will Hamilton, Vincent Létourneau, and Prudencio Tossou. Rethinking graph transformers with spectral attention. Advances in Neural Information Processing Systems, 34:21618 8126299, 2021.
Veronica Lachi, Antonio Longa, Beatrice Bevilacqua, Bruno Lepri, Andrea Passerini, and Bruno Ribeiro. Over 100x speedup in relational deep learning via static gnns and tabular distillation. 2024.
Derek Lim, Joshua David Robinson, Lingxiao Zhao, Tess Smidt, Suvrit Sra, Haggai Maron, and Stefanie Jegelka. Sign and basis invariant networks for spectral graph representation learning. In The Eleventh International Conference on Learning Representations, 2022.
Andreas Loukas What graph neural networks cannot learn: depth width. arXiv preprint arXiv:1907.03199, 2019.
Qiheng Mao, Zemin Liu, Chenghao Liu, and Jianling Sun. Hinormer: Representation learning on heterogeneous information networks with graph transformer. In Proceedings of the ACM web conference 2023, pp. 599-610, 2023.
Grégoire Mialon, Dexiong Chen, Margot Selosse, and Julien Mairal. Graphit: Encoding graph structure in transformers. arXiv preprint arXiv:2106.05667, 2021.
Christopher Morris, Martin Ritzert, Matthias Fey, William L. Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe. Weisfeiler and leman go neural: higher-order graph neural networks. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium 2019.
Educational Advances Artificial Intelligence, pp.
Luis Müller, Mikhail Galkin, Christopher Morris, and Ladislav Rampášek. Attending to graph transformers. arXiv preprint arXiv:2302.04181, 2023.
A Paszke. Pytorch: An imperative style, high-performance deep learning library. arXiv preprint arXiv:191..01733, 2019.
Ştefan Postăvaru, Anton Tsitsulin, Filipe Miguel Gonçalves de Almeida, Yingtao Tian, Silvio Lattanzi, and Bryan Perozzi. Instantembedding: Efficient local node representations. arXiv preprint 1XXX:2211000692 2020.
Ladislav Rampášek, Michael Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu, Guy Wolf, and Do- minique Beaini. Recipe for general, powerful, scalable graph transformer. Advances in Neural Information Processing Systems, 35: 15:14501-14515, 2022.
12

---

Published as a conference paper at ICLR 2026
Joshua Robinson, Rishabh Ranjan, Weihua Hu, Kexin Huang, Jiaqi Han, Alejandro Dobles, Matthias Fey, Jan Eric Lenssen, Yiwen Yuan, Zecheng Zhang, et al. Relbench: benchmark for deep learning relational databases, Advances in Neural Information Processing Systems, 37:21330- 21341, 2024.
Ryoma Sato, Makoto Yamada, and Hisashi Kashima. Random features strengthen graph neural networks. In Proceedings of the 2021 SIAM international conference on data mining (SDM), pp. 333-341. SIAM, 2021.
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. The semantic web: 15th international conference, ESWC 2018, Heraklion, Crete, Greece, June 3-7, 2018, proceedings 15, pp. 593-607. Springer, 2018.
Hamed Shirzad, Ameya Velingker, Balaji Venkatachalam, Danica Sutherland, and Ali Kemal Sinop. Exphormer: Sparse transformers for graphs. arXiv preprint arXiv:2303.06147, 2023.
Maciej Sypetkowski, Frederik Wenkel, Farimah Poursafaei, Nia Dickson, Karush Suri, Philip Fradkin, and Dominique Beaini. On the scalability of gnns for molecular graphs. Advances in Neural Information Processing Systems, 37: 37:19970-19906, 2024.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems, 30, 2017.
Petar Velickovie, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pictro Lio, Yoshua Bengio, et al. Graph attention networks. stat, 1050(20):00-485557 2017.
Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S Yu. Heterogeneous graph attention network. In The world wide web conference, pp. 2022-2032, 2019.
Yanbo Wang, Xiyuan Wang, Quan Gan, Minjie Wang, Qibin Yang, David Wipf, and Muhan Zhang. Griffin: Towards graph-centric relational database foundation model. arXiv preprint arXiv:2505.05568, 2025.
Yili Wang. Htgformer: Heterogeneous temporal graph transformer. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2550 2554,, 2025.
Qitian Wu, Wentao Zhao, Chenxiao Yang, Hengrui Zhang, Fan Nie, Haitian Jiang, Yatao Bian, and Junchi Yan. Sgformer: Simplifying and empowering transformers for large-graph representations. Advances in Neural Information Processing Systems, 36:64753-64773 2023.
Marek Wydmuch, tukasz Borchmann, and Filip Graliński. Tackling prediction tasks in relational databases with llms. arXiv preprint arXiv:241 1829, 2024.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In International Conference on Learning Representations, 2019. URL https //openreview net.
pepperreveev.nee///oommmmmm fttet//formm1ddd.yi.... Am5km
Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie Yan Liu. Do transformers really perform badly for graph representation? Advances Neural Information Processing Systems, 34:28877-28888, 2021.
Jiaxuan You, Jonathan M Gomes-Selman, Rex Ying, and Jure Leskovec. Identity-aware graph neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 10737-10745, 2021.
Yiwen Yuan, Zecheng Zhang, Xinwei He, Akihiro Nitta, Weihua Hu, Dong Wang, Manan Shah, Shenyang Huang, Blaž Stojanovič, Alan Krumholz, et al. Contextgnn: Beyond two-tower recom- mendation systems. arXiv preprint arXiv:2411. 199513, 2024.
13

---

Published as a conference paper at ICLR 2026
Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhaojie Gong, Fangda pallion-parameter sequential transducers
Gu, Michael He, et al. Actions speak louder than words: Trillion parameter for generative recommendations, arXiv preprint arXiv:2402.17152. 2024.
Zaixi Zhang, Qi Liu, Qingyong Hu, and Chee-Kong Lee. Hierarchical graph transformer with adaptive node sampling. Advances in Neural Information Processing Systems, 35:21171-21183, 2022.
Jianan Zhao, Chaozhuo Li, Qianlong Wen, Yiqi Wang, Yuming Liu, Hao Sun, Xing Xie, and Yanfang Ye. Gophormer: Ego-graph transformer for node classification. arXiv preprint vrXiv:2110.1309: 2021.
Wenhao Zhu, Tianyu Wen, Guojie Song, Xiaojun Ma, and Liang Wang. Hierarchical transformer for scalable graph learning. arXiv preprint arXiv: 02866, 2023.
14

---

Published as a conference paper at ICLR 2026
Table
Dataset and task statistics from RelBench used for our evaluation.
Dutaset
Task
Tusk type
Train
#Kows of training table Validation
Test
#Uniqqu kanniies
Secraiihesst Ennny Ovestlp
useE-chuzn
classilication
4.732,555
609,792
351,855
1,585,983
88.0
classification
2.559,26
177.689
166,82
416.352
93.1
usoz-lty
regression
4,732,355
351,955
1,585,983
20L-0mazon
regression
2,707,679
166.978
178,334
427,537
93.5
user-clicke se
classification
59,654
21,183
47.996
66,449
45.3
clavsification
86,619
20,979
36,129
53,405
64.6
selnvviLo
5,100
1,766
1,816
4,997
59.8
user-repeat ggnoe user abtendencee
classification
3,842
268
246
1,514
11.5
clavsificationn
19.230
4,885
4,010
regression
19,261
2,006
9,604
14.6
driver-dutt
classiffcationn
50.0
del pei-ve-to 3g3 Gelowz posit Lon
claviffcation
1,353
588
726
$0.0
regrossion
7,453
499)
760
826
44.6
usez-chozn
classification
3871,410
70,56
74,575
1,002,984
89.7
262-bn
5,48,184
105.542
105,542
105,542
100.0
uset-cngagemant user-badge
classification
1,360850
85,838
58,137
88,137
97.4
classification
3,386,276
zol-stack
post-19t5s
regression
2,45,9211
156.216
100,903
160,903
97.1
stody-cubcom atud--ddeeee
classification
11,994
960
13,779
regrossion
43.335
3,596
3.098
50,009
0.0
Ictriall
wite auccess
regrossion
151,407
19,740
22,617
129,542
42.0
A APPENDIX
A.1 BENCHIARK DETAILS
In this section, we include the details on the datasets and the tasks in RelBench (Robinson et al. 2024) which
for evaluation. RelBench consists of 7 datasets from diverse relational database domains, including c-commerce, clinical records, social networks, and sports, among others. These datasets are curated from their respective source domains and consist wide range of sizes, from 1.3K to 5.4M records in the training set for the prediction tasks, with total of 47M training records. For each dataset, multiple predictive tasks are defined, such as predicting user's engagement with an advertisement within the next four days determining whether clinical trial will achieve primary outcome within the next year. In total, RelBench has 30 tasks across the datasets, covering entity classification, entity regression, and recommendation. For our evaluation, we focus on 21 tasks on entity classification and regression as RELGT primarily serves as a node representation learning model in RDL. We exclude recommendation tasks in this work since they involve specific considerations, such as identifying target nodes (You et al. 2021) or using pair-wise learning architectures (Yuan et al. 2024) and using REL.GT trivially in RDL is sub-optimal. We detail the dataset and task statistics in Table 3
A.1.1 DATASETS
cel-amazon. The Amazon E-commerce dataset consists of product details, user information, and review interactions from Amazon's 's platform, including metadata like pricing and categories, along with review ratings and content.
rel-avito. Avito's marketplace dataset contains search queries, advertisement characteristies, and contextual information from this major online trading platform that facilitates transactions across various categories including real estate and vehicles.
The Event Recommendation dataset from Hangtime mobile app tracks users' social planning, capturing interactions, event details, demographic data, and social connections to reveal how relationships impact user behavior.
rel-event.
The Fl dataset provides comprehensive Formula I racing information since 1950, doc- umenting drivers, constructors, manufacturers, and circuits with detailed records race results, standings, and specific data various racing sessions and pit stops.
rel-f1.
15

---

Published as a conference paper at ICLR 2026
rel-hm. H&M's dataset contains customer- customer-product interactions from their e-commerce platform, featuring customer demographics, product descriptions, and purchase histories.
The Stack Exchange dataset documents activity from this network of Q&A websites, including user biographies, posts, comments, edits, votes, and question relationships where users earn reputation through contributions.
rel-stack.
The clinical trial dataset from the AACT initiative has study protocols and outcomes, containing trial designs, participant information, intervention details, and results metrics, serving as key resource for medical research.
rel-trial.
A.1.2 TASKS
The following entity classification and regression tasks are defined in RelBench for the above datasets.
1. rel-amazon
(a) user-churn Predict whether user will discontinue reviewing products within the next three months.
(b) item-chiee :temnchurn:
Predict if a product will have no reviews the next three months. (c) user-ltv: Estimate the total monetary value of merchandise in dolloar that user will purchase and review within the next three months.
(d) item-ltv: Estimate the total monetary value of purchases and reviews product will receive during the next three months.
2. rel-avito
(a) user-visits: Predict a user will engage with several (advertisements) ads within the upcoming four days.
(b) user-clicks Predict whether user will interact with multiple ads through clicking within the upcoming four days.
(c) ad-ctr: Estimate the interaction probability for ad, assuming it receives interaction within four days.
3. rel-event
(a) user aser-attendance: Estimate the number of of events user will confirm attendance to (RSVP yes or maybe) within the upcoming seven days.
(b) user-repeat: Predict whether user will join an event (RSVP yes or maybe) within the upcoming seven days, provided they attended in an event during the previous fourteen days.
(c) user-ignore: Predict whether will disregard ignore than events invitations within the upcoming seven days.
4. rel-fl
(a) driver-dnf: Predict if a driver will not finish race within the upcoming month,
(b) driver-top3: Determine a driver will achieve a top-three qualifying position in race within the upcoming month.
(c) driver-positions Estimate driver's average finishing placement across all races in the upcoming two months.
5. rel-hm
(a) user-churn: Predict whether a customer will not perform any transactions in the upcoming week.
(b) ttem-salest Estimate total revenue generated by a product in the upcoming week.
6. rel-stace
(a) user-engagement: Predict whether a user will contribute through voting, posting, or commenting within the upcoming three months.
(b) user-badge: Predict whether a user will secure a new badge within the upcoming three months.
16

---

Published as a conference paper at ICLR 2026
Table 4: Study of node initialization in Subgraph GNN PE. Relative drop is expressed as percentage drop of using LaPEE vs. Trmdon and runtime ratio compares the time for ZapPE vs.
Zramdom-
Performance
Epoch time (m)
Rel
Runtime Ratio
Dataset
Task (# train)
MAE
Trandan
LapeE
Drop
Zrandun
LappE
Text
0.035
0.0369
-543
0.76
2.57
3.38
rel-avito
ad-otr
Val
0.0314
0.0314
Test 0.326
0.3452
-5.89
32.88
36.09
1.1
rel hel-trial
aatc-s
0.359
0.3683
Test Val
0.0536
0.0573
-6.9
49.26
53.8
1.09
rel hm
item: aales
0.0627
0.0667
Rel
Runtime Ratio
Dataset
Task (# train)
A 1 Zrandon
ZLapPE
Zyaadon
Z
Drop
Test
0.607
0.583
-3.95
6.42
7.43
1.16
user-olicka
0.656
0.6564
rel-avito
Text
0.664
0.6626
00.11
9.26
10.50
1.13
user-visite
Val
0.699
0, 0,7002
Test
0.8
0.7988
-0.15
1.85
2.77
1.5
rel-event
1ser-ignore
0.881
0.8916
Test
0.674
0,6532
-3,09
1.41
1.52
1.08
rel-trial
study-outcome
Val
0.689
0.6719
Test 0,7039
0.7044
0.07
168.00
170.55
1.02
rel-amazon
user-churn
Val
0.7036
0.7036
(c) post post-votes: Estimate the number of votes user's post will accumulate over the upcoming three months.
7. rel-trial
(a) study-outcome Predict whether clinical trial will achieve its principal outcome within the upcoming year.
(b) study-adversed Estimate the number of patients who will experience significant adverse effects mortality in clinical trial over the upcoming year.
(c) s sesscccssss: Estimate the success rate of clinical trial site in the upcoming
year.
A.2 NODE INIIIALIZATION FOR SUBGRAPH GNN PE IN RELGT
As described in Section 3.1 employ lightweight GNN PE to capture local graph structures that cannot be represented by other elements of the token, particularly the parent-chile relationships among nodes the local subgraph. The GNN is implemented as:
(v, = GNN Alocal Zrandom); gd
(10)
where GNN(:: -) is lightweight GNN applied to the local subgraph, yielding the encoding for node V/.. Here, AJscal IR. K X represents the adjacency matrix of the sampled subgraph containing K nodes, and Zrandom IR d denotes randomly initialized node features for the GNN (with dnat as the initial fcature dimension). In RELGT, we set diait 1.
The randomly initialized node features (Zrandom) ) provide enhanced properties discussed in Section 3.1 We investigate the alternative approach of using Laplacian PE (Zappr) computed over the subgraph instead of random initialization and report these results in Table 4 For these results, we utilized a positional encoding dimension size of 4. Our findings indicate that ZLapPE consistently underperforms compared to Zrandam, while also introducing additional computational overhead ranging from 1.02x to 3.38x across the 8 selected tasks in study. This shows the challenges of using existing PEs such as Laplacian PE relational entity graphs and signify the use GNN PE as part of RELGT's tokenization strategy.
A.3 LEARNABLE SPATIO-TEMPORAL PE
In this section, we explore learnable spatio-temporal positional encoding (PE) for RELGT. Instead of using the relative time encoder (Eqn. 4
the 'relative time" term initialize nodes in
17

---

Published as a conference paper at ICLR 2026
Table 5: Performance comparison of RELGT (Full) with the Temporal PE (Eqns. Negative scores suggest performance drop with the spatio-temporal PE in RELGT.
ReIGT (Full)
ReIGT
% Rel.
Dataset
Task
MAE +
(Spatio-Temporal PE)
DirT
Test
0.0345
0.0355
2.90
rel-avito
ad-ctr
Val
0.0314
0.0315
Text
0.3262
0.3554
8.95
rel-trial
te-successs
Val
0.3393
0,3883
Test
0.0536
0.0630
17.54
rel-hm
tte--sa les
Val
0.0627
0.0718
ReIGT
ReiGT
% Rel.
Dataset
Task
A
(Full)
(Spatlo-Temporee PE)
Diff
Test
0.6830
0.6465
tser-clicks
Val
0.6649
0.6519
rel-avito
Test
0.6678
0.6641
-0.55
sser-visita
Val
0.7024
0.7017
Test
0.8157
0.8152
-0.06
rel-event
iggooe
Val
0.8868
0.8870
Test
0.6861
-4.72
rel-trial
study-outcome
Val
0.6678
0.6757
Test
0.7039
0,7036
-0.04
rel-emazon
user-chura
Val
0.7036
0,7037
Table 6: RELGT results entity classification tasks in RelBench compared with Griffin 2025). AUC the performance metric. Higher is better.
(Wang et al.
RelGT (ours)
% Rel. Gain
Dataset
Task
Griffin
driver-dnf driver-top3
0.745
0.7587
1.84
rel-f1
0.825
0.8352
1.24
user-clicks
0.630
0.6830
8.41
rel ral-avito
user-visits
0.650
0.6678
2.74
rel-trial
study-outcome
0.689
0.6861
-0.42
user-churn
0.700
0.7039
0.56
rel-amazon
item-churn
0.811
0.8255
1.79
user-engagement
0.898
0.9053
0.81
rel-stack
user-badge
0.870
0.8632
-0.78
rel-hm
user-churn
0.683
0.6927
1.42
the Subgraph GNN PE, where relative time T(U;) ((;) denotes the temporal difference between
neighbor node , and seed node v;- This approach repurposes the Subgraph GNN PE as a learnable spatio-temporal PE, which is defined as:
hape (1₂) = GNN GNNaai Zyelative time); E IRd
(11)
where rj. time =(1)) (v:). The token representation, then, becomes:
Meoken (vy) = [1teat (1pp(()))) (noo((1s)) 03) || hape (vy)]
(12)
where || denotes the concatenation of the individual encoder outputs, and O E Rd is a learnable matrix to mix the embeddings.
Table presents our evaluation results over three regression and five classification tasks RelBench. Across all tasks, replacing the original temporal encoder and subgraph GNN PE encoder with the PE leads to consistent performance decline REL.G.
A.4 COMPARISON WITH GRIFFIN
In this section, compare RELGT with Griffin (Wang al. 2025), which is GNN based relational foundation model that integrates unified feature encoders, cross- attenttton and hierarchical message passing process relational entity graphs from diverse domains. We report the results in Table 6
18

---

Published as a conference paper at ICLR 2026
Runtima Comparison:
HGI (Sasalin))
Additional
2
L20K
L-M
3
1AK
Sudge
INCOBS
USD
engageewet CICS
31
G23
FRODI
Figure 5: Runtime Comparison of HGT and HGT+PE baseline. Adding the Laplacian Positional Encoding increases computational overhead, with penalties on average training time per epoch. The overhead for PE reaches up to 761 % relative to the training time of HGT on the same dataset.
for 10 entity classification tasks RelBench, where Griffin finetuned on each task following the same procedure used to train RELGT. RelGT outperforms Griffin on 8 out of 10 tasks with relative gains of up 8.41 %, demonstrating the advantages of a Graph Transformer backbone for processing relational entity graphs
A.S HGT BASELINE
In the main experiments (Section 4). we use the Heterogeneous Graph Transformer (HGT) (Hu et al. 2020) graph transformer (GT) baseline, and report results for two variants to demonstrate the advantages of RELGT over existing GT models. Specifically, we consider the standard HGT model and enhanced version, HGT+PE, which incorporates Laplacian positional encodings (LapPE). These positional encodings are computed on sampled subgraphs rather than the full graph.
For implementation, we the FGTConv layer from PyTorch Geometric (Fey & Lenssen, 2019) and integrate it into the RDL pipeline (Robinson et al.) (024)) by replacing the default GNN module. Both variants use attention heads and layers, similar the configuration of the GNN module in RDL, with residual connections and layer normalization applied between layers. For the HGT+PE variant, we use LapPE of dimension 4 for all tasks, except for rel-ama zon item-lty and rel-hm item-saless where we use dimension 2. Notably, because the relational entity graphs are heterogeneous, the Laplacian positional encodings computed multiple times for each node type, unlike the original homogeneous setting for which LapPE was designed (Dwivedi et al. 2020).
In addition the main results in Table we report per-epoch runtimes in Figure and Table We observe significant computational overhead from precomputing Laplacian positional encodings, with slowdowns ranging from 1,8x to 8.62x, highlighting the challenge of directly applying existing graph PE techniques as is to relational entity graphs, and signifying the contributions of RELGT.
A.6 DETAILED RESULTS
In Table g
we report the full results of different configurations we tuned for RELGT, particularly the smaller datasets with lesser than million training nodes. Table(pprovides the full scores for the RELGT component study in Table while Table I0 provides the supporting results for Figure 4 Finally, provide the elaborated version of the Tables and Tblin Tables and 12 respectively.
19

---

Published as a conference paper at ICLR 2026
Table 7: Relative performance drop (%) when position encoding (PE) removed from HGT+PE models and average training time per epoch of HGT and HGT+PE. Negative scores suggest the PE is critical, and vice-versa. HGT+PE consistetlly requires more training time per epoch compared to HGT without PE across all datasets.
Dataset rel-f1
Task
No PE
HGT(s)
HGT+PE(s)
driver-position
1.79
1.47
4.56
rel-avito
ad-ctr
10.73
1.63
4,00
rel-event
uscr-atterdance
-2.85
4.36
37.57
rel-trial
study-adversa
15.02
rel-trial
site-success
1.29
45.73
63.41
xel-amazon
user-ltv
3.45
73.59
106.21
rel-amazon
item-ltv
rel-stack
post-vates
0.15
191.23
528.25
rel-hm
item-sales
2.188
94.66
135.05
rel-fl
driver-dnf
0.46
2.54
6.84
rel-f1
driver-topa
23.39
0.38
1.38
rel-avito
user-olicks
3.08
11.09
24,66
rel-avitu
user-visits
-1.24
17.16
40.07
rel-tvent
user-repeat
1.93
1.35
5.16
rel-event
user-ignore
2.29
4,49
651.10
rel-trial
study-oat
-0.21
4.09
4.83
rel-amazon
user-chare:
0.29
78.56
115.53
rel-amazon
item-churt
0.20
75.51
152.06
rel rel-stack
user ungg-engattmnnt
0.52
175.16
356.07
ral-stack
user-badge
153.68
212.21
rel-hm
user-charr:
4.34
77.73
127.04
Average
-0.05
52.28
128.64
A.7 COMPUTATIONAL COMPLEXITY AND RESOURCE INFORMATION.
RELGT has O(K²,, d) complexity for local attention and O(K d) for global attention per node, where K is local context size, B is number of global centroids, and d is hidden dimension. We implement RELGT using PyTorch framework (Paszke 2019), PyTorch Geometric framework (Fey & Lenssen 2019) and adapt the codebase of relational deep learning (Robinson et al., 2024) https: //g heg.gttuuuie thub. com/ snap ataanaaaooooooooo fora//orr////ooooo reaa//eelbaeee renchh All our experiments are conducted on an NVIDIA A100 GPU server with GPU nodes.
Table 8: RELGT results using L E 1, 8 and dropout E 0.3, 0.4, 0.5 for the smaller datasets with less than million training nodes.
Task
L1
L.1
1.4
L4
L8
L8
L8
re1=F"
delive ven-post i fon (7k)
Test
Val
3.1897
4.912
3.1817
5.6131
3.3257
3.917
3.1046
1,6316
3.3352
4.0851
3.1276
4.0012
3.1589
5.5273
3.2907
5.5569
3.1843
1.6085
sel-avito ad- (5k)
Text
0.0388
0.0322
0.035
0.0666
0.038
0.0358
Val
0.0322
0.0313
0.0314
0.0314
0.0322
0.0335
0.0317
0.0322
0.0321
rel-event
aeer-attendanca (19k)
Test
0.2635
0.2618
0.2595
0.2635
0.2502
0.2543
0.2584
0.2635
0.2618
0.2637
0.2599
0.2638
rel-crial
scudy-adveree (13k)
Test
Vil
46.3538
44.8553
44,2260
46.3193
46. 2056
44,848
44 833889
1031
44.4310
45.9498
43,9923
46.2148
44,2245
46.1804
44,5873
46.1381
44,5013
46.4332
site-success (1518)
Text
Val
0.3493
0.3490
0.3455
0.3550
0.3830
0.3771
0.4019
0.392
0.3593
0.3262
0,3783
0,3848
0.3643
0.3431
0000
Dataset
Task train)
AUCT
0,3
1.1
1.1
1.1
0.3
1.4
1.4
1.4
0.3
1.8
0.4
1.8
05987
5440
rel-El
drivez-top3
drivez-dat (11k)
Test
Val
Val
0.7775
0.7845
0.6877
0.6761
0.8203
0.783
0.7764
0.7521
0.6996
00000
00400
0.79
0,6788
SXXS
108000
Test
0.6524
0.6233
0.6212
0,0067
0.5893
0.596
0.6245
08003
rel-avito
user-clicke (59))
(861)
Text
Val
0.6627
0.6649
0.7005
0.6616
0.6663
0.6501
0.6615
0.6564
0.6954
0.6008
0,6884
0.6579
0.6647
0.6587
0.7024
0.66:49
0.6645
117.10
19/44
0,7400
rel-event
uaer-repeat (3k)
Test
Test
Vil
0.6981
0.7172
0.7403
0.7386
0.7452
0.7319
0.7563
0.7245
0.799
0.7236
0.7207
0.7432
0.736
0.7285
0.7609
0.8076
0.7316
0.7209
6,706
Val
0.8739
0.8721
0.8729
0.878
0.8731
0.881
0.8757
0.8801
0.8868
vel-trial scudy--utcoom (11k)
Text
0.6808
0.6753
0.6837
0.6488
0.6818
0.6744
0.6861
0.6562
036649
0.6815
0.6737
20

---

Published as a conference paper at ICLR 2026
Table 9: Relative drop (%) in performance in RELGT after removing a model component. Negative scores suggest the component is critical in RELGT, and vice-versa.
Datasel
Task train)
MAE
RaGT (Full)
BeGT (No Global)
Drop
Rel.
RdGT (Ne GNN)
% Rd.
Drup
(No Dype
BaGT
% Ret.
Drup
ReGT (No Hop)
5 ReL.
Drop
(No Tims)
BeGGT
Drop
Rel.
rel-avste
Ad-ese
Tex
Val
0.014
040350
00121
011771
-6.0
0.0155
0,054
-1.14
00175
00328
-7.14
010663
0,022
-3.63
00337
-9.14
cel taal
site SMccess
Toa
0.3262
0.991
0,3882
0.3127
-9001
0.3561
0.W63
9.17
0.3556
-288
0,396
0.961
21.40
0.3285
0.3515
.0.71
18
Text
00010
0.0627
0.0676
9.33
0.073
-7.33
00N4
0.0596
12,00
0A831
0,95
0.1025
-77.4
Datasel
Task train)
ABC
RelGT
(Fall
(No Global)
RulGT
Drop
Rvl.
(Ne GNN)
RoGG
% Rel.
Doup
(No Dype)
RelGT
5 Mol.
Drup
(No Hap)
ReGGT
% Med.
Drop
(No Timec)
RelGT
Drop
Rel.
rel avito
user clicks
usct-viila
Tex
Tiaa
Val
Val
0.8364
0.9642
0.6067
0.099
0.6892
0164
0.6610
0.6543
4035
7.55
0.6484
0.6511
0.6879
0.5148
15.15
-2.38
0.637]
0.6991
0.6599
06633
-0.11
5.01
0.7016
0.6417
0.6688
0.643
5,77
0.50
0J6575
0U7005
0.6579
016993
40.75
8.37
rel event
userigtore
Test
0,8002
0,8775
0.78m8
1.3
0.6637
0.8012
0.12
0.7993
0.11
0,8855
0.06
0.7995
0.8789
0,09
cel rel-triad
staly-nutoomm
0.6744
0.064
0.66
0.6775
0.6628
3.74
0.715
0.991
2.48
cel-anozon
user-ohuzn
Table 10: Ablation of context size K in RELGT.
RELGT K-100
RELGT K=300
RELGT
Dataset
Task (# train)
MAE
K-500
Test
0.0375
0.0374
0.0351
sel-avito
ad-otr
Val
0.0329
0,0319
0.031
0.3739
0.3674
0,3842
zel Mil-rrial
silersuccess
Val
0.3708
0.372
0.376
rm1-hm
Stan-sales
Test
0.055
0.0532
0.052
Val
0.0643
0.0619
0.061
RelGT
ReGTT
Dataset
Task (# train)
AUC h
K=100
K-300
K=500
user-clicks
Test
0.6628
0.6491
0.6334
zel-avito
Val
0.64337
0.6622
0.6632
Test
0.6664
0.6653
0.6627
usar-vi aitt
Val
0,7013
0,701
0,7005
rel-event
user-ignore
Test
0.7674
0.8105
0.8068
Val
0,8682
0.8853
0.8843
sel Urial
sLudy ouLcome
Test 0.7078
0.6375
0.6526
0.6877
0.666
Test
0,7038
0,7054
0,7043
rel-smazon
user-churn
0.7033
0,7042
Table 11: Results on the entity regression tasks in RelBench. Lower is better. Best values are in bold. Relative gains are expressed as percentage improvement over RDL baseline.
RDL
HGT
RelGT (ours)
% Rel. Gain
Dataset
Task
MAE
Baseline
HGT
rel-El
driver-position
Test
4.022::0.199 4.2261110.080
3.193:0.024
3.1543=00.
4.3921
3.111620.112)
3.9100 3448
3.3257=0.5618
2.61
rel-avito
ad-str
Test
Val
0.041+00.001
0.037.10.000
0.0462:400002
0.0039.0.00019
0.0483=0.0027
0.044410.0024
0.0014:0.0010
0.0345=000009
15.85
Test
0.258=0.006
0.2635 0.0000
0.261120.0045
0.2502=000033
2.79
rel-event
user Lennaanee
Val
0.255+0.0077
0.2616-100000
0.2603-0.0020
0.2548-0.0018
04.279
45.1692.1 2.6927
42.6484
43.992111.5922
study-adverse
Val
46.29 46.290=0.300 304
47.391
17.3913-t:11 1,996
45.7910=00000
46 46.2148:0072210
rel-trial
Test
0.400-000020
0.4428
0.4396-00.0083
0.3263-0.0300
18.43
site-success
1..009 Val
0.4275 0.0062
0.4292.L0.0069
0.3593
Test
14.013
15.4120=+0.040 0.447
15 31.3643300090 0924
14.2665=0001544
0.32
usee-ILv
Val
12.132-0.007
13.2295 +. 1402
13 48.6 0.07733
12.0151
re1-amazon
Ltem-
Test
50.053-0.163
55.8683-10.66
55.8493.00 10..33236
48.9222-107700
2.26
Val
45.140=00068
51.0303-10.222
50.6522=40.6141
43.8161=000048
Test
0.005::00000
0.0679-10.0000
0.0680=00.0000
0.0654:0.0002
0.62
rel-at rel-atack
post-votes ea
Val
0.00900.0000
0.0000
0.06171000000
0.0592::0.0002
rel-bm.
Liem-sales
Val
0.056::0:0000
0.005000.000
0.07:39.000000
0.00412000012
0.062920.0000
0.07399:0.0010
0.0636:000006
0.0627 0.0008
21

---

Published as a conference paper at ICLR 2026
Table 12: Results on the entity classification tasks in RelBench. Higher better, Best values are in bold. Relative gains are expressed as percentage improvement over RDL baseline.
RDL
HCT
RelGT
& Rel.
Dataset
Task
AUC
Buseline
HGT
(ours)
driver-duL
Test 0,7262=+0.0027 0:7077+0.0153
0,7117=0.0084
0.7587=0.0043
4,48
0.6804:10.0420
r81-#11
driver-Lop3
Test
0.7554.10.0663
0.707510.1156
0.7627-10. 1390
0.8352::0.0342
10.56
0.776420.0316
0.6457:000147
0.618620.0287
0.795810.0513
Test 0.6890:1.0115
0.6376100.020
0.457:10.0099
0.6830
3.64
user-c11ckg
0.588610.0231
sel-avito
Test
0.6620:000010
0.643220.0002
0.6295=0000222
0.6678:00.0155
0.88
user-visita
Val
0.6965:000004
0.665240.0040
0.6649:000009
0.702410.0009
Test
0..0599
0..266000220
0.536:20.0137
0.70091:00029
-1.04
user-repeat
0.7125
0.608210.01488
0.6148::.00122
0.7285-:000108
rel-event
Test
0.8162:40.01
0.247:000009
0.8161 0001
115575000004
-0.06
user-ignore
Val
0.9170=20.0033
0.89970000114
0.894020.0000
0.8868:20.0032
Test
0.686020.0101
0.583720.0141
0.3921=0.0303
0.6861
0.01
vel-trfal
study-outcome
Val
0.6818:20.0049
0.603740.0040
0.6025:0.0071
0.6678:1000038
Test
0.7042:20.0005
0.66432:00041
0.6619:20.0422
0. 0.703910000008
+0.04
user-churn
Val
0.0045=000006
0.668020.0029
0.6652=0.000 0030
0.70362200000
sel-amszon
Test
0.8281:000000
0.7797+000039
0.7803=00 0038
0.8255++0.0000
00.3
iten-churn
Val
0.823910.0002
0.781610.0031
0.700310.00030
0.82201-0.0010
Test
0.0021+0.0007
0.844720.004
0.8817=0.000 0046
0,9053:0.000
0.35
rel-stack
engggemnggmmnnt
Val
1.9059-+0.0009
0.88611-000339
0.8811+0.002
0.9033---00003
Test
0.8986.10.0008
0.8608 0.8608.10.044
0.856610. 0.006
0.8632.10..0118
user-badge
Val
0.8886:10.0008
0.87321:00025
0.8710:0.0016
0.874]::0.0000
Test
1L6988--.000
0.6695--00067
0.6569=0.0000
0.6927--0.0019
-0.87
re1-hm
user-churn
Val
0.7042.10.0009
0.672710.00062
0.6605
1.6003410.0034
22


Reviews:
[{"review_id": "cOpWfBsGEc", "reviewer": "Reviewer_1WrR", "summary": "This work propose a new graph transformer architecture, RelGT, for RDB data.  By designing new positional encoding, add temporal information, and using structural feature extractor, RelGT achieves strong performance on RelBench.", "questions": "1. In RelGT, each node is tokenized to 5 tokens. Will it leading to significantly larger computation overhead compared with baselines?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Detailed ablation study verify the effectiveness of model. Tables provides ablation results of all model components on all datasets. These results show consistent performance gain of designs in this work.\n2. Careful architecture design. Table 3 provides architecture illustration in detail.", "weaknesses": "1. Recent strong RDB baseline[1] is missing. Including it may make its contribution more clear.\n2. RelGT is trained on each dataset separately, with no pretrain and transfer learning experiments, making the architectural design contribution less significant. \n\n[1] Yanbo Wang, et al. Griffin:Towards a graph-centric relational database foundation model. ICML 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "r4WWfaRJnH", "reviewer": "Reviewer_ksWe", "summary": "The paper introduces Relational Graph Transformer (RelGT), a graph-transformer architecture tailored for relational deep learning (RDL), where multi-table relational databases are represented as heterogeneous temporal relational entity graphs (REGs). The core idea is a multi-element tokenization that decomposes each node into five components: node features, node type, hop distance from a seed node, relative time, and a subgraph positional encoding obtained by a lightweight GNN run on the sampled local subgraph—so that heterogeneity, temporality, and local topology are captured without heavy global precomputation. These tokens feed a hybrid local-global transformer: local attention over a sampled K-hop neighborhood plus global attention to a small set of learnable centroids updated with EMA K-means during training, yielding both neighborhood-level and database-wide context for prediction heads.", "questions": "1. Attribution of gains to token elements vs. architectural bias remains partially confounded. Table 2 indicates that removing subgraph PE or the global module reduces accuracy on average, but it is unclear whether RelGT’s advantage primarily comes from injecting structural bias via the GNN-based PE rather than the transformer’s attention. The fairness of comparing to HGT(+PE) without, e.g., GraphGPS-style structural encodings or stronger learned PEs on sampled subgraphs is uncertain.\n2. Have you tried replacing the linear time difference with learnable spatio-temporal PEs?\n3. The paper states it avoids exhaustive tuning; however, there are no multi-seed confidence intervals, paired tests, or time-to-target curves, despite notable per-task variability.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The multi-element tokenization is a clean, compositional alternative to “one-shot” global PEs, explicitly encoding heterogeneity, relative structure, and time, plus a lightweight subgraph PE. This is well-motivated for relational graphs and distinct from standard GTs or HGT variants.\n2. Establishing a transformer baseline that consistently competes with or surpasses RDL’s hetero-GNN baseline on RelBench is meaningful; the design choices are broadly applicable in enterprise REG settings and could inform future pretraining/foundation models over relational data.", "weaknesses": "See Questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aKOS7xO0lw", "reviewer": "Reviewer_Y8Ba", "summary": "The paper introduces Relational Graph Transformer (RELGT), a graph transformer architecture for relational tables. \nRELGT employs a multi-element tokenization strategy that includes five components (features, type, hop distance, time, and local structure). The method shows improved performance on a wide range of experiments, tested on the RelBench benchmark", "questions": "1. How do the challenges that you mention in section 2.1 fundamentally differ from the challenges for temporal knowledge graphs? Do you agree with my satement at W1?\n2. Why were relation types excluded from the token representation?\n3. How sensitive is RELGT to the 2-hops sampling assumption?\n4. Were experiments repeated with fixed random seeds , and or multiple runs to estimate variance?\n5. Would it be possible to integrate an interpretability analysis?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. Clear motivation, good explanation of problem\n\n2. Architecture is well-explained, the different modules are well motivated and justified, and an ablation study analyses their benefits. Each component is linked to a specific challenge in relational deep learning, which is very good.\n\n3. Extensive experiments: The evaluation is broad and the results are convincing. That said, as I am not an expert in all recent baselines for this specific area, I cannot definitively assess the completeness of the comparisons.\n\n4. Good writing and clarity.\n\n5. The work successfully leverages and extends the principles of Graph Transformers to handle the challenges of relational data.", "weaknesses": "### 1. Relation to Temporal Knowledge Graphs (TKG)\n* There is no proper distinction between relational entity graphs and TKG (see e.g. TGB 2.0), especially in section 2.1 challenges: while I agree that relational entity graphs are difficult and challenging, and different to conventional graph data, in this section, the difference to temporal knowledge graphs is not clarified in my opinion.  in my opinion, the difference are mostly in the entity-specific attributes - do you agree? if yes,  it would be good to mention and clarify.\n\n###  2. Integration of relation types:\n* The formal definition of the REG includes relation types (psi), but these are not explicitly part of the five-element token representation. Why were edge relation types excluded from tokenization or the attention mechanism? A relational encoder or relation-aware attention might improve expressivity.\n\n###\t3. Choice of hop distance (2 hops) not motivated\n* The tokenization is restricted to 2-hop neighborhoods, but this hyperparameter is not justified. Figure 4 ablates K (number of neighbors), but not the hop radius. Is RELGT sensitive to this parameter?\n\n### 4. Figure 2 clarity:\n* The left part of Figure 2 (“seed nodes with local neighbors”) is confusing to me. It is unclear how the nodes are sampled from the relational entity graph, particularly given the temporal-aware 2-hop sampling strategy, vs the fact that in each line you see 6 (plus …) nodes, where some of them are > 2 hops from the black node. A more explicit visual or step-by-step example would help readers follow the process.\n\n### 5. Experimental infos missing\n* The paper does not mention the random seed setting (fixed?) and the number of repetitions, and variance across runs.\n\n### 6. Potential idea for improvement: Interpretability analysis\n* Given that the model is relatively complex, some insights on interpretability, e.g. examples visualizing attention or token importance, would be very valuable.\n\n## Minor:\n### 7. Figure formatting\n* Figures 2 and 3 have small font sizes, which makes them very hard to read.\n\n### 8. Reproducibility, LLM usage\n* The reproducibility statement and LLM usage are missing.\n\n## Overall comment\nThe paper presents a valuable contribution to relational deep learning. The authors apply the concept of graph transformers, and modify and extend them to tackle the challenging task. The paper is written well, and the evaluations make sense. The ablation studies are good, the structure as well.\nThe paper could benefit from clarifying above questions, e.g. differentiation from TKG, improving the figure, and explaining the motivation for excluding the relation type.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "l1HjpWelJ1", "reviewer": "Reviewer_nFvD", "summary": "This paper introduces RELGT (Relational Graph Transformer), a novel Transformer architecture designed specifically for Relational Deep Learning (RDL),  i.e., learning directly from multi-table relational databases represented as heterogeneous temporal graphs. The model tokenizes each node into five elements: features, type, hop distance, time difference, and subgraph-based positional encoding, aiming to capture heterogeneity, temporality, and structural complexity. RELGT combines local attention (over sampled subgraphs) and global attention (to learnable centroids), and is evaluated on 21 tasks from the RelBench benchmark, outperforming heterogeneous GNNs (HeteroGNN) and HGT baselines by up to 18%.", "questions": "- How does RELGT differ concretely and conceptually from existing Heterogeneous Temporal Graph Transformers?\n- Can the authors clarify what they mean by “schema-defined structure” and how it is different from the one of other heterogeneous graph datasets like IMDB, DBLP, ACM and why this structure requires a special architecture?\n- How is the subgraph GNN PE trained and does it introduce additional computational cost?\n- Why other multi-relational GNN models [5-8] have not been tested even if they should be able to work in this multi-relational setting?\n- Could the authors quantify the scalability and training time of RELGT compared to RDL and other competitors?\n\n\n[5] Schlichtkrull et al., Modeling relational data with graph convolutional networks\n\n[6] Yuet al.,  Heterogeneous graph representation learning with relation awareness.\n\n[7] Zhu et al.,  Relation structure-aware heterogeneous graph neural network.\n\n[8] Ferrini et al., A Self-Explainable Heterogeneous GNN for Relational Deep Learning", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "- Ambitious motivation: Addresses an important gap, the adaptation of Transformers to relational databases, an increasingly relevant setting (RDL).\n- Unified treatment of temporal, heterogeneous, and structural properties within a single architecture.\n- Comprehensive benchmark evaluation on RelBench (21 tasks), with reasonable performance improvements and ablations to justify design choices.\n- The multi-element tokenization idea (features, type, hop, time, subgraph PE) provides a structured approach to encode complex relational contexts.\n- The local–global hybrid design aligns with recent scalable GT approaches and avoids expensive positional encoding precomputation.\nAblation studies are thorough and help identify which token elements contribute most.", "weaknesses": "- Novelty is overstated.\n   - Similar approaches already exist, e.g. Heterogeneous Temporal Graph Transformers and other models combining subgraph tokenization with structural encoders. The paper presents RELGT as “the first GT for relational graphs,” but other prior works [1-3] already tackle heterogeneity with a transformer .\n   - The tokenization strategy appears as a straightforward concatenation of known encodings (type, hop, time, PE), rather than a fundamentally new design.\n  - The subgraph GNN PE encoder appears conceptually very similar to the approach proposed in [4], as also acknowledged by the authors.\n\n- Conceptual gaps / unclear definitions.\n   - The notion of “schema-defined structure” in 2.1 is not clearly formalized. Other heterogeneous graphs (DBLP, IMDB, ACM) also have schema-defined relations, so this does not uniquely characterize relational data.\n\n- Overstatements and misleading claims.\n   - The paper states that RDL GNNs outperform LightGBM on all tasks, while the original RDL paper shows a few opposite results.\n   - Although the abstract claims that RELGT “consistently matches or outperforms” GNN baselines, this statement is not supported by the reported results. In Table 1b (entity classification), the proposed model performs worse than the RDL baseline in about half of the tasks, with gains often close to zero and only a few cases showing meaningful improvements. In Table 1a (regression), the trend is more favorable to the proposed model, although it would be more informative to report the gains in absolute rather than percentage terms.\n   - In Section 4.2, the authors state that HGT underperforms even compared to RDL, and they provide a time complexity analysis showing that adding Laplacian Encodings (LE) to HGT improves performance but at the cost of higher runtime. However, what is missing is a temporal (runtime) analysis of the proposed model compared to all competitors. Moreover, the HGT method has been improved in paper [3], specifically addressing the runtime efficiency issue, but this variant was not considered, making the temporal analysis incomplete.\n   - Furthermore, the reported results (of model and competitors) do not include mean and standard deviation across multiple runs, which are crucial to assess statistical significance and reproducibility of the claimed improvements, as explained in [4].\n\n- Technical novelty and clarity.\n   - Most encoders (feature, type, hop, time, PE) are standard components and not surprising.\n   - The model essentially applies a Transformer on subgraph samples with concatenated encodings, which is conceptually simple and not theoretically grounded.\n   - Lack of formal analysis of computational complexity, scalability, or positional encoding stability.\n\n- Missing broader context.\n   - No discussion on relational inductive bias or how RELGT generalizes to unseen tables or relations.\n   - No exploration of few-shot or inductive RDL settings where Transformers could excel.\n \n[1] Wang et al. HTGformer: Heterogeneous Temporal Graph Transformer\n\n[2] Hu et al, Heterogeneous Graph Transformer\n\n[3] Yun et al, Graph transformer networks: Learning meta-path graphs to improve gnns.\n\n[4] Demšar et al., Statistical Comparisons of Classifiers over Multiple Data Sets", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)