SyntheRela: A Benchmark For Synthetic Relational Database Generation
Authors: Valter Hudovernik
Martin Jurkovic
Erik Štrumbelj
Venue: Accepted by TMLR ()

Abstract: Synthesizing relational databases has started to receive more attention from researchers, practitioners, and industry. The task is more difficult than synthesizing a single table due to the added complexity of relationships between tables. For the same reasons, benchmarking methods for synthesizing relational databases introduces new challenges. Our work is motivated by a lack of an empirical evaluation of state-of-the-art methods and by gaps in the understanding of how such an evaluation should be done. We review related work on relational database synthesis, common benchmarking datasets, and approaches to measuring the fidelity and utility of synthetic data. We combine best practices, a novel robust detection metric, and a novel approach to evaluating utility with graph neural networks into a benchmarking tool. We use this benchmark to compare 6 open-source methods over 8 real-world databases, with a total of 39 tables. The open-source SyntheRela benchmark is available on GitHub with a public leaderboard.

Published in Transactions on Machine Learning Research (02/2026)
SyntheRela: A Benchmark For Synthetic Relational Database Generation
Valter Hudovernik', Martin Jurkovië', Erik Strumbelji Faculty of Compuler and Information Science, Universily of Ljubljana "Corresponding author: erik. www.mbelj@@fri..oo..
Reviewed on OpenReview: https: // openrevee.. net/ forun? id- Mc8Xioazky
Abstract
Synthesizing relational databases has started to receive more attention from researchers, practitioners, and industry. The task is more difficult than syuthesizing single table due to the added complexity of relationships between tables. For the same reasons, benchmarking methods for syuthesizing relational databases introduces new challenges. Our work is motivated by a lack of an empirical evaluation of state-of-the-art methods and by gaps in the understandin anding of how such an evaluation should be done. We review related work on relational database synthesis, common benchmarking datasets, and approaches to measuring the fidelity and utility of synthetic data. We combine best practices, a novel robust detection metric, and novel approach to evaluating utility with graph neural networks into benchmarking lool. We use this benchmark to compare 6 open-source methods over 8 real-world databases, with a total of 39 tables. The open-source SYNTHERELA benchmark is available on GitHub with a public leaderboard.
Data & Code: github. gttatb...o//arrriuupuppo///////s
Leaderboard: buggiiaggaee...oppeee////aaaaaaa
1 Introduction
Synthesizing relational databases generating relational databases that preserve the characteristics of the original databases an emerging field. It promises several benefits, from protecting privacy to addressing data scarcily, while preserving the complexity and dependencies present in the original databases. This makes it attractive for healthcare (Appenzeller al. 2022), finance (Assefa al. 2020). and education (Bonnéry
et al. 2019). where accessing and utilizing data be challenging due to privacy concerns, data scarcity,
biases Ntoutsi et al. 2020 Rajpurkar et al. 2022).
The foundations of synthesizing relational databases were laid by the Synthetic Data Vault Patki et al. 2016). Recently, several deep learning methods have been proposed (Gueye et al. 2023 et al. 2024 Mami
et al. 2022; Xu et al., 2023) Canale et al.] 2022 Solatorio & Dupriez 2023 Pang et al. 2024 Hudovernik 2024 The field has also received attention from the industry, with several commercial tools now available and with Google, Amazon, and Microsoft integrating them into their cloud services (Gretel.ai 2024).
Although there are several packages for evaluating the quality of synthetic tabular data, only the SDMetrics package (Patki et al., 2016) provides some support for the evaluation of synthetic relational databases. As such, the field lacks not only an empirical comparison of available mothods but also an understanding of how such an evaluation should be done. We address this gap with an evaluation methodology that combines established evaluation metries (Section 2.2), best practices, sampling procedures, and real-world relational databases (Table D. We also propose baaeetion-based metric C2ST-Agg (Section 3.1 specialized for
* Eqqual contribution.

---

Published in Transactions on Machine Learning Research (02/2026)
SyntheRela
Synthetic Data Diagnostics
Benchmark
G
Synthetic
Fidelity Evaluation
Data
SyntheRela
SyntheRela Leaderboard
Relational
O
Relational Generative
RDL Utility
Metadata
Methods
Figure 1: SYNTHERELA enables the evaluation of synthetic relational database generation methods using real-world relational databases. SYNTHERELA supports the evaluation fidelity using established metrics and relational dotection-basee metric, C2ST-Agg. It also assosses utility via relational deep learning (RDL) utility. The benchmark provides synthetic data diagnostics using ML explainability and leaderboard for comparing method performance.
relational databases and use ML explainability to diagnose issues with synthetic data generation methods, and novel approach evaluate the utility of synthetic relational databases with relational deep learning utility (Section 3.2
We implement the methodology in SYNTHERELA, a benchmark and evaluation tool that is available as an open source package and can be easily extended with new metrics and datasets (see Appendix [ Finally, we use the benchmark to evaluate current stato-of-the-art methods (Section 2.1 over several relational databases. This is the first comprehensive evaluation and comparison of methods for the syuthesis of relational databases and provides valuable insights into their ability to synthesize relational aspects of the data (Section 4 We release the code of our benchmark under an open source license alongside public leaderboard.
2 Related Work
The quality of synthetic data consists of three core aspects: fidelity, wtility and privacy (Snoke et al. 2018 Beaulieu-Jones et al.] 2019 Qian et al. 2023a). Fidelity captures how much the synthetic data resembles the statistical properties of the real data. Utility reflects how well the synthetic data can replace the original in downstream tasks. Privacy measures the degree of protection of the identities the original subjects. is typically defined through the theoretical framework of differential privacy (DP) (Dwork 2006) or the notion of membership disclosure (El Emam et al. 2022ah. Our evaluation is focused on the quality of synthetic data (fidelity and utility) while ensuring it does not compromise the privacy of the original subjects.
2.1 Methods for Synthesizing Relational Databases
In this work, we focus on relational databases collection of tables linked by primary and foreign keys. We distinguish this from synthesizing tabular data (a single-table), which is special case and even more active field [Borisov et al.) 2022 Qian et al. 2023b). Existing methodologies for generating synthetic relational data can be broadly categorized into two main approaches: neural network based methods and marginal-based methods. We provide a concise summary of each below.
Patki al. (2016) are the first to propose a method for relational database synthesis The Synthetic Data Vault (SDV), which employs Gaussian copulas with hierarchical modeling algorithm and does not fit into either of these two main categories.
2

---

Published in Transactions on Machine Learning Research (02/2026)
Neural network-based methods aim to preserve data fidelity and utility across arbitrary schemas. Row Conditional-TGAN (RCTGAN) (Gueye et al. 2023) and Incremental Relational Generator (TRG) (Li et al. 2024) are based on GANs. The Realistic Relational and Tabular Transformer (REaLTabFFrmerr) (Solatorio & Dupricz 2023 and Composite Generative Models (Canale et al., 2022) are based on transformers. The work of Mami el al. 2022) is based on graph variational autoencoders, while Xu el. al. (2023) propose a framework for synthesizing many-to-many datascts using random graphs. Pang et al. (2024) propose ClavaDDPM, a method based on classifierguuieed diffusion models. [Hudovernik] (2024) proposes RGCLD, a method based graph neural networks (GNNs) and conditional diffusion models. Recently, Tiwald et al. (2025) propose TRGN framework for tabular and relational data synthesis based on autoregressive tabular models.
Marginal-based methods, by contrast, involve taking low-dimensional measurements of the dataset, such as marginal queries, and adding calibrated statistical noise to ensure DP. While well established in the
single-table setting (Zhang et al., 2017 McKenna et al., 2022 Cai et al. 20210, extensions to relational data are more recent. PrivLava (Cai et al., 2023) models inter-table dependencies using graphical models with latent, variables. MARE (Kapenekakis et al. 2024) largets medical relational datasets, such as electronic health records, with temporal and sequential characteristics. More recently, Alimohammadi et al. (2025) proposed a method for adapting single-table DP generators to relational data which models foreign keys by learning a bi-adjacency matrix. Finally, PrivPetal et al. 2025) synthesizes relational data by modeling flattening relational databases in an efficient manner. These approaches generally prioritize privacy and query accuracy over flexibility and general fidelity.
Our evaluation in this work will focus on neural network-based methods for the generation of synthetic relational data. This focus is driven by the fact that their implementations are generally applicable to generating synthetic data for arbitrary relational schemas. In contrast, marginal-based methods often include implementations tailored towards specific datasets with predefined query workloads. We provide a more detailed overview of related work in Appendix A
2.2 Metrics for Evaluating Synthetic Data
The quality of synthetic tabular data and relational databases is evaluated primarily based on two criteria, fidelity and utility. Fidelity measures the degree of similarity between synthetic and real data in terms of its properties, while utility mcasures how well synthotic data can be used in place of real data when the data are part of downstream task, for example, predictive modeling (Hansen et al. 2023). The utility of synthetic data is typically assessed with the train-on-synthotic evaluate-on-real methods (Beaulieu-Jones et al. 2019 We further divide fidelity metrics into slatistical, dislance-basel and dedection-bused metrics.
Statistical fidelity methods are typically used to assess marginal distributions, sometimes bivariate distributions. The most commonly used mothods are the Kolmogorov-Smirnov test and the X² test for numerical and categorical variables, respectively. For relational data, cardinality shape similarity is used, where for each parent row the number of child rows is calculated. This yields a numerical distribution for both real and synthetic data, on which Kolmogorov-Smirnoc test is performed.
Similar to statistical fidelity, distance-based fidelity is typically used to assess the quality of marginal dis- tributions. However, some distance metries also assess entire tables. Commonly used distance-based methods are total variation distance, Kullback-Leibler divergence, Jensen-Shannon distance, Wasserstein distance, maximum mean discrepancy, and pairwise correlation difference. To evaluate inter-table relationships Pang et al. (2024) use k-hop similarity, where they compute correlations between tables at distance (0-hop refers to columns within the same table, 1-hop refers to columns in tables directly connected via a foreign kcy, etc.). Unlike statistical methods, reports of distance-based fidelity do not include hypothesis testing or any other quantification of uncertainty. This is an issue both when evaluating a method and when comparing two methods. In the former, method can achieve scemingly high distance that is in a high probability region when taking into account the sampling distribution. In the latter, a seemingly large difference between the two methods can be explained away by the variance of the sampling distribution.
The basic idca of batection-based fidclity is to learn a model that can discriminate between real and synthetic data. The betection-based metric can be interpreted as mull-hypothesis test for comparing two distributions (two sample testing) with elassification accuracy as proxy Kim et al. (0211) The classifier
3

---

Published in Transactions on Machine Learning Research (02/2026)
serves map from high-dimensional data to one-dimensional test statistic. In machine learning literature, this is referred to as classifier two sample test (C2ST) (Lopez-Paz & Oquab 2017). Tf the model can achieve better-than-random predictive performance, this indicates that there are some patterns that identify synthetic data. Zein & Urvoy (202)) show that using appropriate discriminative models can highlight the differences wetween real and synthetic tabular data.
The most common detection-based metric is logistic detection (LD) (Gueye et al, 2023 Solatorio & Dupriez, 2023 Li et al. 2024 Pang et al.) 2024), where a logistic regression model is used for discrimination. An extended version of LD known as parent-child logistic detection (P-C LD) is used to evaluate relational databases. P-C LD applics LD to denormalized pairs of synthetic parent and child tables, assessing the preservation of parent-child relationships. A serious issue with denormalization is that it may introduce correlation between rows, breaking the í.í.d. assumption. This results in an over-performanc of the discriminative model and in underestimating the quality of the method for synthesizing relational data. It. also makes impossible to set a detection threshold for testing fidelity (for example, accuracy would be greater than 50% even if both datasets were from the same data generating process). For these reasons, we do not consider P-C detection.
Note that logistic regression is unable to capture interactions between columns unless these interactions are explicitly included as features. This implies a lenient evaluation of the state-of-the-art methods (we demonstrate this empirically in Appendix C.3 Tree-based ensemble models are a better alternative, which is also suggested by the findings of Zein & Urvoy (2022) for tabular data.
The utility of synthetic data is most commonly measured with machine learning efficacy comparing the hold-out performance of predictive model trained on the original data with predictive model trained synthetic data (Canale et al., 2022 Li et al., 2024 Mami et al. 2022 Solatorio & Dupriez, 2023 Pang et al. 2024).. Patki et al. (2016) measured utility with a user study and Hansen et al. (2023) with the ability to retain model or feature importance ranking (measured with rank correlation) in the train-on-synthetic evaluate-on-real paradigm. Note that all of these studies evaluated utility on a single-table, even those that investigated synthetic relational databases.
3
Evaluating Synthetic Relational Databases
3.1
Multi-Table Fidelity Using Aggregation
We build our approach on cxisting detection-based approaches. First, we address the lenient evaluation by replacing logistic regression commonly used in related work with tree-based ensemble methods (Borisov et al.
2023
Zein & Urvoy 2022).
Current fidelity metrics fail to thoroughly evaluate interactions indueed by relationships between tables: (i)
cardinality similarity [ assesses only the cardinality of foreign key relationships, (ii) k-hop similarity (Pang
ct al..
2024b evaluates only lincar relationships between column pairs in related tables, and (iii) cxisting implementations of PC-LD do not properly account for denormallzatiin breaking the i.i.d. assumption required for C2ST, leading to unprincipled inference.
To address these limitations, we introduce a classifier two-sample test with aggregation (C2ST-Agg). Instead of denormalixing lables, we incorporate the multi-table structure by augmenting both real and synthetic parent tables with aggregated features derived from their child tables.
Aggregation is an established technique in the field of relational reasoning (Getoor et al. 2007 Džcroski 2010) and C2ST-Agg can be thought of as a propositionalization (Kramer et al. 2001 approach to the C2ST on relational databases. By summarizing child-table columns and relationship cardinality through aggregation functions (e.g., mean, count, max), C2ST-Agg maintains the i.i.d. assumption while enhancing classifier-basee fidelity assessment.
htttps://docs.sdv..dtt detectionn/prent,_rm

---

Published in Transactions on Machine Learning Research (02/2026)
Our approach addresses the issues of current fidelity metrics: it accounts both for relationship cardinality (i) and high-level interactions across all columns in related tables (ii), while maintaining the ii.d. assumption for each table (iii).
In practice, users can select aggregation functions based on the specific aspects of relational data they wish to evaluate. We provide general guidelines for choosing these aggregations, along with a detailed explanation of the C2ST-Agg metric, in Appendix [32.1
A benefit of using predictive models to evaluate fidelity is that diagnose issues with our data generation. By examining the features that the discriminative model uses to distinguish between synthetic and real data, we can identify which aspects of the data the generative method fails to model well. We can employ standard ML interpretability methods such as Shapley values (Strumbelj
use ML explainability methods to
& Kononenko 2010 Lundberg & Lee 2017), partial dependence plots (Friedman 2001), accumulated local effects (Apley & Zhn 2020), or modcl-spocific methods such as relative variable importance in boosted trees or coelficients of linear regression models. See Section 4.2 for an example.
3.2 Relational Deep Learning Utility
Relational deep learning (Fey et al. 2024 has emerged as an alternative to traditional ML methods by
transforming relational databases into graphs (Robinson et al. 2024 Papamarkou et al., 2024) and utilizing graph neural networks (GNNs). Subsequently, RRobinson et al. (2024) proposed RelBench, benchmark for relational deep learning. The authors of RelBench show that the performance of GNN pipeline is comparable to a traditional MT. pipeline approach done by a data scientist.
Up until now, ML utility has been evaluated only on single-tables (see Section 2.2 which means inter-table relationships have not been taken into account even when evaluating multi-tan table synthetic data. We could create a feature engineering pipeline for each database, but that would require domain knowledge, is labor- intensive, and is not casily extensible. Instead, we incorporate the graph-based approach of RelBench into SYNTHERELA. We adopt the RelBench pipeline to fit. GNN models on real and synthetic data respectively, and then evaluate them on the test set consisting entirely of real data (see Appendix for more details).
Splitting the data into train and test sets is challenging for relational databases with complex inter-table dependencies. To address this, we adopt a time-based splitting strategy, consistent with the methodology proposed by RelBench. This allows us evaluate data generation performance for any relational database with a time component, without the need for manual data processing. It can also be easily extended to new databases, provided they contain datetime column allowing train-test split. Notably, this is the only approach to relational utility that includes the entire relational database, ensuring a more comprehensive evaluation. We call this approach relational deep learning utility (RDDL-utlity).
4 Benchmarking and Results
We combine our findings into synthetic relational database benchmark. We report existing metrics for evaluating single column, single-table, and multi-table fidelity to which we add a new metric for measuring multi-table fidelity C2ST-Agg and a novel approach to evaluating the utility of relational synthetic data, RDL-utility.
We compare the following methods for synthesizing relational data: SDV, RCTGAN, REaLTabFormer CDavaDDPM, RGCLD, and AabularARGG Other related work does not have available source code, does not have an API or were not able to run the source code the selected databases.
We include 6 datasots featured in related work (Airbnb. Rossmann, Walmart, Biodegradability, MovieLens). In addition we include Cora datasel by McCallum et al. (2000), a popular datasel in graph representation learning, and the F1 dataset from the relational deep learning benchmark Robinson et al. (2024), for a total of 8 benchmark datasets. The datasets vary in types of relationships and number of tables and columns, which are sunmarized in Table (see Appendix B.3 for details and Appendix G for database schemas).
5

---

Published in Transactions on Machine Learning Research (02/2026)
Table 1: Summary of the 8 benchmark datasets. The number of columns represents the number of non-id columns. The collection diverse and covers all types of relational structures.
Dataset Name
# Tables
# Rows
# Columns
# Relationships
Hierarchy Type
Rossmann
2
59,085
16
Linear
Airbnb
2
57,217
20
Linear
Walmart
3
15,317
17
Multi Child
Cora
57,353 3
2
Multi Child
Biodegradability
21,895 5
6
Multi Child Parent
IMDB MovieLens
7
1,249,411
14
Multi Child & Parent
Berka
757,722
37
Multi Child Parent
F1
9
74,063
33
13
Multi Child & Parent
We evaluate all three levels of synthetic relational databases, with focus on multi-table evaluation (see Appendix [.2 for all benchmark metrics). Most methods are non-deterministic, so we report results for three different replications. However, all results are stable across replications. Four of the methods are capable of synthesizing all of the datascts, irrespective of their relational structure. REALTABFORMER only capable of generating databases with linear structure and ClavaDDPM is unable to model datasets with two or more foreign keys between pair of tables (Biodegradability and CORA). For single-column metries (see Table 12 in the Appendix for results), we report the complement of the Kolmogorox-Smirnov statistic and the Total Variation Distance (the complement to KS/TV distance between two distributions and Q is 1 - (P))))(P)l))). For single-table metrics, we report the complements of the KS and TV distances between column pair corrclations. For multi-table metrics, we report the average cardinality shape similarity and the k-hop (k 1) correlations between tables. At all levels, we report the C2ST using XGBoost (Chen & Guestrin 2016) the discriminative model. We 5-fold stratified cross- cross-validatte validation to estimate detection accuracy, For C2ST-Agg, we augment the rows with (a) counts of child rows for each row in each parent table, (b) the mean values of the numeric columns in the child table corresponding to the parent row; and (c) the number of unique categories in related rows.
Neural- based single-table generators evaluate privacy with metrics that use a held-out test set, as opposed to marginal- based generators, that provide provable privacy guarantees (DP). As representative sampling for relational databases non-trivial, evaluating privacy multi-table data remains an open problem. Recently, the MIDST Challenge (Vector Institute, 2025
Wu et al. 2025) provided the first analysis of the privacy of neural-based relational data generators, focusing on membership inference attacks (MIA) (Shokri et al. 2017
These attacks are tailored to individual generative models and are therefore not generally applicable. Instead, we performed privacy sanity check against the SMOTE baseline, following related work (Pang et al. 2024 Tiwald et al. 2025). We use the distance to closest record metric and perform our analysis on the Airbnb dalasel as we can re-sample a hold-out set of equal size without a temporal shift. The results of the privacy check show that all the methods outperform SMOTE, providing no evidence of systematic data copying (see
Appendix C.1 We also explore how C2ST can be used as data copying diagnostic in Appendix C.4
4.1 Single- Table Performance
We first evaluate the fidelity of individual lables. We focus on the detection metric and how well column pairs (bivariate distributions) are modeled. Table summarizes the results. On the databases that it is able to generate, the diffusion-based ClavaDDPM performs best, followed by the autoregressive TabularARGN specializing in sequential (linear structured) databases.
The rankings of methods for single column metrics are similar to those of single-tables. As expected, the mothods model individual columns better. See Appendix(C.2 details. Interestingly, TabularARGH performs best on modeling marginal distributions, indicating their discretization approach is a robust preprocessing step.

---

Published in Transactions on Machine Learning Research (02/2026)
Table 2: Single-table results. For each dataset and metric we report the average and standard error of C2ST accuracy (lower is better) and column pair trends (Pairs higher better) across all tables for three independent samples. The best result for each dataset is in bold.
Dataset
Metric
TARGN
RGCLD
ClavaDDPM
RCTGAN REALTABF.
SDV
C2ST (4)
24.23
70.25ga
78.10
88.37
83.97.400
99.75
Airbub
Pairs (f)
a3.48
88.51
87.7810.12
79.37a0.59
53.90g1.00
49.03
C2ST (4)
56.072084
68.97
66.7720.1
88.0220.0
74.70
96.90ae
Rossmann
Pairs (t)
91.34
90.08
84.7820.9
84.3820.4
84.5820.s
67.77 as
C2ST (U)
83.54
66. 76
53.5 43.50
76.40
70.87
87.02
Walmart
Pairs (t)
23.80
94.02sa.34
86.60
83. 83.1020.40
87.61
C2ST (1)
72.31
64.17+200
54.48
68,12+0.44
82.40
Berka
Pairs
70.43 20.2
73.77
88. 19
74.22+0.25
64.01
C2ST ())
81.93-0.40
69.63 +1.
71.42 to.45
80,67
89.84-c.22
F1
Pairs (1)
81.31 no.4s
42.3
84.65
90.17+0.03
+3.05
C2ST (
20.92
55.01
49.83
55.38
IMDB
TLE
Pairs (1)
97. 2o
23.89
98.66 a0..6
81.65
C2ST (4)
58. 19
45
58.26
68.59
Biodeg.
Pairs (t)
a4.82
20.89
85.4428.19
97.58 g0.50
Cora
C2ST (L)
50.94
52.98
48.97
75.45a
4.2 Multi Table Performance
Multi-table metrics examine how well the relationship cardinality and the relationships between different. tables are preserved. Cardinality shape similarity examines only the former, while k-HOP similarity evaluates the latter; C2ST-Agg examincs both. We report the results in Table 3 Similar to single-table fidelity, the diffusion-based methods perform best.
C2ST with aggregation uses the same features for single-table and multi-table evaluation (with the exception of the aggregation attributes). This allows us to directly compare single and multi-table performance. For most methods, we observe a significant drop in fidelity when adding aggregations. Figure 2 shows how C2ST detection accuracy increases when we add information about relationships between tables. We investigate this further using explainability methods and show how these can be used to debug generative methods.
Relational A
Avg. Multi Table
Detection Accuracy
0.06
Avg. Single Table
0.16
0.15
0.14
0.08
Perfect Fidelity
SDV 2016
REALTABFORMER 2023
RCTGAN 2023
CLAVADDPM 2024
RGCLD 2024
TabularARGN 2025
Figure 2: Comparing single and multi-table performance. While there is an overall trend of improvement time of both single-table and multi-table fidelity, most methods still exhibit significant, gap between single-table and multi-table fidelity (relational A).
7

---

Published in Transactions on Machine Learning Research (02/2026)
Table 3: Multi-table results. For each dataset we report the average and standard error of C2ST-Agg accuracy (lower is better), cardinality similarity (higher is better) and k-hop correlation similarity (higher is better) across all tables for three independent samples. "." denotes a method is unable to generate the dataset, and TLE timcout. The best result is in bold and results within standard deviation are underlined.
Dataset
Metric
TARGN
RGCLD
ClavaDDPM
RCTGAN
REALTABP.
SDV
C2ST-Agg (1)
63. 63.47 au
79.53
Fs 100.0
98.222u.s
99. 13
99 20.9
Airbub
Cardinality (1)
98, 1..59.ca
99.36
99.65.m..
95.45
76.38
26.36
I-HOP (f)
79.66
84.50=2.74
86. 86.6920.14
68.782u.4
33.99
24.58au.us
C2ST-Agg (L)
60. g...ca
75.5620.74
85.77=0.07 o7
86.11 11:.0
85.90
98.37
Rossmann
Cardinality (1)
_4.17
98.95
82.69
11.82210.8
99.16..
1-HOP (+)
92. su.77
88.30
82. 92.81=0.4r 47
87. 87.02=au17
80 +0.254
73.84 73.84
C2ST-Agg (T)
94.81
88.89 1os
73.33
24
90.0010.01
88.52a1
Walmart
Cardinality (1)
65.93_1 Aa
14.81
93.33_ ss
88. 88.1521.51
85.5614.87
86.30g1 09
1-HOP (+)
75.40_1.4 49
82.05 77
86.4 86.40
79.0220.15
74.9920.9
76.64
C2ST-Agg (1)
80.56-1.se
72.71-a.aa
69. 19.112:a +n.4a
76.86
77. 77.43
Cardinality (t)
85.17
Rs 100.0
96.43 so
81.28
80 g0.53
Berka
I-HOP (+)
72.82
80.7421.4
$7.92.11
78.87
59. 59.0922.4
2-HOP (+)
65.51
73
84.41
77 77.9822..
23.09
H-HOP (+)
59.34 sia
=4.81
80. =2
78.65 0v
58 58.23.2...
C2ST-Agg (T)
95.90 94
74.18
82.52
91.23110.0
9-1.55.o..e4
Cardinality (1)
18.17
100.0
88, 45 os
56.82.1.65
71.88
F1
1-HOP (t)
77.37 as
88.35
79.35_.o. ea
79.11..077
68. 15.0.0
2-HOP (t)
76.25
87.53
84, 84,18=0.12
83.50=u.x2
76.93
C2ST-Agg (L)
73.76_1.7 7.
65.37_5.54
65. 00..00....
81.56....0 on
IMDB
Cardinality (1)
81.1920
Rs 100.0
98.95 oo
79.53.1.2 1.77
TLE
I-HOP (t)
88.6420
85.66
91.5721.9
81.7620.00
C2ST-Agg (4)
88. 20.
70.51
83. (3.82
98.02 08.02
Cardinality (1)
79.53
97.95
85 e5.22
61.17
Biodeg.
1-HOP (+)
61.36 47
78. 78.4624..0
75. 75.8021.00
49.09go.a s9
2-HOP (+)
60.54 44
75. 12
77.0421.v IM
47 87.80
C2ST-Agg (1)
68.80_a.or
62.33
73.74...47
99.59
Cora
Cardinality (t)
96.27
Rs 100.0
90.48.a.16
68.88 28..2
1-H1PP (1)
80. 10.42
63.81
68.391..0s
4.95
ML explanation with feature importance reveals that methods struggle with preserving the relationships between columns across tables. Figure 3a shows an example of how aggregation attributes summarizing information about child table rows are the most discriminative features. We further examine two such attributes in Figure 4 The partial dependence plots of the first and fourth most important features from Figure 3 show how subsets of both categorical (Fig. 4a) and numerical (Fig. 4b) features' conditional distributions are informative to the discriminative model. In Figure 3b we illustrate how a simple interaction between two aggregation attributes sperffuttes-seefffaallly. relationship cardinality and the number of unique categories in a child table renders the synthetic data nearly separable from the original. This highlights how C2ST-Agg captures both structural and attribute-based discrepancies, including their interactions, offering more comprehensive evaluation than simpler metrics like k-hop similarity. We include the interpretability methods directly into our implementation of the detection metric, allowing users to immediately investigate where their method is underperforming after evaluating the synthetic data.
4.3 Relational Deep Learning Utility Performance
We run extensive experiments for RDL-utility across 5 databases that contain a temporal feature, using 6 GNN architectures with hyperparameter tuning for each architecture and database pair on original data (see Appendix D for more details). Table 4 summarises the RDI.-uillit results (Section 3.2). Details of the tasks are provided in the Appendix D The utility scores follow the same trend as the multi-table results, with models achieving high fidelity also performing best in utility tasks. The best performing models are
8

---

Published in Transactions on Machine Learning Research (02/2026)
the diffusion based approaches ClavaDDPM and RGCLD, with the autoregressive approach TabularARGN performing best on one simpler two-table database. The drop in utility performance for databases with more complex relational structures is also consistent with fidelity results and shows that utility of relational databases should be evaluated using multi-table approaches.
To be able to estimate the value and validity of RDI-utility measuring the utility performance of the generated syuthetic databases, we implement two baseline metrics for measuring utility (see Appendix D.3 for more details). The first one is a simple bascline, where we train LightGBM (Ke et al.) 2017) model directly the entity table with no feature engineering. This corresponds to the typical machine learning utility evaluation used in related work (Solatorio & Dupricz 2023 Pang et al. 2024]. The second bascline is an automated feature engineering baseline using Deep Feature Synthesis (DFS) (Kanter & Veeramachaneni 2015p, based on calculating aggregated features of the child tables and appending them to the entity table, also trained with LightGBM model.
Entity table LightGBM results (Table 5 again show that diffusion based models are able best generate single table overall. The DFS feature engineering baseline scores (Table ( show a similar story. The drop in performance in the RDL-utility compared to the entity-table baseline stemss from the noise introduced by poorly generating the relational structure and features of child tables.
RDI-utility, alongside naive baselines, demonstrates that the models can learn patterns from synthetic databases that generalize to held-out test sets even in cases of non-perfect fidelity.
4.4 Privacy Evaluation
Our primary focus is on evaluating the fidelity and utility of synthetic relational data. However, high data quality should not come at the expense privacy. Following prior work (Kotelikov et al., 2023 Pang et al.] 2024 Zhang et al. (024),, we conduct. a privacy comparison against SMOTE (Chawla et al. 2002 an interpolation- beeepollaiion-beedd method that synthesizes new data points via convex combinations of real samples and their nearest neighbors.
To assess privacy risks, we compute the distance to closest record (DCR) Zhao et al. 2021 a common metric that quantifies how close synthetic records are to real data. Specifically, we report the DCR score the probability that a synthetic sample is closer to a real training point. than to any sample from a held-out Lest. set. We conduct our privacy check on the Airbab dataset containing anonymised information of real users.
Movies2nctors Cast Num #Unique
Synthetic
Movies2directors Genre #Unique
Transaction Account Counts
500
Reai
Movies2directors Counts
400
Ulbase Rating Mean
U2bsse Counts
300
Movies2actors Counts
200
Runningtime
AGGREGATE NUMERICAL CATEGORICAL
Country
100
IsEnglish
0.0
0.1
0.2
0.3
0.4
Feature importance
Transaction Account Bank #Unique
(a)
(b)
Figure 3: ML Interprotability Diagnostics of C2ST. (a) Feature importance for C2ST-Agg using XGBoost. Results are for the best-performing method. The added features that incorporate relational information (red) are the most important for discriminating between real and synthetic data. (b) Two-way PDP showing the interaction between the relationship cardinality of the marginal distríbution shown on the right) and the number of unique banks associated with an account's transactions. The plot illustrates how combination of structural and informational aggregation attributes can render the synthetic data nearly separable from the original.
key (with its
9

---

Published in Transactions on Machine Learning Research (02/2026)
Table 4: RDL-utility results. We report mean + SE of ROC-AUC (classification) and MAE (regression).
Naive baseline (mean/majorigh is in perentheses. indicates invalid time columns. Best is bold; within one SE is underlined.
Dataset
Model
ORIG.
RGCLD
TARGN
CLAVA
ROTGAN
REALTABF.
SDV
GAT
211 (324)
252
304
230
289
564
963
GATv2
205 (324)
215
330
239
251
531
1212+184
GIN
178 (324)
197
224
193
219ga
257
3648
Rossmann
MAE (1)
G-Conv
178 (324)
206
229
192
216+4
253
3430
G-SAGE
178 (324)
204
229
193g
224g
256gas
3428
ReIGNN
178 (324)
210+11
223
192
219+a
254+22
3440
GAT
13118 (14.7k)
13626
138462.0
13496_s7
13813as
14364
13795
GATv2
13114 (14.7k)
14081
13841:4
13497 as
13750..4
14362
13760
GIN
13056 (14.7k)
15165
13841
13423
13755
14305
13539a:44
Wahmart
MAE (1)
G-Conv
13033 (14.7k)
15268
13850
13329
13849
14267
13581
G-SAGE
13052 (14.7k)
15090=1204
13838:4s
13421
13738:43
14270=1x0
13761+:09
ReIGNN
13097 (14.78)
15243g10e
1384621*
13408_50
13791
1431 1
13722 11
GAT
0.72 (0.5)
0.64g..m
0.67
0.64
0.4820.0
0.4629.01
GATv2
0.74 (0.5)
0.62
0.56
0.50
0.51:
GIN
0.75 (0.5)
0.62 2.62.0.04
0.67.a.0
0.66
0. 0.5620.01
0.562o.01
Airbub
AUC (t)
G-Conv
0.74 (0.5)
0.63 g..3g..
0.69
0.65
0.54g.000
0.5 0.54go.00
G-SAGE
0.78 (0.5)
1.68....
0.7 0.70so.02
0.68
0. 0.63
0. 0.63
ReIGNN
0.77 (0.5)
0.66.0.0
0.71
0.67
0.5620.0
0.60 0.60
GAT
0.92 (0.5)
0.56
0.71+
0.73 70.00.00
GATv2
0.96 (0.5)
0.44go.0
0.52 oe
0.64
GIN
0.96 (0.5)
0.61an.0z so.os
0.53
0.68 4o:
Berka
AUC (1)
G-Conv
0.98 (0.5)
0. ax..0.
0.35
0.56
G-SAGE
0.93 (0.5)
0.56go.0x
0.70
0.61
ReIGNN
0.95 (0.5)
0.55
0.31+
6.65
GAT
0.85 (0.5)
0.55go os
0.66 oa
0.66
0.68
2.49
GATv2
0.86 (0.5)
0.73
0.54
0.74
0.50
0.56...00
GIN
0.87 (0.5)
0. 77770.00
0.562e
0.76
0.42
0.64
F1
AUC (t)
G-Conv
0.87 (0.5)
0.78
0.43aa.
0.75
0. 0.462
0.75go.02
G-SAGE
0.86 (0.5)
0.78
0.44
0. 0.76aa.cz
0.49
0.68
ReIGNN
0.84 (0.5)
0.77 20.01
0.44g o8
0.69
0.4620.12
0.60
Table 5: Entity table LightGBM baseline. We report mean (regression). Naive baseline (mean/ /majority)) is in parentheses. *_ indicates invalid time columns. Best is bold; within one SE is underlined.
+ SE of ROC-AUC (classification) and MAE
Dataset
ORIG.
TARGN
RGCLD CLAVADDPM RCTGAN REALTABE.
SDV
Rossmann
MAE (1)
225 (324)
227
2251000
22520.00
218
224
42861s
Walmart
MAE (1)
13664 (14.7k)
13841+28
13704
13666+s0
13818
14433+
13949+*
Airbnb
AUC (t)
0.73 (0.5)
0.73+o.m1
0.74+o.u
0.62+a.os
0.7
0.51+o.02
Berka
AUC (T)
0.71 (0.5)
1.53
0.62
0.73
AUC (1)
0.61 (0.5)
0.53+
0.63+o.u4
0.57+a.02
0.58+
0.60
Table 6: DFS feature engineering LightGBB baseline. We report mean + SE of ROC-AUC (classifica-
tion) and MAE (regression). Naive baseline (mean/ (oajarrjaorit)) is in parentheses. indicates invalid time columns. Best is bold; within one SE is underlined.
Dataset
ORIG.
TARGN
RGCLD
CLAVADDPM ROTGAN
REALTABE.
SDV
Rossmann
MAE (1)
100 (324)
215+a
210+7
203+a
210+a
230+a
2290+ar
Walmart
MAE (1)
13251 (14.7k)
188262a
14560zs0
133922s
13741
14130212
1348624s
Airbub
AUC (f)
0.74 (0.5)
0.72+o.a2
0.73
0.60-o.as
0.71
0.54+o.m
Berka
AUC (T)
0.90 (0.5)
0.64+o.us
0.65+o.ox
0.71+
AUC (T)
0.74 (0.5)
0.45
0.72
0.682n.o1
0.65.o.0s
0.51
10

---

Published in Transactions on Machine Learning Research (02/2026)
10
Partial dependence
Partial dependence
0.8
0.8
66
04
64
02
Individual CEs
62
Individual CEs
Average
Average
SE
SE
13
20
3,0
3.5
43
5.
Movies2actors Cast Num #Unique
U2base Rating Mcan
(b)
Figure 4: Partial dependence plots of C2ST. Results are for the 1st and 4th most important feature from Figure 3. With ideally generated synthetic data, features would not differ between synthetic and original data and every partial dependence plot would be horizontal line at 50% probability. We can observe that (a) the synthetic data have too many unique actor cast numbers (higher probability of being synthetic when feature value is larger than 4) and (b) the mean movie ratings in the original data vary more than in the synthotic data, where they are more concentrated around 3.5.
However, recent studies have highlighted limitations of distance-basee privacy metrics Yao et al. 2025
Ganev
& De Cristofaro 2025 Ward et al. 2024). showing that such measures can fail to capture informat ion leakage in high-dimensions complex datasets and adversarial scenarios. To provide a more robust and comprehensive evaluation, we therefore complement DCR with attack-based privacy assessment via membership inference attacks (MIA).
In MIA, an adversary attempts to infer whether given record from the real data was part of the training set used to generate the synthetic data (El Emam el al. 2022b). We adopt the implementation of Lautrup lt al. (2024), which assumes a black-box adversary with knowledge of the population but no insight into the synthesis model. The attacker is modeled as a LightGBM classifier (Ke et al. 2017) trained to distinguish between synthetic samples and real records from an external holdout set. The classifier's discriminative performance serves as an estimate of membership disclosure risk. Analogous to the DCR evaluation, MTA is conducted at the table level, as no multi-table variant currently exists.
We report DCR results in Table 7 and MIA AUC scores in Table g Across both tables, all methods except for ClavaDDPM on the Users table-maintal maintain reasonable DCR values close to the ideal 50% range, suggesting no systematic data copying or excessive proximity between synthetic and real samples. However, the MIA evaluation tells a different story. As shown in Table g most generators expose the training data to a non-negligible membership inference risk. With the exception of SDV and Rea..TabFormer, whose MTA AUC scores remain close to random guessing (Re 50%) but correspond to low data quality (sce Tables 2 the adversarial classifier achieves above-random discrimination for nearly all methods. We include detailed results of our privacy experiments in Appendix C.1
Our privacy evaluation shows that despite not achieving perfect fidelity nor systematically copying the original dala, current methods still pose measurable privacy risks to their training data. These findings further confirm recent results highlighting the limitations of distance-based privacy metrics and underscore the importance of attack-basod evaluations such as MIA. More broadly, they motivate future rescarch into privacy evaluation protocols for synthetic relational data and the development of privacy-preservee generative models that balance fidelity and privacy protection more effectively (Izzo al. 2024 Jiang al. 2025
11

---

Published in Transactions on Machine Learning Research (02/2026)
Table 7: DCR Scores on the Airbnb dataset repre- sent the probability of a random sample being closer a training sample rather than randomly drawn sample from a held-out datasct. Values near 50% in- dicate no systematic data copying, while larger values indicate privacy risks.
Table 8: MIA AUC Scores on the Airbnb dataset measure the overall ability of an attacker to distin- guish between training records (members) and held- out records (non-members). Values near 50% indicate the attack is equivalent to random guessing, suggest- ing high privacy protection.
Method
Sessions
Users
Method
Sessions
Users
TARGN
50.40
+0.20
49.87
+0.60
TARGN
55.18 #0.10
59.77 +0.30
RGCLD
51.77
20.21
50.30
10.50
RGCLD
50.94
20.06
78.52 20.72
ClavaDDPM
51.77
+0.21
84.83
+0.36
CLAVADDPM
64.43
#0.25
68.88 #0.32
RCTGAN
49.48
10.22
48.93
1:050
RCTGAN
62.35
10 02
98.76 10.18
REALTABF.
46.65
51.38
REALTABE.
50.98 #0.02
50.11
SDV
46.16
48.01
+:0.0
SDV
50.00 00.00
50.02
#0.01
SMOTE
88.86 +0.10
99.82 +00.04
SMOTE
94.03 +0.11
92.35 +0.15
5 Conclusion
To address the need for standardized evaluation in the emerging field of synthetic relational database generation, the primary contribution of this work is SYNTHERELA, the first open-source benchmark specifically designed for this lask. In developing SYNTHERELA, we critically reviewed existing methods for evaluating the fidelity and utility of syuthetic data and introduced two novel contributions: C2ST-Agg, a robust detection- based metric for assessing multi-table fidelity, and RDL-utility, a general framework for evaluating the utility of synthetic relational databases. SYNTHERELA is designed to offer practical insights for synthetic data users and features public leaderboard to serve as baseline for future methods. We provide SYNTHERELA as open-source python package (Appendix B.I..
We also applied the benchmark to current state-of-the-art synthetic data generation methods. The best methods generate marginal distributions well and are on par with single-table methods. Diffusion-based approaches also perform well on single-table fidelity. However, all methods decline in performance when evaluated on multi-table fidelity, highlighting the added complexity of modeling relational databases. Although no method perfectly preserves inter-table relationships, when evaluated on RDL-utility, most outperform naive basclines, and some achieve model performance scores comparable to those trained the original database. This suggests that despite not achieving perfect fidelity, these synthetic data remain valuable for downstream machine learning tasks.
5.1 Limitations
Our benchmark, SYNTHERELA, evaluates neural-based generative methods focused on fidelity and utility on rcal-world relational databascs. This excludes marginal-based methods focused on differential privacy and query error metrics, which are often tailored to specific dalasets and queries. The current benchmark datasets lack predefined queries for evaluating these differential privacy-centric approaches. Future work could incorporate datasets with predefined queries and incorporate marginal-based methods to bridge the gap between the two classes of methods.
Several aspects of the evaluation of synthetic data are limited by the difficulty of representative sampling. We circumvent this in RDI.-uiility by using temporal-based split; however this limits our current implementation to databases that contain datctime columns. More work needs be done understand the limitations and prepare new benchmark datasets dataset generators.
12

---

Published in Transactions on Machine Learning Research (02/2026)
References
Airbnb. Airbab new user bookings. 2015.
Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next- generation hyperparameter optimization framework. In The 25th ACM SIGKDD International Conference 2019.
on Knowledge Discovery & Dala Mining, pp.
Kaveh Alimohammadi, Hao Wang. Ojas Gulati, Akash Srivastava, and Navid Azizan. Differentially private synthetic data generation for relational databases, 2025. URT. htttss://aaaat: wwtpe/iwwwrggabs/a/eeeeo httg/abb/2405..887777
Daniel W Apley and Jingyu Zhu. Visualizing the effects of predictor variables in black box supervised learning models. Journal of the Royal Statistical Society Series B: Statistionl Methodology, 82(4):1059-100) 1086, 2020.
Arno Appenzeller, Moritz Leitner, Patrick Philipp, Erik Krempel, and Jürgen Beyerer. Privacy and utility of private synthetic data for medical data analyses. Applied Sciences, 12(23))12320, 2022.
Samuel A Assefa, Danial Dervovie, Mahmoud Mahfouz, Ro(Beaulieu-Jones al. 2019). E Tillman, Prashant Reddy, and Manuela Veloso. Generating syuthetic data in finance: opportunities, challenges and pitfalls. In Proceedings of the First ACM International Conference on AI in Finance, pp. 1-8, 2020.
Brett K Beaulieu-Jones, Zhiwei Steven Wu, Chris Williams, Ran Lee, Sanjeev P Bhavnani, James Brian Byrd, and Casey S Greene. Privacy-preserving generative deep neural networks support clinical data sharing. Circulation: Cardiovascular Quality and Outcomes, 12(7):c005122) 2019.
Petr Berka al. Guide to the financial data set. PKDD2000 discovery challenge, 2000.
Hendrik Blockeel, Saão Džeroski, Boris Kompare, Stefan Kramer, Bernhard Pfahringer, and Wim Laer. Experiments in predicting biodegradability. Applied Artificial Intelligence, 18, 06 1999.
Daniel Bonnéry, Yi Feng, Angela K Henneberger, Tessa L Johnson, Mark Lachowiez, Bess A Rose, Terry Shaw, Laura M Stapleton, Michael E Woolley, and Yating Zheng. The promise and limitations of synthetic data as a strategy to expand access to state-level multi-agency longitudinal data. Journal of Research on Educational Effectiveness, 12(4):616-66) 647, 2019.
Vadim Borisov, Tobias Leemann, Kathrin SeBler, Johannes Haug, Martin Pawelczyk, and Gjergji Kasneci. Deep neural networks and tabular data: A survey. IEEE Transactions on Newral Networks and Learning Systems, 2022.
Vadim Borisov, Kathrin Sessler, Tobias Leemann, Martin Pawelezyk, and Gjergji Kasneci. Language models are realistic tabular data generators. In The Eleventh International Conference Learning Representations, 2023.
Teodora Sandra Buda, Thomas Cerqueus, John Murphy, and Morten Kristiansen. Cods: A representative sampling method for relational databases. In Database and Expert Systems Applications: 24th International Conference, DEXA 2013, Progue, Czech Republic, August 26-29, 2013. Proceedings, Part I 24, pp. 342-356. Springer, 2013.
Lars Buitinck, Gilles Louppe, Mathieu Blondel, Fabian Pedregosa, Andreas Mueller, Olivier Grisel, Vlad Niculae, Peter Prettenhofer, Alexandre Gramfort, Jaques Grobler, Robert Layton, Jake VanderPlas, Arnaud Joly, Brian Holt, and Gaël Varoquaux. APT design for machine learning soft ware: experiences from the scikit-learn project. In ECML PKDD Workshop: Languages for Data Mining and Machine Learning, pp. 108 122, 2013.
Kuntai Cai, Xiaoyu Lei, Jianxin Wei, and Xiaokui Xiao. Data synthesis via differentially private markov random fields. Proceedings of the VLDB Endowment, 14(11):2190 2202, 2021.
Kuntai Cai, Xiaokui Xiao, and Graham Cormode. Privlava: Synthesizing relational data with foreign keys under differential privacy. Proc. ACM Manag. Data, 1(2), jun 2023.
13

---

Published in Transactions on Machine Learning Research (02/2026)
Kuntai Cai, Xiaokui Xiao, and Yin Yang. Privpetal: Relational data synthesis via permutation relations, 2025. URT. http:////aaxiiw.og/aba////////// 22970
Luca Canale, Nicolas Grislain, Grégoire Lothe, and Johan Leduc. Generative modeling of complex data, 2022.
URL htttpe///axxiv..rgg/b///2222222
Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321-351 2002.
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16 pp- 785-794 New York, NY, USA, 2016. Association for Computing Machinery. ISBN 9781450342322.
Inc. DataCebo. SDMetrics, 10 2022. URL https://docs... sdv
dww.//dmettrccc/////// http:://boo...trr..er//deeeeeee Version 0.8.0.
Asim Kumar Debnath, Rosa L. Lopez de Compadre, Gargi Debnath, Alan J. Shusterman, and Corwin Hansch.
Structuro-cctivity relationship of mutagenic aromatic and heteroaromatic nitro compounds. correlation with molecular orbital energies and hydrophobicity. Journal of Medicinal Chemistry, 34(2):786-77) 797, 1991.
Cynthia Dwork. Differential privacy. In International colloquium on automala, languages, and progromming, pp. 1 12. Springer, 2006.
Saão Džeroski. Relational data mining. Springer, 2010.
Khaled EI Emam, Lucy Mosquera, and Xi Fang. Validating membership disclosure metric for synthetic health data. JAMIA open, 5(4), 2022a.
Khaled El Emam, Lucy Mosquera, Xi Fang, and Alaa El-Hussuna. Utility metrics for evaluating synthetic health data generation methods: validation study. JMIR medical informatics, 10(4):e35734, 2022b.
F1. F1 db, 2021. URL httph::// https://githhbb httpitthu..co/ffffiat
Matthias Fey, Weihua Hu, Kexin Huang, Jan Fric Lenssen, Rishabh Ranjan, Joshua Robinson, Rex Ying, Jiaxuan You, and Jure Leskovec. Position: relational deep learning-graph representation learning on relational databases. In Proceedings of the 41st International Conference on Machine Learning, pp- 2024.
Will Cukierski FlorianKnauer. Rossmann store sales, 2015. URL
Sessmann-store-ie
Jerome H Friedman. Greody function approximation: a gradient boosting machine. Annals of statistics, pp. 1189 1232, 2001.
Georgi Ganev and Emiliano De Cristofaro. The inadequacy of similarity-based privacy metries: Privacy attacks against "truly anouymous" synthetic datasets. In 2025 IEEE Symposium on Security and Privacy (SP), pp. 4007-4025. IEEE, 2025.
Rainer Gemulla, Philipp Rösch, and Wolfgang Lehner. Linked bernoulli synopses: Sampling along foreign keys. In Scientific and Statistical Database Management: 20th International Conference, SSDBM 2008, Hong Kong, China, July 9-11, 2008 Proceedings 20, pp. 6 23. Springer, 2008.
Lise Getoor, Nir Friedman, Daphne Koller, Avi Pfeffer, and Ben Taskar. Probabilistic relational models. Tn Introduction to Statistioal Relational Learning. The MIT Press, 08 2007.
Gretelai. Gretel blog,
2024. Accessed on March 24th, 2024.
Mohamed Gueye, Yazid Attabi, and Maxime Dumas. Row conditional-taan for generating synthotic relational databases. In ICASSP 2023-200 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. TEEE, 2023.
14

---

Published in Transactions on Machine Learning Research (02/2026)
Lasse Hansen, Nabeel Seedat, Mihacla van der Schaar, and Andrija Petrovic. Reimagining synthetic tabular data generation through data-centric AT: A comprehensive benchmark. In Thirly-sevenlh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023.
F. Maxwell Harper and Joseph A. Konstan. The moviclens datascts: History and context. ACM Trans. Interact. Intell. Syst., 5(4), dec 2015. ISSN 2160-6455 doi: 10.1145/2827872.
Valter Hudovernik. Relational data generation with graph neural networks and latent diffusion models. Tn NeurIPS 2024 Third Table Representation Learning Workshop, 2024. URL https://openrevvexx net/
forum?
Zachary Irzo, Jinsung Yoon, Sercan O Arik, and James Zou. Provable membership inference privacy. Transactions on Machine Learning Research, 2024. ISSN 2835-8856.
Zhanhong Jiang, Md Zahid Hasan, Nastaran Saadati, Aditya Balu, Chao Liu, and Soumik Sarkar. Balancing utility and privacy: Dynamically private SGD with random projection. Transactions on Machine Learning Research, 2025. ISSN 2835-8856.
James Max Kanter and Kalyan Veeramachaneni. Deep feature synthesis: Towards automating data science endeavors. In 2015 IEEE International Conference on Data Science and Advanced Analytics, DSAA 2015, Paris, France, Oclober 19-21, 2015, pp. 1-10. IEEE, 2015.
Antheas Kapenekakis, Daniele Dell' Aglio, Charles Vesteghem, Laurids Poulsen, Martin Bøgsted, Minos Garofalakis, and Katja Hose. Synthesizing accurate relational data under differential privacy. In 2024 IEEE International Conference on Big Dala (BigDala), pp. 433 439. IEEE, 2024.
Shingo Kato, suharay, Will Cukierski, and haisland0909. Coupon purchase prediction, 2015. URL https:
/ //kaggle.
Guolin Ke, Qi Meng, Thomas Finlcy, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tic-Yar Liu. Lightgbm: highly efficient gradient boosting decision tree. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
Immun Kim, Aaditya Ramdas, Aarti Singh, and Larry Wasserman. Classification accuracy as a proxy for two-sample testing. Annols of Statistics, 49(1):411-4344 2021.
Akim Kotelníkov, Dmitry Baranchuk, Ivan Rubachev, and Artem Babenko. Tabddpm: Modelling tabular data with diffusion models. In International conference machine learning, pp. 17564 17579. PMLR, 2023.
Stefan Kramer, Nada Lavrač, and Peter Flach. Propositionallzation approaches to relational data mining. Relational data mining, pp. 262-291, 2001.
Anton D. Lautrup, Tobias Hyrup, Arthur Zimek, and Peter Kamp.. Syntheval: a framework for detailed utility and privacy evaluation of tabular synthetic data. Data Mining and Knowledge Discovery, 3911,, 2024. doi:
Jiayu Li, Zilong Zhao, Vikram Chundawat, Biplab Sikdar, and Y. C. Tay. Irg: Generating synthetic relational databases using deep learning with insightful relational understanding, 2024. URL https
7/ /aaxxiv.rr//bs///////... 15187
David Lopez-Paz and Maxime Oquab. Revisiting classifier two-sample tests. In International Conference on Learning Representations, 2017.
Scott M Lundberg and Su-In Lee. A unified approach interpreting model predictions. Advances in neural
information processing systems, 30, 2017.
15

---

Published in Transactions on Machine Learning Research (02/2026)
Ciro Antonio Mami, Andrea Coser, Eric Medvet, Alexander T. P. Boudewijn, Marco Volpe, Michael Whitworth, Borut Svara, Gabriele Sgroi, Daniele Panfilo, and Sebastiano Saccani. Generating realistic synthetic relational data through graph variational autoencoders, 2022. URL https://axxi..orr/abb/////
16889
Andrew Kachites McCallum, Kamal Nigam, Jason Rennie, and Kristic Seymore. Automating the construction of internet portals with machine learning. Information Retrieval, 3:127 163, 2000.
Ryan McKenna, Brett Mullins, Danicl Sheldon, and Gerome Miklau. Aim: an adaptive and iterative mechanism for differentially private synthelic data. Proceedings of the VLDB Endowment, 15(11), 2022.
Anna Montoya, LizSellier, Meghan O'Connell, Wendy Kan, and alokgupta. Airbnb new user bookings, 2015. wew.
URL https: http///awaaaeacaoe//eetatpttttee
Jan Motl and Oliver Schulte. The ctu prague relational learning repository, 2024. URL attps://arxiv.org abs/ /////1111.0000 03086
Beata Nowok, Gillian M Raab, and Chris Dibben. syuthpop: Bespoke creation of synthetic data in r. Journal of statistical software, 74:1-26, 2016.
Eirini Ntoutsi, Pavlos Fafalios, Ujwal Gadiraju, Vasileios Iosifidis, Wolfgang Nejdl, Maria-Esther Vidal, Salvatore Ruggieri, Franco Turini, Symeon Papadopoulos, Emmanouil Krasanakis, et al. Bias in data-driven artificial intelligence systems- an introductory survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 10(3):e1356, 2020.
Wei Pang, Masoumch Shaficinejad, Lucy Liu, Stephanic Hazlewood, and Xi Hc. ClavaDDPM: Multi-rclational data synthesis with cluster- puster-guided diffusion models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Theodore Papamarkou al. Position: Topological deep learning is the new frontier for relational learning. In Proceedings of the 41st International Conference Machine Learning, volume 235 of Proceedings of Muchine Learning Research, pp. 39529 39555. PMLR, 2024.
Neha Patki, Roy Wedge, and Kalyan Voeramachaneni. The synthetic data vault. In 2016 IEEE International Conference on Dala Science and Advanced Analylies (DSAA), pp. 399 410, 2016.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and F. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825 2830, 2011.
Zhaozhi Qian, Bogsan-Constantin Cebere, and Mihacla van der Schaar. Synthcity: facilitating innovative use cases of synthetic data in different data modalities, 2023a. URL
Zhaozhi Qian, Rob Davis, and Mihacla van der Schaar. Synthcity: a benchmark framework for diverse use cases of tabular synthetic data. In seirtg- Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023b.
Pranav Rajpurkar, Emma Chen, Oishi Banerjee, and Eric J Topol. Ai in health and medicine. Nature medicine, 28(1):31 38, 2022.
Joshua Robinson, Rishabh Ranjan, Weihua Hu, Kexin Huang, Jiaqi Han, Alejandro Dobles, Matthias Fey, Jan Eric Lenssen, Yiwen Yuan, Zecheng Zhang, Xinwei He, and Jure Leskovec. Relbench: A benchmark for deep learning on relational databases. In The Thirty-eight Conference on Neural Information Processing Systems Dalasets and Benchmarks Track, 2024.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on securily and privacy (SP), pp. 3 18. IEEE, 2017.
Andrey Sidorenko, Michael Platzer, Mario Scriminaci, and Paul Tiwald. Benchmarking synthetic tabular data: A multi-dinensional evaluation framework, 2025. URL http:///wwxiivvvaaa///////e
16

---

Published in Transactions on Machine Learning Research (02/2026)
Joshua Snoke, Gillian M. Raab, Beata Nowok, Chris Dibben, and Aleksandra Slavkovic. General and Specific Utility Measures for Synthetic Data. Journal of the Royal Stalistionl Society Series A: Stalistics in Socieby, 181(3):663 688, 03 2018. ISSN 0964-1998.
Aivin V. Solatorio and Olivier Dupriez. Realtabformer: Generating realistic relational and tabular data using transformers, 2023. URL htttp::/aarxx..oor//ab/////0 02041
Erik Strumbelj and Igor Kononenko. An efficient explanation of individual classifications using game theory. The Journal of Machine Learning Research, 11:1-18, 2010.
C Task, K Bhagat, and G Howarth. Sdnist v2: Deidentified data report tool. National Institute of Standards and Technology, 2023. doi: (0.18434/mds2-2900
Paul Tiwald, Ivona Krchova, Andrey Sidorenko, Mariana Vargas-Vieyra, Mario Scriminaci, and Michael Platzer. Tabularargn: A flexible and efficient ruto-regrossive framework for generating high-fidelity synthetic data, 2025. URL
22001
Jan Tönshoff, Martin Ritzert, Eran Rosenbluth, and Martin Grohe. Where did the gap go? reassessing the long-range graph benchmark. Transactions on Machine Learning Research, 2024. ISSN 2835-8856.
Vector Institute. Announcing the Winners of the MIDST Challenge! http:://vectorrnnsiiiteee 1o/MIDST/ 2025. Accessed: 2025-05-15.
Will Cukierski Walmart. Walmart recruiting store sales forecasting, 2014. URL https://kaggle.com
compotiiioons//laaattteeeeeee
Joshua Ward, Chi-Hua Wang, and Guang Cheng. Data plagiarism index: Characterizing the privacy risk of
data-copying in labular generative models, 2024. URL httppp//aardii.orr//////20....00
gdantel Wendy Kan. Telstra network disruptions, 2015. URL http::/laagga..co////m/eetttttts
Sohier Dane World Bank. World development indicators, 2019. URL http://ww..kkglll.ccmnatattes
theworldbaiin /hterorrdbbaan///oooordddeeeeeee httpr//eerrreappeeeeeeiiiiiiatrr
Xiaoyu Wu, Yifei Pang, Terrance Liu, and Steven Wu. Winning the midst challenge: New membership inference attacks on diffusion models for tabular data synthesis, 2025. URL http::::///xxxv.org//aa/////// / 12008
Kai Xu, Georgi Ganev, Emile Joubert, Rees Davison, Olivier Van Acker, and Luke Robinson. Synthetic data generation of many-to-many datasets via random graph generation. In The Eleventh International Conference on Learning Representations, 2023.
Lei Xu, Maria Skoularidou, Alfredo Cuesta- and Kalyan Vecramachaneni. Modeling tabular data using conditional gan. Advances in neural information processing syslems, 32, 2019.
Zexi Yao, Nataša Krčo, Georgi Ganev, and Alexande de Montjoye. The der delusion: Measuring the privacy risk of synthetic data. In European Symposium on Research in Computer Securily, pp- 469-487. Springer, 2025.
EL Hacen Zein and Tanguy Urvoy Tabular data generation: Can we fool XGBoost ? In NewrIPS 2022 First Table Representation Workshop, 2022.
Hengrui Zhang, Jiani Zhang, Zhengyuan Shen, Balasubramaniam Srinivasan, Xiao Qin, Christos Faloutsos, Huzefa Rangwala, and George Karypis. Mixed-type tabular data synthesis with score-based diffusion in latent space. In The Twelfth International Conference on Learaing Representations, 2024.
Jun Zhang, Graham Cormode, Cecilia M Procopiue, Divesh Srivastava, and Xiaokui Xiao. Privbayes: Private data release via bayesian networks. ACM Transactions on Database Systems (TODS), 42(4):1-41, 2017.
Zilong Zhao, Aditya Kunar, Robert Birke, and Lydia Y Chen. Ctab-gan: gan:: Effective table data synthesizing. In Asian Conference on Machine Learning, pp. 97-112. PMLR, 2021.
17

---

Published in Transactions on Machine Learning Research (02/2026)
Appendix
A A Survey of Synthetic Relational Database Generation Methods
The Synthetic Data Vault (SDV) PPtki et al. 2016) introduced the first learning- based method for generating relational databases. The method is based on the Hierarchical Modeling Algorithm (HMA) synthesizer, which is a multivariate version of the Gaussian Copula method. The method converts all columns to a predefined set of distributions and selects the best-fitting one. To learn dependencies, columns are converted to a standard normal before calculating the covariances. Tables are modeled with a recursive conditional paramcter aggregation technique, which incorporates child table covariance and column distribution information into the parent table. The method requires the relational structure or metadata, which has since become a common practice.
The work of Mami et al. (0022) leverages the graph representation of relational database using Graph Variational Autoencoders. They focus on the case of one primary table connected by an identifier to an arbitrary number of secondary tables. The approach begins by transforming categorical, datetime, and numeric attributes into a normalised numeric format using an invertible function. Subsequently, all tables' attributes merged into single-table, where rows from each table are vertically concatenated. This merged table, along with an adjacency matrix based on foreign key relations, forms a homogeneous graph representation of the datasct. Message passing is then applied to this graph reprosentation using gated recurrent units (GRU). Following the message passing phase, the data is processed through a variational autoencoder, which encodes the joined table and random samples are taken from its latent space. These samples are then decoded back to the data space.
Composite Generative Models (Canale al. 2022 propose generative framework based on codecs for modeling complex data structures, such relational databases. They define codec a quadruplet: C = (E,,,,,L), consisting of an encoder E producing embeddings and intermediate contexts, decoder D for distribution representation, a sampler S and loss function L. The authors define the following codecs: Categorical and Numerical Codecs for individual columns, while composite data types are encoded using Struct and List Codecs, allowing for relational database synthesis. They also propose a specific implementation using causal transformers as generative models.
The Row Conditional-TGAN (RCTGAN) (Gueye et al. 2023 extends the conditional tabular GAN model (Xu et al. 2019) to relational databases. RCTGAN incorporates data from parent rows into the child table GAN model, allowing it to synthesise data conditionally on the connected parent table rows. The ability for conditional synthesis allows the method to handle various relationship schemas without additional processing. They enhance RCTGAN to capture the influence of grandparent rows on their grandchild rows, preserving this connection even when the relationship information is not transferred by the parent table rows. Database synthesis is based on the row conditional generator of RCTGAN model trained for each table. First, all parent tables are synthesised, followed by sampling the tables for which parents are already sampled. This allows using the synthesised parent rows as features when synthesizing child table rows.
The Incremental Relational Generator (IRG) [Li et al., 2024) uses GANs to incrementally fit and sample the relational dataset. They first define a topologically ordered sequence of tables in the dataset. Parent tables are modeled individually, while child tables undergo a three-step generation process. First, a potential context table is constructed by combining data from all related tables through join operations and aggregation. Then, the model predicts the number of child rows to be generated for cach parent row, which they call its degree. They then extend the context table with corresponding degrees. Taking this table context, they use conditional synthetic tabular data generation model to generate the child table.
The Realistic Relational and Tabular Transformer (REaLTabFormer) (Solaoorio & Dupricz 2023 focuses on synthesizing single parent relational data and employs a GPT-2 encoder with a causal language model head to independently model the parent table. The encoder is frozen after training and used to conditionally model the child tables. Each child table requires a new conditional model, implemented as a sequence-to-sequueeee (Seq2Seq) transformer. The GPT-2 decoder with a causal language model head is trained to senthesise observations from the child table, accommodating arbitrary-length synthetic data
18

---

Published in Transactions on Machine Learning Research (02/2026)
conditioned on imput. While this method supports conditional synthesis of child rows, only one level is supported by this method.
Xu et al. 2023 propose a method for modeling many-to-many (M2M) datasets via random graph generation. They leverage a heterogeneous graph representation of the relational data and propose a factorization for modeling the graph representation incrementally. First, the edges of the graph are generated unconditionally using a random graph model. Second, one of the tables is generated conditionally on the topology of edges. One way to achieve such conditioning is by using a node embedding. Lastly, the remaining tables are generated using the conditional table model, which requires the generation of each node of the table based on the currently generated tables and all connections. They achieve this by using set embeddings to conditionally generate connected tables. The authors propose two variants using different conditional table models BayesM2M and NeuralM2M.
The Cluster Latent Variable guided Diffusion Probabilistic Models (ClavaDDPM) (Pang et al., 2024) utilizes classifier-guided diffussion models, inlegrating clustering labels intermediaries between tables connected by foreign-key relations. The authors first propose model for generating a single parent-child relationship. The connection between the tables is modeled by a latent variable obtained using Gaussian Mixture Model clustering. ClavaDDPM learns a diffusion process on the joint parent and latent variable distribution, followed by training a latent, variable classifier on the child table to guide the diffusion model for the child table. Additionally, it includes a model to estimate child group sizes, to preserve relation cardinality. The authors then extend this to more parent-child constraints through bottom-up modeling and address multi malti-parent scenarios by employing majority voting to mitigate potential clustering inconsistencies.
Hudovernik 2024) adapts the tabular latent diffusion-based model TabSyn (Zhang et al. 2024 for conditional generation of relational databases. The method Relational Graph-Conditioned Latent Diffusion (RGCLD) utilizes a heterogeneous graph representation of a relational database. The rows of each table are represented as nodes of a particular type, and the foreign keys between tables are represented by edges connecting the nodes. The method trains a graph neural network for each table to encode the relationships between connected tables. The embeddings of the GNN are used to guide the diffusion process in the latent, space. During sampling, the method generates tables sequentially based on topological order defined by the datasct's 's schema.
Tiwald et al. (025) propose TabularARGN, an auto-regressive model for generating synthetic data for flat and sequential tables. TabularARGN is a shallow any-order auto-regressive network architecture. The method homogenizes diverse datatypes by discretizing them and is trained by minimizing the categorical cross-entroopy loss across the discretized attributes. TabularARGN is designed to generate labular senthetic data by learning the full set of conditional probabilities across features in a datasct. The sequential table TaGN model can handle sequences of arbitrary lengths. Tab''s sequential table model can utilize a flat table as context during training and generation, enabling the synthesis of two-table setups, such as a flat table containing time-independent information (e.g., bank customer data) and a sequential table containing time-dependent data (the bank customer transaction histories). The method was open-sourced by the commercial provider Mostly.ai, while the method is specialized for sequential data, they also support generating multi multi-parent schemas. For these datasets, the method will retain the context for one of the parent tables and retain the referential integrity for the rest
In contrast to neural-based approaches described above, marginal-based approaches focus on preserving low-dimensional measurements of the dataset, such as marginal queries, often under differential privacy (DP), and are typically evaluated by the accuracy of synthesized marginal queries. While marginal-based methods
for single-table synthesis are well-established (Zhang et al. 2017 McKenna et al. 2022 Cai et al. 2021 their extension to relational data is a more recent focus. For instance, PrivLava Cai et al. 2023) achieves DP in relational data through graphical models with latent variables to represent inter-table dependencies. Following this, (Kapenckakis et al. 2024) proposed a method for synthesizing relational data, with a focus data with sequential aspects like electronic health records. More recently, (Alimohamnadi et al.) 2025)
adapt single-table DP generators to relational data by modeling relationships with a learned bi-adjacency
For details on multi-table generation see fttp:://lleelllllllccccccoooooet multi-table
19

---

Published in Transactions on Machine Learning Research (02/2026)
matrix, and PrivPetal (Cai al. 2025 synthesizes DP relational data by modeling flattened version of the table using permutation relations.
B SyntheRela: Synthetic Relational Database Generation Benchmark
We provide our work as a Python package SYNTHERELA. The main goal of the package is the evaluation of the quality of synthetic relational databases. We can compare multiple methods across multiple databases with the Benchmark class or evaluate a single method on single database with the Report class. All of the results of the benchmark saved as JSON files and then parsed by our package for results summarization and visualization. The package is open source under the MIT license and can be easily extended with new methods, evaluation metrics, or databases.
B.1 Using SyntheRela
This section provides technical guide for users to evaluate synthetic relational data using SYNTHERELA.
Installation and Setup. The primary way to interact with the benchmark is through the syntherela Python package. is designed to be highly modular and can be easily installed via the Python Package Index (PyPI): pip install syntherela.
Data Organization and Structure. Our current implementation assumes that synthetic data generated in CSV format. It requires a specific directory structure to ensure the Benchmark class correctly maps ground-truth relational tables to their synthetic counterparts:
bata/original/... Contains the ground-trute relational databases. Alongside the csv files the firectory must also contain a metadata. json filc defining the databasc schema.
data/syytheticc/.. //ethod nwwe/run/ sample/: Contains the synthetic databases for evaluation. We further subdivide by run and sample subdirectories account for sample variability.
The Benchmark Class. The Benchmark class is the central interface of the package. It is compatible with all metrics described in the main paper and provides an extensible framework for users to include their own custom metrics. The class automates the evaluation pipeline, allowing multiple synthesis methods and datasets to be evaluated simultaneously a single execution. After processing, the benchmark stores all results in structured JSON format for standardized analysis and leaderboard submission.
Example. The following example demonstrates how to programmatically execute the benchmark to evaluate selected methods:
Import metrics and benchmark class
from benchhek benchmark import Benchmark
from syntherels muntherola.metrices metrics. single_columa statistical import ChiSquareTest from cachherela.mttricce. mubiiisssllttttaaalllaa statistical import CardmalltyySiapeSiiiaatrtt
Configuration
benchmark Benchmark
results_dirr
11
multi nut.._.hhh.geeriifffoccaaaaaaaaa (),
18
methods-("SDV", " "RCTGAN"] # Simultaneous evaluation
15 )
17 Run evaluation and generate JSON results
18 benchmark. run
20

---

Published in Transactions on Machine Learning Research (02/2026)
B.2 Evaluation Metrics
We list the evaluation metries for data. fidelity and utility currently supported in our benchmark in Table g based on the granularity of the data they evaluate. We define three levels of qranularity; single-columi metrics that evaluate the marginal distributions, single-table metrics that evaluate tables, and multi-table metrics that evaluate the relational aspects. For single column fidelity, we use the Kolmogorov-Smirrov and X² statistical tests, total variation, Hellinger, Jensen-Shannon, and Wasserstein distances, alongside a single columm C2ST. We include column pair correlation similarity, maximum mean discrepancy, pairwise correlation difference, and C2ST for single-table fidelity. We evaluate multi- table fidelity with cardinality shape similarity and k-hop correlation similarity alongside C2ST-Agg- We also implement the Parent- Child C2ST (PC-C2ST) for comparison with related work. In our implementation of PC-C2ST, we first split the parent table rows (as in C2ST-Agg) and then apply denormalization. This way preserve the i.i.d. assupption for the classifier two-sample test, while also including all information from the two tables. Single-column utility is generally covered by fidelity metrics and not evaluated in related work. For single-table utility, we implement tabular machine learning utility metrics, and for multi-table utility, we include RDL-utility.
Table 9: Evaluation metrics supported in the benchmark.
Single Column
Single-Table
Multi-Table
Statistical
KS Test, X²2 Test
Column pair correlations
cardinality shape similarity k-hop correlation similarily
Distance
Total Variation, Hellinger,
Maximum Mean
Discrepancy,
Jenson-Shannon, Wasserstein
Pairwise Correlation
Difference
Detection
C2ST
C2ST
C2ST-Agg, PC-C2ST
Utility
Tabular ML- ML-Utility
Relational DL-Utility
B.2.1 C2ST with Aggregation
Fidelity methods are concerned with measuring the similarity between two databases with the same schema but different data. Typically, these will be the real database DREAL and synthetic database DsYN, with the goal of detecting if, to what extent, and where the synthetic data differ from the real data.
Let a relational database D be a collection of tables T = [T] ..... d } E A define the tables' attributes. Each table is a set T = {V1. Bny) consisting of clements e, called rows. Each row v E T has three components e = (po, Kn, v.p). primary key Pe that uniquely identifies the row v; the set of foreign keys Kr {Pe' : v T' and (T) E R}, where Pe the primary key of the row v'; and the set of values F. ((a,z): : E Ar} corrosponding to attributes of table T.
Tn) and a schema S = (R, where
R T x T are the relations between the tables and An: = {
A Algorithm ] describes how we add aggregations to the target table. For each child table, we add Cownt a count of the number of child rows corresponding to a parent row. For each attribute (i.e. column) in each child table, we compute an aggregation attribute (mean, count, etc.). The aggregation attributes are added to the target table. In practice, different aggregation functions may be applied, as long as they maintain the i.i.d. assumption of the data. In our benchmark, we use column means for numerical columns and the number of distinct categories in categorical columns along with the child row count.
Our implementation allows us to directly control how many levels of aggregations we add (similarly to k-hop correlation similarity (Pang et al. 2024h). Each level accounts for one application of Algorithm [ When aggregating for 1 level, only the information directly from the table's children is aggregated; at the second level, the aggregated grandchild columns are also added to the table. In our benchmark, we choose to add only one level of aggregation, as most methods struggle to preserve the relationships at distance k 1.
It has been shown that the accuracy-based approach to two-sample testing is consistent and controls for Type error and (asymptotically) Type IT error (see Kim et al. (2021) for theoretical results and summary
21

---

Published in Transactions on Machine Learning Research (02/2026)
Algorithm 1 Relational Aggregation.
Require: relational database D with tables T and relational schema 8 = {R, {Az, ... Ar.
Require: Largel table
1: agggregation <-
2: for each C. E (C () R] do
3: Add CountRows(Ci, T) to aggregation.Attributes
> Count number of child rows for each row in T. > Iterate through child table columns.
4: for each oj E Ac; do C₁ 5: Add Agg(Ci, o T) to aggregationAttributes
b Compute aggregation (e.g,, mean, distinct...).
end for end for
7:
for each e T do
Attriggtt do 10: Add (a.name, value) to 2
9: for each
Append computed aggregation attributes.
11: end for
12:
end for
13: return T.
D Return table with aggregations applied.
of empirical results). In practice, we are also interested in finite sample behavior. Experiment-based recommendations show that the approach should have an advantage power when the data are well- structured or we have a lot of data, or when it is difficult to specify a test statistic, which very common for higl-dinensional data. Therefore, the large, higher-dimensional and structured nature of relational data is perfect fit for C2ST.
B.3 Datasets
B.3.1
Relational Databases and Sampling Procedures
An important issue with evaluating relational data is that representative sampling is difficult (Buda et al., Gemulla et al. 2008). If the dataset does not include a time component or the relationships are non-linear, the sampling becomes non-trivial and directly impacts the performance of the generative method. Even if the data. have strict hierarchy between tables, the rows in child table are related via their parent, which breaks the assumption of í.i.d. sampling. This makes splitting the dataset into reprosentative train and lest set difficult. Consequently, the methods for synthesizing relational dalabases are typically trained using the entire original dataset.
2013
We organize the datascts used in related work based on the structure of their relational schema. Datasets using only linear relationships (one parent and one child table) include Airbnb (Montoya et al. 2015 and Rossmann Store Sales (FlorianKnauer (015).. While this structure may be sufficient for some practical applications, Gueye et al. (2023) and Xu et al. (2023) highlight the need for methods supporting more complex, multiple-parent relational structures found in datasets like MovieLens (Harper & Konstan, 2015) and
World Development Indicators (World Bank 2019). Datasets including multiple child tables include Telstra Network Disruptions (Wendy Kan 2015). Walmart Recruiting - Store Sales Forecasting (Walmart) 2014), and
Mutagenesis (Debnath et al.] 1991). Datasets with multiple children and parents include Coupon Purchase
Prediction Kato et al.] 2015), World Development Indieators (World Bank 2019), MovieLens (Harper
Konstan 2015 Biodegrudability [Blockeel et al. 1999) and Berka Berka et al. (2000).
B.3.2 Benchmark Datasets
Table 1 summarizes the relational datasets used in our benchmark. Six datasets are from related work and we add the Cora dataset by McCallum et al.] (2000), which contains a simple yet challenging relational schema, and F1 F1 (2021) from the RelBench benchmark. We include datasets per hicrarchy type to progressively add complexity in generation. The datasets used in our evaluation are diverse in terms of the number of columns, tables and relationships.
22

---

Published in Transactions on Machine Learning Research (02/2026)
The Airbnb (Airbnb 2015) dataset includes user demographies, web session records, and summary statistios. It provides data about users' interactions with the platform, with the aim of predicting the most likely country of the users' next trip. See Figure 9a for schemn..
The Berka (also known as the Financial dataset) Berka et al. 2000 dataset contains 606 successful and 76 unsuccessful loans along with their information and transactions. The standard task to predict the loan outcome for finished loans (A vs B) the time of the loan start. See Figure 12 for schema.
The Biodegradability dataset (Blockeel et al. 1999) comprises collection of chemical structures, specifically 328 compounds, each labeled with its half-life for aerobic aqueous biodegradation. This dataset is intended for regression analysis, aiming to predict the biodegradation half-live activity based on the chemical features of the compounds. See Figure 13 for schema.
The Cora dataset (McCallum et al., 2000) is widely-used benchmark dataset in the field of graph representation learning. It consists of academic papers from various domains. The dataset consists of 2708 scientific publications elassified into one of seven classes and their contents. The cilation network consists of 5429 links. See Figure 10b for schema.
The F1 dataset F1 2021 contains Formula 1 racing data and statistics dating back to 1950. It contains information on drivers, constructors, race results, and standings covering every season in F1 history. See Figure I for schema.
The IMDB MovieLens dataset (Harper & Konstan 2015) comprises information on movies, actors, directors, and users' film ratings. The dataset consists of seven tables, each containing at least one additional feature besides the primary and foreign keys. See Figure 14 for schema.
The Rossmann Store Sales (FlorianKnauer 2015 features historical sales data for 1115 Rossmann stores. The dataset consists of two tables connected by a single foreign key. This makes it the simplest type of relational dataset. The first table contains general information about the stores and the second contains sales-related data. See Figure 9h for schema.
The Walmart dataset (Walmart 2014) includes historical sales data for 45 Walmart stores across various regions. It includes numerical, date-time and categorical features across three connected tables store, features and depls. The dataset is from Kaggle competition, with the task of predicting department-wide sales. See Figure 10a for schema.
B.4 Comparison with Existing Evaluation Tools
Several packages for evaluating synthetic tabular data exist (Nowok et al. 2016 Task et al. 2023 Lautrup et al.
2024 Sidorenko et al. 2025) with the most popular and comprehensive package being Syntheity (Qian et al.
2023a b). It supports many statistical, privacy and detection-based (with several different models)
metrics.
The only package that supports multi-table evaluation SDMetrics (DataCebo 2022
It includes multi-table metrics cardinality shape similarity and parent-child detection with logistic detection and support vector classifier. The package is not casy to extend and limits the adaptation of motrics. We re-implement detection metrics (discriminative detection, aggregation detection, and parent-child detection) to be used with an arbitrary classifier supporting the Scikit-learn (Pedregosa et al., 2011 Buitinck et al., 2013) classifier API. In SDMetrics, the results of different metrics are aggregated into a single-value, which limits the comparison of individual metrics between the methods and datasets. We re-implement the distance and statistical metrics so that each statistic, p-value, and confidence interval is easy to access.
Our benchmark package can be casily extended with new methods, metries, and datasots. The process for adding custom metrics and new datasets described in /tttttts///giiiiii cww.///arriiiuuirorttt////ttttea blob/main//ocs//DDD ADDING A METRIC.
23

---

Published in Transactions on Machine Learning Research (02/2026)
B.5 License and Privacy
We oblain the Airbnb, Biodegrudabilily, Cora, IMDB MovieLens, Rossmann and Walmart dalasets from the public SDV relational demo datasets repository (nttps://docs.. sdv htee///d//iigggetaaaaaea data/ attaapprrppratio/oodddnggg
accessed June 6th, 2024.). The SDV project is licensed under the 1-o--m--------fllltrredddda which allows use for research purposes. The Berka dataset was obtained from the CTU Relational Dataset. Repository (MotI & Schulte 2024) an open-access repository of relational databases. The Fl dataset was obtained using the official RelBench implementation Robinson et al. (022)). It is released under the CC-BY- 4.0 license hhttps://g thub. http://bbbbbbbbbbbcc------- 001--40001---leeee---- We remove all textual columns from the F1 dataset as relational generative methods are generally unable to generate those. We manually check all of the datasets to ensure they do not include any personally identifiable information. Some of the datasets contain processed columns, including aggregations of numerical values and connected table rows (e.g., nb rows in (related table)). The authors of SDV (Patki et al. 2016) confirmed that these aggregations are not part of the original datasets, so we post-process all of the datasets to include only the columns found in their original form and update the metadata accordingly. We adapt some of the metrics from the SDMetrics (DataCebo 2022) (MIT License) and Synthcity (Qian et al.) 2023a b) (Apache-2.0 License) synthetic data evaluation tools. We acknowledge Flaticon.com for providing the icons used in Figure
Business Source License 1.1 (https://githut) http://///.sttto..oo//o/////////
C Additional Experiments
C.1 Privacy Evaluation
In this section, we present additional results from our privacy evaluation experiments, providing a more granular view of the behavior of each generative model. We visualize the full DCR distributions for all evaluated mothods, and report the precision and recall metrics of the membership inference attacks.
As shown Figures and the DCR distributions for most methods exhibit substantial overlap between synthetic training and synthetic holdout distances, indicating no systematic data copying, However, Claval> DPM the users table shows noticcable skew toward lower distances between synthetic and training records, consistent with the elevated DCR score reported in Table 7 suggesting potential data copying and privacy issues.
In Tables II and 10 we report the recall and precision of an adversarial classifier, respectively. Rocall reflects the probability that rows from the training data are correctly identified as such, and therefore serves as a direct measure of membership disclosure risk. Precision, on the other hand, reflects the probability that a record predicted as a member truly originates from the training set, and serves measure of the reliability of the attack's positive predictions (a scores of 50% is analogous to random guessing).
We also examine an additional aspect of privacy: the potential of the C2ST test to detect data copying, similar to the DCR score. To explore this, we conduct a data copying simulation across the benchmark datasets. The experimental setup and a discussion on how common implementations C2ST metrics may inadvertently mask such privacy risks are provided in Appendix C.4
C.2 Single Column Performance
We evaluate the marginal distributions of individual columns by evaluating the column shapes and using the detection metric. Results are summarized in Table 12 TabularARGN performs best on modeling marginal distributions. We hypothesize this is a result of their preprocessing, which removes outliers and discretizes all columns. The method is closely followed by ClavaDDPM, which models individual tables best.
C.3 Shortcomings of Logistic Detection
significant limitation of LD is its inability to capture interactions between columns. can thus assign a perfect fidelity score a dataset that is completely corrupted. In this section, empirically show this shortcoming. We conduct the experiment by selecting table from each dataset.
As explained in Scction 2.2
24

---

Published in Transactions on Machine Learning Research (02/2026)
Train Holdout
Train Holdout
10
10
Frequencyllog
Frequency(log
10
10
10
10
10
0
1
2
4
5
6
7
8
0
20
40
60
80
100
120
140
DCR
DCR
(a) TARGN
(b) RGCLD
10
Train
Holdout
10
10
Frequancy(log)
Frequency(logg
10
10
10
10
10
10
0
2
4
6
0
6
8
10
DCR
DCR
(c) ClavaDDPM
(d) RCTGAN
10
Train Holdout
Train Holdout
10
Frequencyllog
Frequency(log)
10"
10
10
10"
10
3
5
6
10
14
DCR (e) REALTABE.
DCR
(f) SDV
Figure 5: DCR Distributions for the users table in the anonymized Airbnb dataset. The y-axis is log-scaled for better comparison.
(with the exception of CORA which no table has two columns, which are not primary or foreign keys). We first select the table and split it half to simulate the original table and a perfectly generated (by the underlying DGP) synthotic table. We then copy the 'generated" table and randomly shuffle values in cach column, completely ruining the fidelity of the dataset while keeping the marginal distributions intact. We then evaluate the perfectly generated and shuffled datasets using LD and C2ST using XGBoost. The results are visualized in Figure
Notably LD assigns both versions of the dataset the same score, labeling them indistinctive from the original data. If the fidelity aspect of interest would be solely the marginal distributions, the LD results would be more appropriate than those of C2ST using XGBoost (as marginals are identical in both datasots). However, given that we are interested in single-table fidelity, our experiment showcases a fundamental shortcoming of LD measure of single-table fidelity.
25

---

Published in Transactions on Machine Learning Research (02/2026)
10
Train Holdout
10
Train
Holdout
Frequency(log
10
Frequency(log)
10
10
10 -2
10
10
10
10
0
5
0
2
3
4
DCR
DCR
(a) TARGN
(b) RGCLD
10"
Train
10
Train
Holdout
Holdout
10
10
Frequencylogg
Frequency(lap
10
10
10
10
10
DCR
DCR
(c) ClavaDDPM
(d) RCTGAN
10
Train Holdout
10
Holdout
Frequancy(log)
Frequency(log)
10"
10
10
10
10
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
0
3
4
5
6
(e) REALTABE.
(f) SDV
Figure 6: DCR Distributions for the sessions table in the anonymized Aárbrb dataset. The sessions table consists primarily of categorical columns, resulting in highly discretized DCR distributions. Despite this, most methods maintain a reasonable level of privacy. The y-axis is log-scaled for better comparison.
C.4 Classifier Accuracy Data Copying Diagnostic
We investigate how the accuracy of the discriminative model in classifier two sample testing can be used to diagnose data copying in Figure g We also demonstrate how the classifier performance commonly reported in LD (2 max(AUC, 1 1) masks this issuc.
As in the previous experiment, simulate perfect synthetic generating dataset by splitting the original table in half. However, instead of introducing corruption into the second half, create exact copy of the original data (i.c., the first half). The commonly used LD implementation fails to detect data copying and assigns the copied data perfect score. In contrast, C2ST (with XGBoost) accuracy successfully detects data copying as accuracy drops significantly below 50%. We then examine the behaviour of C2ST when only a portion of the data is copied. We keep portion the dataset as an identical copy and sample the rest of the values from the "perfectly generated' half. For most of the datasets, even when relatively low percentage of the data is copied, the model detects the duplication.
26

---

Published in Transactions on Machine Learning Research (02/2026)
Table 10: MIA Recall denotes the probability that the attack predicts record as a member given that it is truly a member of the training set. Values signif- icantly above 50% indicate privacy concerns.
Table 11: MIA Precision denotes the probability that a record truly a member of the training set given that the attack predicted it as a member. We estimate uncertainty with standard errors.
Method
Sessions
Users
Method
Sessions
Users
TARGN
11.27 20.15
17.58
$2.06
TARGN
92.14 20.48
99.49
RGCLD
2.14 10
61.31
22:21
RGCLD
86.04
10.05
99.78
10.00
CLAVA
30,03 +2.74
37.49
+0.71
CLAVADDPM
92.81
+0.71
99.94
+0.08
RCTGAN
28.16 20.85
98.55
10.35
RCTGAN
93.11
20.21
99.72
10.09
REALTABF.
3.14 +0.38
0.19 +0.07
REALTABF.
70.76
+0.15
62.67
#17.02
SDV
0.03 10.00
0.03 10.02
SDV
85.00
10.00
35.00
221.79
SMOTE
91.02 10.17
85.24 2:034
SMOTE
97.14 20.18
99.86 20.03
Table 12: Single Column Results. We report the detection accuracy for C2ST (lower is better), and the KS/TV complement for column shapes (higher is better). For each dataset and metric, we report the average across all tables for three independent samples. SDV exceeds the sampling time limit on IMDB (TLE) and "_" denotes method is unable to generate the datasct. The best result is in bold and results within standard deviation underlined.
Dataset
Metric
TARGN RGCLD
ClavaDDPM
RCTGAN REALTABE.
SDV
MARE
Airbub
C2ST (1)
28.80
52.4020.3
55.342no2
56.37
61.682om
58.57=tea 70.362m.0
Shapes (t)
95. 75.70
95.37
94.42gom
89.18
71.66g
59.37
83.82
C2ST (4)
$1.07....
52.76.0.0
54.79.aax
54.98.0.2
53. 6r
59.99...10
Rossmann
Shapes (t)
96 96.
93.8020.5
94.05...a7
21.31
90.65.2
81.0520.19
C2ST (L)
59 25
59.34gm
52 60)
64. 12
59.08g
66. 20
Walmart
Shapes (t)
89. ca.s
88.43
92 :0.5a
s2.31
81.7
81.80
C2ST
58. 6d
60.00.1.50
47 25.32
57. 57.35.0020
70.77
Berka
Shopes (t)
82. 22.200gn
78.91
91. 1..22g
81 11.90gg
56.27 20
C2ST
61 2aos
57.69
60.76
63. 63.71amas
79.85
F1
Shapes (t)
81171a
91.04
84.65
89.68
62.62.0.37
C2ST
533.02a1. 10.362211
49. 86.2
53.74
IMDB
Shapes (f)
98. 10..4
93.61a2m
99.01 99.01.0.0
92.70
C2ST
55 42 ro.4a
61.85 11.87
57.81
61.24
Biodegradability
Shepes (t)
90.85
84.5323.4
90.91
79.4620.47
C2ST (4)
30.94gnar
m2.98
48.97
75.45
Cora
Shapes 92.65
88.27
96.38.
10.24
Relational Deep Learning Utility
We adopt the RelBench (Robinson et al. 2024) relational deep learning benchmark to support synthetic data into the relational deep learning utility framework. It designed to support training GNN models both on real-world and synthetic relational datasots, and evaluating on real held-out test set, in a train-on-syntheticc evaluate-on-redt paradigm. RDL-utility allows integration of new databases and models.
The framework creates two separate dataset-task pairs. One pair includes the synthetic dataset with training and validation splits used for model training. The other pair contains the real datasct with an additional test split used for final evaluation. Both pairs follow the same schema but are kept separate to avoid data leakage. Graph construction converts relational tables into heterogeneous graphs for both synthetic and real datasets. The process maps tables to node types and foreign key relations to edge types. Temporal splits are applied consistently to ensure only past data is available during training and testing. GNN models are trained with temporal awareness, restricting access to data strictly before the test time.
27

---

Published in Transactions on Machine Learning Research (02/2026)
Perfectly Generated Data
Shuffled Data
1.0
1.0
XGa
Clessifiaaion Accuracy
0.8
0.8
0.7
0.7
0.5
0.5
0.4
0.4
0.3
Arbob (users)
Blodegr
Immhecale))
INDB (mowiss)
(store)
Valmert (depts)
Aitosb (users)
Blodegr odabiity (eroee
IMEIO (nowes))
Restmann
(store)
Valmert
(deptt))
Figure Issues with logistic detection. For each dataset, we simulate a perfectly generated table by splitting the original table in half. We copy part of the table and shuffie the values in each column and thus completely ruin the fidelity of the table. While the XGBoost classifier can almost perfectly segment the corrupted rows, logistic regression assigns both of the datasets the same score.
Copied Data
Copying Behaviour by Dataset
LD maxiAUC, 1/2)
XGB
0.8
Classification Accuracy
0.4
0.2
Airbnb
Biodegrddability
IMDB
Rossmann
Walmart
0.0
Auterb (users]
IMDB (mnoves)
Rosimann (sture)
Welmart (depts)
0.0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
Mlstegracbblig
Fraction Copied
Figure 8: Detecting data copying with C2ST. The left plot demonstrates how the error estimation of LD (2 max(AUC, masks data copying, while C2ST with XGBoost detects across all datasets. In the right plot, we observe how copying only a fraction of the original data affects discriminative model accuracy, with accuracy consistently decreasing as more data is duplicated.
D.1 Utility Tasks
The summaary of the RDL-utility tasks is displayed in Table 13 We define the AutoComplete task the Rossmann, Airbnb, Walmart and Berka datasets based on the predefined tasks by the dataset authors from Kaggle competitions other dataset information. The task objectives per datasets are as defined as follows:
Rossmann [Regression]: We predict the daily number of customers for each store.
28

---

Published in Transactions on Machine Learning Research (02/2026)
Table 13: Task statistics for relational datasets used in evaluation.
#Rows of Task Table
Dataset
Task name
Task type
# Uniqqe Entities
%Train Vainin-Val//estt Entity Overlap
Train
Validation
Test
rossmann
Autocomplete Autocomplete
Regression
46,750
9,350
28,985
46,750
0.0
Walmart
Regression
6,925
2,952
11,946
6,925
0.0
Airbub berka fl
Autocomplete
Clussification
1,661
217
999
1,661
0.0
Autocomplete
Classification
328
196
158
328
0.0
Driver-top3
Classification
1,353
588
726
92
50.0
Walmart [Regression]:
We predict the weekly sales for cach department of each store.
Airbab [Classification]:
We predict whether a user has already made a booking or not.
Berka [Classification]:
We predict the binary status of the loan (successful or unsuccessful).
F1 [Classification]: We predict whether driver will qualify in the top-3 for race in the next month, which one of the original RelBench predictive tasks for this dataset.
D.2 Hyperparameter Tuning
As GNN models sensitive to hyperparameters Tönshoff al. (2024), perform exhaustive hyper- parameter search across all datasets and GNN variants. We use the Optuna library Akiba et al. (2019) to create hyperparameter soarch for each dataset and GNN variant pair on the original data. We run 100
experiments for each dataset-GNN pair and optimize the parameters specified in Table 14
Table 14: Hyperparameter search specification for GNN training. We rmn 100 experiments for each dataset-GNN variant pair.
Hyperparameter
Type
Search Space
Values
Learning rate
Categorical
{0.0001, 0.001, 0.01, 0.1, 0.5, 1.0) {1, 2, 3}
Number of layers
Categorical
Number of neighbors
Categorical
{-1, 128]
Weight decay
Continuous (log scale)
[le-9, 0.01]
Number of MLP layers
Categorical
{1, 2, 3)
Aggrogation method
Catcgorical
{sum, mcan, max, min)
The best hyperparameters used in RDL-utility are specified in the Reprodueibility section of the Appendix in
Table 15
D.3 Utility Baselines
To assess the effectiveness of RDL-utility in quantifying the utility performance of synthetic relational databases, we implement two baseline approaches that measure utility on single tables.
The single-table bascline evaluates utility using only the entity table. As the predictive model, we use LightGBM Ke et al. (2017), a gradient boosting decision tree implementation. The model is trained with ten iterations of automated hyperparameter tuning the training and validation splits, and the best configuration is evaluated on the test set.
The second baseline implements the aulomated feature generation deep feature synthesis (Kanter & Veera- machaneni 2015). Here, relational structure is incorporated through automated feature generation: for categorical columns, we apply count aggregation; for numerical columns, we use mean and sum; and for datetime columns, we extract the month and year. Aggregations are applied recursively throughout the database to produce flattened feature table. This table then used to train LightGBM with the same
29

---

Published in Transactions on Machine Learning Research (02/2026)
Table 15: RDI-utility hyperparameter specification. We list the best hyperparameters from the hyperparameter search of 100 runs on original data.
GNN
Dataset
LR
Layers
Neighbors
Weight Decay
MLP Layers
Aggr
Het
Rossmann
0.001
128
5.4e-05
sum
Walmart
1.0
2
128
2.6e-06
1
sum
Airbnb
0.0001
2.7c-07
Berka
0.01
2
128
1.3e-07
2
mean
F1
0.5
128
6.4c-05
2
mean
Rossmann
0.001
2.1e-08
sum
Walmart
0.5
2
128
1.6c-07
2
Airbnb
0.0001
1.8e-09
2
max
Berka
0.1
128
8.7c-06
2
min
F1
0.1
-1
3.5e-07
3
sum
Hetero-GIN
Rosmann
1.0
1.3e-05
8u1
Walmart
0.1
128
1.3e-08
2
Airbnb
0.001
-1
3.4e-04
2
max
Berka
0.1
128
1.6e-05
max
F1
0.5
3
-1
1.5c-06
mean
Rossmann
0.5
128
1.0c-04
3
min
Walmart
1.0
128
2.5e-08
2
Airbnb
0.001
-1
3.0e-04
1
Berka
0.01
128
2.3e-09
mAx
F1
0.1
128
9.9e-07
3
Hetero-Grae Serro-GraphSAGGe AGE
Rossmann
0.1
-1
0.00152
3
SUm
Walmart
0.5
2.1c-09
mean
Airbnb
0.001
2
-1
8.5e-07
Berka
0.001
8.5e-07
3
max
F1
0.1
128
4.0e-07
3
ReICNN
Rossmann
1.0
128
2.3c-05
3
Sum
Walmart
128
5.2c-05
sum
Airbnb
0.001
2
-1
4.4e-06
1
Berka
0.01
128
5.6c-06
2
min
F1
0.01
-1
6.5e-05
procedure as in the single-table baseline, allowing a direct comparison between learning from the entity table alone and learning with additional relational features.
E
Experimental Setup
E.1
Computational Resources
The generative methods were trained on NVIDIA 32GB V100S GPUs and H100 80GB GPUs. The total number of GPU hours spent across all experiments approximately 500. Results which do not require GPU were machines running AMD EPYC 7702P 64-Core Processor with 256GB of RAM. All experiments were performed on internal HPC cluster.
E.2 Reproducibility
E.2.1
Datasets and Data Splitting
Scripts for downloading the datasets and their motadata the SDV format (Patki al. 2016 are available in the project repository, well as the corresponding synthetic data samples for all methods to enable the reproduction of the benchmark results.
30

---

Published in Transactions on Machine Learning Research (02/2026)
We opt not to split the datasets into train, test, and validation sets for generative model training. When temporal information is included and the structure is non-linear the representative sampling in relational datasets is non-trivial. We delegate this to future work.
Due to computational limits (also reported by Solatorio Dupriez 2023 ) we subsample the Rossmann Store Sales, Airbub, and Walmart datasets. Additionally, we subsample the Berka and F1 datasets for the purposes of obtaining a held-out test set for our utility experiments.
Our choice to subsample these datasets may limit the generality of our findings to longer time horizons. As the evaluated methods are primarily deep-learning methods, they should generally achieve better performance given more data. However, we apply a fixed subsampling procedure and the same data is used for all methods. Meaning that our results remain comparable within our benchmark and should be interpreted as method comparisons under a consistent computational budget. We also provide full versions of these databases, fully compatible with our benchmark.
Rossmann Store Sales: Subsampled table historical, column Date by taking the rows of two month period from 2014-07-31 to 2014-09-30, similarly to Solatorio & Dupriez (2023).
Airbnb: Subsampled the dataset by only including the users who have less than 50 sessions and then sampled 10k users, as done by (Solatorro & Dupriez (2023).
Walmart: Subsampled on tables deparbments and fealures on the column Date by laking the rows from January 2012.
Berka: We use the data prior to 1998-01-01 and use the 1998 data as the test set.
Fl: We use the data prior 2010-01-01 for training. This corresponds to the test set timestamp the official RelBench implementation
E.2.2 Experimental Details and Hyperparameters
To provide some quantification of the variability from the non-deterministic nature the methods, we generated synthetic data for each of the methods for cach of the datasets 3 times with different fixed random seeds. We ran the benchmark for each replication.
Scripts for reproducing the generative model training and instructions for training commercial methods are included in the project repository.
It is possible that better performance could be achieved by investing more effort into parameter tuning. However, due to our choice not split the data, it was not clear how to optimize hyperparameters: therefore, we selected default hyperparameters for all methods (see Tables 16 and 17
F Broader impacts
The research introduces a benchmark for evaluating the quality of synthetic relational databases. This could improve the development of synthetic data, which may be beneficial in fields with privacy restrictions, for example, healthcare (Appenzeller et al. 2022), finance (Assefa al.) 2020), oducation (Bonnéry al. 2019), and in cases of limited or biased data Ntoutsi et al. 2020 Rajpurkar et al. 2022). However, there are possible negative aspects. Synthesis processos might have weaknessos that affect privacy and amplify biases already present in the original data. Also, synthetic data that resembles real data could be misused. Therefore, further work on privacy protection, bias reduction, and detection of synthetic data is needed for its proper use.
https://reebeechh etaafoor.ee///ataaa//rrrttt
31

---

Published in Transactions on Machine Learning Research (02/2026)
Table 16: Hyperparameter specification for RCTGAN RealTabFormer Solatorio & Dupriez 2023
Gueye et al.
2023
SDV (Patki al.
2016 and
model
lyperparameter
valtue
embeddiin dim generator dim discriminator dim generator
128
(256, 256)
(256, 256)
0.0002
generator derny diseriminator Ir discriminator dexay batch size discriminator epochs
1e-06
0.0002
1c-D6
500
RCTGAN
1000
10
grand_ parent
True
field transformers
None
None
rounding min valuue
aulo'
autoo
max value
"auto"
locales
None
werbose:
True
table_ sahle_sytthesizer
SDV
enforce_mii max values
True
deforce_rounding
True
numerical distributions
defantl distribtoittiittiionn
'beta"
epochs
100
batch size train size output length
8
512
5
tt___stpppg___heeeeee mask
0
numerie_ _ptarts____s______ cumeeric_precision numerie max len evaluation itttaatiooosttaagryy metric_ for hewr_beet__me__ model gradient. accumulstion steps
4
10
REALTABPORMER
'steps'
'loss"
1
remove unused columns logging steps steps
True
100
100
eval steps
100
lund best model at and
True
save total limit
optim
'adamww torch.
32

---

Published in Transactions on Machine Learning Research (02/2026)
Table 17: Hyperparameter specification for TabularARGN et
Tiwald et al. 2025), ClavaDDPM Pang
al. 2024 and RGCLD Hudovernik
(0224).
model
hyperparameter
value
Configuration presets
Accuracy
Max sample size
100%
Model size
Large
TARGN (API)
Batch size
Auto
Flxxible generation
Off
Value protection
Off
clusters
50
parent_scaale
classifier_soll
timestops
2000
hatch_
4096
layers_diffusion diffusion
[512, 1024, 1024, 1024, 1024, 512]
iterations diffusion
200000
CLAVADDPM
Ir diffusion
0.0006
weight decay diffusion
1e-05
scheduler diffusion
"cosine"
layers classifier iterations classifier
[128,
256, 512, 1024, 512, 256, 128
20000
Ir c_elessifier
0.0001
dim
125
GNN hidlen dim
128
GNN aggregation
GNN layers
# Tables
CNN Ir
0.008
GNN weight decay
le - 5
GNN epochs
250
VAE layers
VAE token dim
4
RCCLD
VAE hidden dim
128
VAE 5
0.7
VAE (8maz, Bmiu)
(0.01, 1e 5)
VAE
le - 3
VAE epochs
4000
Diff model dim
1024
Diff
0.001
Diff weight decay
Le - 6
Diff epochs
4000
33

---

Published in Transactions on Machine Learning Research (02/2026)
G Dataset Schema
users
affiliate channel
_ffllate_channel
categorical
affiliate _poovdde
categorical
age
numerical
country_destination
categorical
date_accom account acccoutttcereeeeeddddcccccccccdd
datetime
date first booking
datetime
first affiliate_ttacceee
categorical
CompesiionnpeenSiceeeess
numerical
first_brower browser
categorical
CompetifonOOennSineMMooo
numerical
first first_devins deleiceeeeeppeee
categorical
Competiton
numerical
Promo2
boolean
gender ld
categorical
Promo2Sincels Yeacee
numerical
id
Promo2Since
numerical
language
categorical
Store
signup_app
categorical
Store
categorical
signup_flow flow
categorical
Assortment
categorical
signup_ method
categorical
Promointerval
categorical
timestamp first active
datetime
historical
Date
datetime
sessions
Store
id
action
categorical
DayOfWeek
categorical
detail
categorical
Promo
categorical
action_type
categorical
StateHoliday
categorical
device seyie__yppe
categorical
Open
categorical
secs elapsed
numerical
SchoolHoliday categorical
user id
Customers
numerical
(4) Airbnh
(b) Resemann
Figure 9: Two table database diagrams.
stores
Size
numerical
Store
id
Type
categorical
features
CPI
numerical
Date
datetime
Fuel_ Price
numerical
depts
IsHoliday
boolean
Date
datetime
Dept
categorical
MarkDown1
numerical
paper
MarkDown2
numerical
class label categorical paper id
IsHoliday
boolean
MarkDown3
numerical
Store
id
MarkDown4
numerical
Weekly_Salee
numerical
MarkDown5
numerical
Store
id
cites
content
Temperature numerical
cited
Id ching paper Id
cited
categorical
cited _ite__pape__k
paper Id word cited peper_ ic
Unemployment numerical
cfting dfing_paper__t
(a) Walmart
(b) Cora.
Figure 10: Walmart and CORA database diagrams.
34

---

Published in Transactions on Machine Learning Research (02/2026)
CI
country
categorical
numerical
Ing
numerical
alt
numencal
races
constructors
constructoria
raceld
year numerica driverld round numerica dob nationality cotegorical
ic
drivers
rationalty
categorical
circuitid ic
datetime
datetime
datstime
results
restifti
id
raceld
driverld
id
constructor standings
constructori d
id
qualifying ving
standings
categorical
stavvrstanddnnssdd
quallivl
raceld
constuctorid
position
numerical
numerical
id
raceld
constuccodld
points
date
rumerical
detetime
id
CosttionCrde-
number
points
rumerical
numerical
rumericel
rumarical
raceld
constructor
position
driverld
number
numerical
numenical
wins
raceld
difverld
points
position
numerical
numerical
id
laps
millsacondss
rumarical
fastestLap
numarical
rark
date
datetme
Figure 11: F1 database diagram.
35

---

Published in Transactions on Machine Learning Research (02/2026)
district
A12
numerical
A10
numerical
A6
numerical
A14
numerical
A16
numerical
A4
numerical
A11
numerical
A13
numerical
A5
numerical
A3
categorical
A7
numerical
A15
numerical
A2
categorical
A9
numerical
district id
id
A8
numerical
client
account frequency date
birth_ date
datetime
categorical
client id
id
datetime
district_id
id
account id
id
gender
categorical
district id
trans
loan
balance amount numerical
numerical
Ioan id
id
order
disp
amount
numerical
amount bank to k symmol
numerical
date bank trans id operation k k_symbol
datetime
client id
id
date
datetime numerical categorical id
categorical categorical
categorical
disp
id
duration status
id
type
categorical
account id order id
id id
categorical
account id
id
account id
categorical
payments
numerical
type account id
categorical
id
card
card id
id
issued
datetime
disp_id id
type
categorical
Figure 12: Berka database diagram.
36

---

Published in Transactions on Machine Learning Research (02/2026)
molecule
activity
numerical
mweight
numerical
logp
numerical
molecule id
id
atom
group
atom id
id
type categorical group id
type molecule id
categorical
id
id
bond
gmember
atom_ie id
id
atom_id
id
atom id atom id2
id
atom id group id id
type
categorical
group_ic id
id
atom_id2
id
Figure 13: Biodegradability database diagram.
movies
users
actors
country year
categorical numerical
directors
age numerical
a_gender categorical id
d_quality numerical avg_rew revenue numerical directorid
occupation categorical
actorid
movieid id
userid id
a_quality
numerical
runningtime
numerical categorical
id
u_gender
categorical
Englissh
movies2actors
movies2director
u2base
movieid actorid
id
directorid
id categorical
movieid
id
cast_num num
categorical
genre
userid
id
actorid
id
movield directorid
id rating
numerical
movieid
id
movieid
id
userid movieid
id
Figure 14: IMDB MovieLens database diagram.
37


Reviews:
[{"review_id": "eTZUrwCkf4", "reviewer": "Reviewer_cyMs", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "### Summary\nThe authors present a benchmarking framework for evaluating methods that generate synthetic relational databases. The evaluation spans multiple scenarios, including single-table and multi-table fidelity analysis, utility evaluation, and privacy risk assessment. The authors compare several generative approaches and report error bars for all results. In addition, a qualitative analysis using selected examples (Figures 3 and 4) illustrates how the benchmark can help reveal characteristic failure modes of existing methods.\n\n### Strengths:\n* (S1 – relevance): Relational data synthesis is practically important, making the problem both relevant and timely for the community.\n* (S2 – clarity of positioning): The paper clearly motivates why relational data synthesis is challenging and why a dedicated benchmark is necessary.\n* (S3 – experiments, results, and metrics): The authors conduct a comprehensive experimental evaluation covering data fidelity, downstream utility, and privacy risk. Reporting error bars strengthens the significance of the results, and the chosen metrics appear reasonable and well motivated.\n\n### Weaknesses:\n* (W1 – narrative flow, clarity, and presentation):\nWhile the overall motivation becomes clear after reading the full manuscript, the narrative flow feels fragmented at several points. Restructuring and language polishing would improve readability. For example:\n    - Before reviewing generative methods, it would be helpful to first introduce the core evaluation criteria for synthetic data quality (e.g., fidelity, utility, and privacy). In particular, differential privacy (DP) is used later without being clearly introduced.\n     - In Section 2.2, the discussion of statistical, distance-based, and detection-based fidelity metrics is well structured, but the preceding paragraph on data granularity appears misplaced and interrupts the flow.\n    - Although the paper proposes a benchmark, it remains unclear which components are part of the final benchmarked framework and how they are intended to be used in practice (see W4).\n\n* (W2 – metric sensitivity / minor):\nThe benchmark relies in part on classifier-based fidelity metrics. While effective, such metrics can be sensitive to the choice of discriminator and may be susceptible to models exploiting weaknesses of the scorer rather than genuinely improving data quality. This is a minor concern and does not affect the overall contribution, but a brief discussion of this limitation would strengthen the paper.\n\n* (W3 – limited citation support for fidelity claims):\nIn Section 3.1, the authors argue that existing state-of-the-art fidelity metrics fail to adequately capture inter-table relationships. While this claim would benefit from stronger support through explicit references or a more detailed discussion of prior limitations.\n\n* (W4 – guidance for practitioners):\nAlthough the comparative evaluation is thorough, the manuscript provides limited guidance on how practitioners are expected to use the benchmark. For instance, it is unclear whether users interact with SyntheRela via a Python package, how generative models or results are submitted to the leaderboard, and what level of automation is provided. Clarifying these aspects would improve usability and adoption.", "claims_and_evidence_yn": "No", "claims_explanation": "Generally yes, only (W3) should be fixed.", "audience_yn": "Yes", "audience_explanation": "Data scarcity is a relevant problem; therefore, generative approaches for synthetic data and corresponding benchmarks are important. This also applies to relational data.", "requested_changes": "### Critical\n* (W3): The claim that existing fidelity metrics fail to adequately capture inter-table relationships should be supported with additional literature references and a more detailed discussion, as this claim directly motivates the introduction and use of the proposed C2ST metric.\n\n### Not critical but recommended\n* (W1): The manuscript would benefit from further polishing to improve narrative flow and readability, as some sections currently feel fragmented.\n* (W2): A short discussion of the potential risk that generative models may exploit weaknesses of classifier-based fidelity metrics (rather than improving true data quality) would strengthen the evaluation.\n* (W4): Providing clearer guidance on how the framework is intended to be used in practice (e.g., workflow, tooling, or interaction with the leaderboard) would improve usability for practitioners.", "broader_impact_concerns": "No concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "HmPzDCxvi5", "reviewer": "Reviewer_9amB", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces SyntheRela, a benchmark for evaluating synthetic relational database generators. It contributes: 1. C2ST-Agg, a new detection-based metric for multi-table fidelity. 2. RDL-Utility, a relational deep learning utility evaluation using GNNs. 3. A comprehensive empirical comparison across 6 models and 8 real-world relational datasets. Code and leaderboard are provided. The benchmark addresses a gap in evaluating full relational synthetic data, going beyond single-table methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper’s main claims are supported by clear experiments using six generative models and eight relational datasets. The proposed C2ST-Agg metric and RDL-Utility are validated through consistent empirical results, diagnostics and comparisons to existing metrics. The benchmark is well documented and limitations are acknowledged. The evidence is sufficient to support the contributions.", "audience_yn": "Yes", "audience_explanation": "Relational synthetic data is an active research area and the community lacks established evaluation tools. A benchmark that evaluates multi-table fidelity and relational utility would interest researchers working on synthetic data, relational learning and data-centric AI. The topic is relevant to a meaningful part of the TMLR audience.", "requested_changes": "These are not essential for acceptance but would strengthen the paper:\n1. Clarify how dataset subsampling may affect the generality of the results.\n2. Add a short note explaining when RDL-Utility is applicable since it requires temporal structure.\n3. Provide a short discussion on how differential privacy oriented methods could be incorporated in future versions of the benchmark.\n4. Give one simple example of relational errors that C2ST-Agg may not detect to help readers understand its scope.", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "NcaQHDb1bJ", "reviewer": "Reviewer_98gm", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a new benchmark suite for evaluating generative models for relational data. The authors propose two new metrics: classifier two-sample test with aggregation (C2ST-Agg) and relational deep learning utility (RDL utility). Along with these new metrics, the authors evaluate 6 methods on 8 datasets including various characteristics, including single and multi table generation. Thus, the authors benchmark the applicable models in a broad setting and various evaluation methods. \n\nWhile the authors suggest that their benchmark suite is well suited for other methods, it remains a bit vague why certain choices have been made (e.g., why the 8 chosen datasets).", "claims_and_evidence_yn": "Yes", "claims_explanation": "While I am not an expert on generative models of relational data, the chosen dimensions that are being evaluated appear reasonable to me and align well with how I intuitively would structure such a benchmark.", "audience_yn": "Yes", "audience_explanation": "The field of generating relational tables seems to be emerging and surely such works benefit from systematic evaluation procedures in literature.", "requested_changes": "The impact of such benchmarks is largely driven by the usability of the code. However, the readme is rather scarce on how to add a new model. The authors should spend more time on providing simple exaplanations etc. on what needs to be done.", "broader_impact_concerns": "No concerns", "additional_comments": "The statement is confusing:\n> We include 6 datasets that feature in related work (Airbnb, Rossmann, Walmart, Biodegradability, MovieLens)\n\nSometimes it is unclear why abbreviations are introduced. E.g.,  \"machine learning efficacy (ML-E)\"", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)