ClavaDDPM: Multi-relational Data Synthesis with Cluster-guided Diffusion Models
Authors: Wei Pang
Masoumeh Shafieinejad
Lucy Liu
Stephanie Hazlewood
Xi He
Venue: NeurIPS 2024 poster (2024)
Keywords: diffusion models
synthesis
tabular data
generative models
Primary area: diffusion_based_models

Abstract: Recent research in tabular data synthesis has focused on single tables, whereas real-world applications often involve complex data with tens or hundreds of interconnected tables. Previous approaches to synthesizing multi-relational (multi-table) data fall short in two key aspects: scalability for larger datasets and capturing long-range dependencies, such as correlations between attributes spread across different tables. Inspired by the success of diffusion models in tabular data modeling, we introduce 
 \textbf{C}luster \textbf{La}tent \textbf{Va}riable guided \textbf{D}enoising \textbf{D}iffusion \textbf{P}robabilistic \textbf{M}odels (ClavaDDPM). This novel approach leverages clustering labels as intermediaries to model relationships between tables, specifically focusing on foreign key constraints. ClavaDDPM leverages the robust generation capabilities of diffusion models while incorporating efficient algorithms to propagate the learned latent variables across tables. This enables ClavaDDPM to capture long-range dependencies effectively. 
 Extensive evaluations on multi-table datasets of varying sizes show that ClavaDDPM significantly outperforms existing methods for these long-range dependencies while remaining competitive on utility metrics for single-table data.

ClavaDDPM: Multi-relational Data Synthesis with Cluster-guided Diffusion Models
Wei Pang Masoumeh Shafieinejad', Lucy Liu', Stephanie Hazlewood', and Xi He 1,2+
University of Waterloo
Vector Institute
Royal Bank of Canada
wSpangluwater.oo. ca, masommendvectooristituu ai, lucy www.uubbbcccom stephanie hazlewood@rbc. com, xi. www.waaweerrr..oooooooo
Abstract
Recent research in tabular data synthesis has focused on single tables, whereas real- world applications often involve complex data with tens or hundreds intercon- nected tables. Previous approaches to synthesizing multi-relational (multi-table data fall short in two key aspects: scalability for larger datasets and capturing long-range dependencies, such as correlations between attributes spread across dif- ferent tables. Inspired by the success of diffusion models tabular data modeling, we introduce Cluster Latent Variable guided Denoising Diffusion Probabilistic Models (ClavaDDPM). This novel approach leverages clustering labels as interme- diaries model relationships between tables, specifically focusing foreign key constraints. ClavaDDPM leverages the robust generation capabilities of diffusion models while incorporating efficient algorithms to propagate the learned latent vari- ables across tables. This enables ClavaDDDMM to capture long-range dependencies effectively. Extensive evaluations multi-table datasets of varying sizes show that ClavaDDPM significantly outperforms existing methods for these long-range dependencies while remaining competitive utility metrics for single-table data.
Introduction
Motivation. Synthetic data has attracted significant interest for its ability to tackle key challenges in accessing high-quality training datasets. These challenges include: i) data scarcity [14 53]. ii) privacy 19), and iii) bias and fairness (46). The interest synthetic data has extended to various commercial settings as well, notably in healthcare (18) and finance (36] sectors. The synthesis of tabular data, among data modalities, a critical task with approximately 79% of data scientists working with it on daily basis [45). While the literature tabular data synthesis has predominantly focused on single table (relation) data, datasets in real-world scenarios often comprise multiple interconnected tables and raise new challenges to traditional single-table learning [38 3] [2] 22)
These challenges have even enforced a join-as-one approach [15] 17], where the multi relations are first joined as single table. However, with more than couple of relations (let alone tens hundreds of them as in the finance sector) this approach neither desirable nor feasible.
Challenges. Synthetic Data Vault 335 and PrivLava [ recent efforts to synthesize multi- relational data using hierarchical and marginal- approaches. These methods exhibit significant
*Corresponding author.
'In the context of databases, table is formally referred to as a relation. Throughout this work, use these terms interchangeably.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

limitations in processing speed and scalability, both with respect to the number of tables and the domain size of table attributes, and they often lack robustmess capturing intricate dependencies. Alternatively, diffusion models have emerged as powerful tools for data synthesis, demonstrating remarkable success in various domains [37]. These models are particularly noted for their strong capabilities in controlled generation. Despite their potential, the application of diffusion models to
tabular data synthesis has been limited to unconditional models [25 50 28 24], leaving a gap in effectively addressing the multi-table synthesis problem.
Solution. To address these challenges, we introduce ClavaDDPM (Cluster Latent Variable guided Denoising Diffusion Probabilistic Models). Our novel approach leverages the controlled generation capabilities of diffusion models by utilizing clustering labels as intermediaries model the relation- ships between tables, focusing on the foreign-key constraints between parent and child tables. This integration of classifier guidance within the diffusion framework allows ClavaDDPM to effectively capture complex multi-table dependencies, offering significant advancement over existing methods.
Contributions. In this work, we: 1) provide a complete formulation of the multi-relational mod- eling process, as well as the essential underlying assumptions being made, 2) propose an efficient framework to generate multi-relational data that preserves long-range dependencies between tables. 3) propose relationship-awe aware clustering as a proxy for modeling parent-child constraints, and apply the controlled generation capabilities of diffusion models tabular data synthesis, 4) apply an approxi- mate nearest neighbor search-based matching technique, as a universal solution to the multi-parent relational synthesis problem for a child table with multiple parents, 5) establish a comprehensive multi-relational benchmark, and propose long-range dependency as new metric to measure synthetic data quality specific to multi-table cases, and 6) show that ClavaDDPM significantly outperforms existing methods for these long-range dependency metrics while remaining competitive on utility metries for single-table data.
2 Related work
Single-table synthesis models. Bayesian network 48 is a traditional approach for synthetic data generation for tabular data. They represent the joint probability distribution for a set of variables with graphical models. CTGAN 47 is a tabular generator that considers each categorical value as a condition. CTAB-GAN
52 includes mixed data types of continuous and categorical variables. Several studies have explored how GAN-based models can contribute to fairness and bias removal 45). In privacy, GAN-based solutions boosted with differential privacy have not been as successful as their
competitors 34 [1]. Recent popular Diffusion Models,
[0]44 42,44 41], offer a different paradigm for generative modeling. TabDDPM (25) utilizes denoising diffusion models, treating numerical and categorical data with two disjoint diffusion processes. STaSy
24 uses score-based generative modeling in its training strategy. CoDi (8) processes continuous and discrete variables separately by two co-evolved diffusion models. Unlike the previous three which perform in data space. TabSyn [50] deploys a transformer- baseed variational autoencoder and applies latent diffusion models. Privacy and faimess research for diffusion models are currently limited to few studies in computer vision 26 16].
Multi-table synthesis models. There have been few proposals for synthetic data generation for multi-relation data. A study proposed this synthesis through graph variational autoencoders [31. the presented evaluation is nevertheless very limited. The Synthetic Data Vault (35] uses the Gaussian copula process to model the parent-child relationship. SDV iterates through each row in the table and performs conditional primary key lookup in the entire database using the ID of that row, making set of distributions and covariance matrices for each match. This inhibits an efficient application of SDV to the numerous tables case. PrivLava [5), synthesizes relational data with foreign keys under differential privacy. The key idea of Privl Lava is to model the data distribution using graphical models, with latent variables included capture the inter-relational correlations caused by foreign keys.
3 Background
Multi-relational databases. A multi-relational database R. consists of m tables (or relations) (R,,... Rm). Each table is a collection of rows, which are defined over a sequence of attributes.

---

Demoggaghc
Bxu
Idey
Deropophy
Accest
Dipoee D3
Ther sictinn
Order
Leen
Ceed Casl
Figure 1: Berka sample tables (left), and the foreign key constraint graph for Berka (right)
One of the attributes, let's consider the first attribute without loss of generality, is the primary key of table R, which serves as the unique identifier for each row in the table. No rows in the same table have repeated values for the primary key attribute. We use Berka database [4] as our running example Note the Account ID, the primary key for the Account table in Berka. this work, as in Figure I
Given a table Ry, we say relation Rg has a foreign key constraint with Rg, or R2 refers to Ry. if Rg has an attribute known as foreign key that refers to the primary key of Ry: for every row ra R. there exists a row rj E RJ such that ry 's primary key value equals to r 's foreign key value. For example, the Account ID of the Loan table refers the primary key of the Account table. If an account row is removed from the Account II) table, so would all the referring rows in the Loan table to this account, for foreign key constraint to hold. Note that the primary key of table can consist of multiple attributes. In this paper, we focus on the case of single attribute that is common in practice. Also note that all keys are considered row identifiers and are thus not treated or modeled alongside the actual table attributes in this work.
multi-relational database under foreign key constraints forms directed acyclic graph (DAG),
- (R,E)) (R + R2) i (l....,,,, m3 j, R2 refers to R2}
(1)
with the tables being the set nodes, and & being the set of edges. addition, for R& referring to Rj, we also call this parent-child relationship, where Rí is the parent and R, is the child. We use the maximum depth to denote the number of nodes on the longest path in G. Figure Tearre@shoop corresponding graph to Berka database and its maximum depth is 4.
shows the
Multi-relational synthesis problem. Given a multi-relationn database R = [RI..... Rm J. we would like to generate synthetic version R {RI..... .......... Rm ) that has the same structure and foreign-key constraints as R and preserves attribute correlations within R. including 1) the inter- column correlations within the same table; 2) the intra- mutra-group correlations within the same foreign key group; 3) the tater-table correlations. The first aspect has been well defined, measured, and tackled
in the literature of single-table synthesis 552 25 50) while the other two aspects raised due to foreign-key constraints between tables [5] For instance, in Berka database (Figure [ the foreign key constraint between the Loan table and the Account table via Account ID) adds an important -roru correlation for the combinations of loans associated with an account and many 1-hop inter-table correlations between columns in the Loan table and the columns in the Account table. Even for the Loan table and the Demographic table that are indirectly constrained by foreign keys, their columns are correlated as well, e.g., how is the average salary in a district related the status of loans, an example for 2-hop inter-table correlation.
massiffierguided DDPM. DDPM 200 uses two Markov chains, forward chain that perturbs data noise through series of Gaussian transitions, and reverse chain that converts noise back to data with the same number of steps of Gaussian transitions (Equation 2p.
(xx (x-1)) == N (zx: /1 11.1-11811
(2)
Pe (I,-1 xx)) := N (æ,-1: M4 (æ,, f), En ())
Prior work 40 shows that given label y, the conditional reverse process has the form
P (F: [ v) X po (#t [++1) Ps (y (x)).
(3)
3

---

By approximating log Pa: (y (x)) using Taylor expansion around 22 = μ,, the conditional reverse
process (Equation 3) can be approximated with a perturbed Gaussian transition [T]]
log (po. (ata Ft+1, y)) log (p (z)) C, N (μ+ Eg, Σ) where C' is a constant and Vm, log (Po (y (x))) læ:=ı computed from the classifier Pa-
4 ClavaDDPM
Here, we elaborate on the training and synthesis process of ClavaDDPM, and each design's 's rationale.
4.1 Modeling generative process for two-table relational databases
Notations. Consider database of two tables R { R2), e.g. (Loan, Account) in Berka, where the child table R₁ refers to parent table R2. To model the entire database, we first use X and Y as the variables for the child table R1 and parent table Rg, respectively (dropping their primary key attributes and indexing their respective row variables starting from one). In this section, we use boldface represent random variables. e.g. Y ~ Y, where Y is the data of R2, and Y is the random variable y being sampled from. In addition, we use subscript to represent the parent row some data random variable refers to. e-g. T., represents the child random variable who refers parent y,- Refer to Appendix A for complete list of notations used and the corresponding design choices.
Assumptions. 1) The parent table has no constraints itself. Hence, can follow previous work on single-table synthesis [13] 25 47 50 [2] to make an i.i.d assumption on the rows in the parent table. The parent table Y can be modeled the index or the primary key value of the jth row, and each row follows a distribution p(y).
list of i.idd row variables { |j L..... [R2]]. where j is
2) The i.dd assumption does not apply to the child table rows (æ, 's) as they are constrained by their respective parent rows. Consider two loans associated with the same account id; if one's status is in debt ("C"), the other one is likely so too. To capture this dependency, we make a Bayesian modeling assumption that, although child rows associated with the same parent are not independent, they are conditionally independent of child rows associated with other parent rows, given their respective parent. For example, consider account table (parent) and loan table (child). Loans related to the same account (i.e., the same parent) are not independent due to shared account-specific factors. However, loans from different accounts be considered conditionally independent when accounting for their respective account-level information. Hence, we model X by {g, |j I,..., [Rll). where each group 9, { L,...: | represents set of child table rows referring to the parent row yy-
3) Without violating the assumptions made above, we further make an i.i.d assumption on (g..y,). which leads to an approximated distribution for the parent-child tables:
|Ra
P(X - = Y)
II j=1
P (g, = -9. = 99) or ((XY)) = Il p(g). Uj)
(5)
where X = 0 = correlations (the correlation between tuples from different tables) and the intra-group correlations.
t ........... 10.13. This model allows us to capture the inter-table
and 9j
Modeling. Despite the simplified formulation with several aforementioned assumptions, learning the distribution p (gy, y1) non-trivial. In particular, (g), y;) cannot be flattened into a matrix form for learning since the set structured attributes in 9,, e.g., the size of group variable 9, is not fixed.
naive solution is model a conditional distribution of the group given the parent row
p (gg yU) p (g) v,) (v,)
(6)
Direct modeling of Equation (6) still has the same issue before for the foreign key group 9y, which can take an arbitrary number of child rows. In particular, when modeling 9, f (æ;) for some function f. there is no trivial structured support for 9, if we model for 9j using only the attributes or features of the child rows. Furthermore, the conditioning space of the parent row y can be very

---

large (e.g., Account table has domain size of more than 11,000), which can lead to poorly learned conditional distribution if we treat y as labels in the classifier-uuided DDPM. The original space of y is high-dimensional and noisy and does not guarantee any spatial proximity smoothness. In the context of deep modeling, this drastically worsens the quality of conditional sampling.
To address these two challenges, we introduce latent random variables c such that gj is independent from ,, conditioned on
وووس le
With this assumption, we get an indirect modeling of inter-table correlations through
plax 11) Σ plgss, = Σ Epssslep(usooppte = Ep pup(v)))
(8)
Compared to Equation 16 when c is selected to be lying low-dimensional, compact manifold, the latent conditional distribution p (gi c) will easier to model than p (gj 1z)).
For each foreign key group 9,- we model its size explicitly with a variable 8z- By making an assumption that 8, is conditionally independent from its child row variables (ar).... x3 given the latent random variable c, we essentially defined a generative process for gy: first sample its size 8j 8,. then sample 9, child row variables. In addition, we make an i.idd assumption the child row variables given the latent variable. Hence, we have
p(aslc) = p(s,le) II p(ay(e)
Putting all together, we have the final formulation of the generative process for a two-table case:
RRal
p (X) II
P
vg)
i.d assumption on (g), y₁) Equation (5)
j=1
(10)
|Rgl
R2 = III c) (e) II (
Σ p(a 1=1
II el
f=1
Unlike our naive method that uses a direct modeling of p (rj 1,;). we model p (zy (g), which greatly reduced the condition space, thus better capturing the inter-table correlation between X and Y. On the other hand, the intra-group correlations are intrinsically addressed, because our modeling of (g): y) and the corresponding dependency assumptions enforce that child rows drawn from the same distribution if and only if they belong to the same foreign key group.
Based on Equation (10). introduce generative process for two-table case.
Phase Latent learning and table augmentation: (1) Learn latent variable e on the joint space (X;Y) Y), such that each parent row yj corresponds to learned latent variable cj. (2) Augment the parent table into Ty- (Y; C), where C' corresponds the latent variable values ey for each row yj-
Phase II: Training: (3) Train diffusion model pa (y, c) on augmented table Ty and child diffusion model Pa (x). (4) Given learned latents and child table, train classifier Pti (c [ (5) Estimate the foreign key group size distributions conditioned on latent variables p (8 |c).
Phase II:: Synthesis: (6) Synthesize the augmented parent table Ty = (Ý; C) ~ po (.::). (7) For each synthesized latent variable ē C, sample group size 8j (( (2))- (8) Given 8j, sample each child row within the foreign key group æt, Po.e (,2z), where Pa.v performs classifier guided sampling by perturbing Pa with the gradient of pv- We denote steps (7) and (8) by X ~ p(-CC).
4.2 Extension more parent-child constraints
We learn the latent variables between parent and a child pair in a bottom-up fashion (starting from the leaf nodes in 9) and pass all the latent variable values the parent table for the next set of latent variables at higher levels. Given parent-child pair XY,X), the child table X also
5

---

has k leaf node children, Zi...., Ze. Let 2, represent the latent variables learned on the joint space (X; 2.). The augmented table for X is formed by appending all its latent variable values. i.e. Tx = Zx. Cx Then, the latent variable cy,x is learned on the joint space of (x: instead of (V; X). Therefore, our latent learning process follows a bottom-up topological order, ensuring each child table is already augmented by the time we learn the latent variable to augment its parent.
The training phase and the synthesis phase are similar to the two-table case, by handling the parent- child tables top-down topological order using the augmented tables. We detail the end-to-ent algorithms for the complex data in Appendix B However, we would like to highlight special case when a table X has multiple parents .i.....y, Yg. During synthesis, we will have i synthetic latent variable CI..... OR corresponding to the k parents, and thus copies of synthetic child tables X1 ~ p(- C ................................ Xx+ ~ p(- |C²).. Unifying these diverged synthetic tables presents challenge and we present universal solution in Section 4.3.3]
Extending the model to include more tables allows for capturing longer-range dependencies. beyond just those between adjacent tables. For example, as shown in Figure [ the dependency between the Demographic table and the Credit Card table can also be captured and quantified. Further details provided in Section 5
4.3
Design choices for CPavaDDPM
We detail how design decisions for ClavaDDPM meet our goals and align with our assumptions.
4.3.1
Relationship-aware elustering
Given the conditional independence between the parent row and its foreign key group (Equation 7 is important to model the latent variable e such that it can effectively capture the inter-table correlation within the same foreign key group. In ClavaDDPM, we learn e using Gaussian Mixture Models (GMM) in the weighted joint space of X and Y, denoted as H = (x:A)) where À is weight scalar controlling the importance of child and parent tables when being clustered. Concretely, we consider l clusters, and model the distribution of h = (z; Xy) with Gaussian distributed around
its corresponding centroid c, i.e., P (h) = Σt P (c) P (h cc) = Eč=1 TNN (h: Mee Ex)
Note that diagonal GMMs are universal approximators, given enough mixtures of Gaussian distri- butions [49] Therefore, we can further enforce diagonal covariance, i.e., Σ,- = diag (.... oi.... which, being properly optimized, immediately satisfies our assumptions that the foreign key groups are conditionally independent of their parent rows given c. In addition, the family of Gaussian Process Latent Variable Models (GPLVM) (30 27 33 has been used as an embedding technique to find low-dimensional manifolds that map to a noisy, nigh-dimensional space. This satisfies need to learn a stochastic map between the noisy parent space and a condensed latent space. Thus, we can achieve a better trade-off by sacrificing some information fidelity during this quantization process while making the conditional space better shaped.
However, such clustering in the joint space (X; XY) could potentially lead to inconsistency when we create the augmented table Ty = (Y: C). Though we add a weight À to the parent rows such that child rows with the same parent rows are likely to be assigned to the same cluster, there is still some chance that they end with different clusters.
particular, for each parent row yj E Y, its child rows are assigned to different clusters. In ClavaDDPM, we impose majority voting step to find the most popular cluster label in each foreign key group and assign it to the parent row yz- In practice, the voting agree rates tend to be high, and this can be further enforced by assigning higher weight to the parent table (increasing X) during GMM clustering. We evaluate the choice of X and voting agree rates in our ablation study in Section 5.3
While alternative latent learning algorithms could potentially be applied, such as TabSyn [50] that demonstrated the utility of latent encoding of tabular data with VAE, this work focuses on demon- strating the effectiveness of a simple diagonal Gaussian Mixture Model (GMM) for CIavaDDPM. Our experiments (detailed in Section 5) reveal that ClavaDDPM with a diagonal GMM achieves Mate-of-the-ar results while maintaining low computational overhead. We leave the exploration of more complex latent learning techniques for future work.

---

4.3.2 Learning with DDPM
Gaussian diffusion backhone. We consider one of the state-of-the-art diffusion models for single tabular data, TabDDPM (25), as the backbone model. TabDDPM models numerical data with Gaussian diffusion (DDPM (0)), and models categorical data with multinomial diffusion ((I) with one-hot encoding, and carries out disjoint diffusion processes. However, the modeling of multinomial diffusion suffers significant performance overheads, and poses challenges guided sampling. Instead, ClavaDDPM uses single Gaussian diffusion backbone to model both numerical and categorical data in unified space, where categorical data mapped the numerical space through label-encoding. To be specific, for categorical feature with m distinct values C' = fci..... Cm J. a label encoding E: C' - {O..... ......... m 1) maps each unique category C; to an assigned unique integer value. For table row represent a categorical feature, we obtain the unified feature by Xuni = [uum .... (Nents) ;.... Based on this encoding, we learn po (y, c) on the augmented parent table Ty (Y: through training a Gaussian diffusion model on the unified feature space (Yune; E (C)).
nuuum: Feate: :. where Znum represents all the numerical features and Zeat,
Classifier guided synthesis. As defined in Equation (O). model (z) |c,) by leveraging classifier-guidde guided sampling of diffusion models, following [IO]. In practice, with the sheer power of diffusion models, we jointly model p (x c) for the entire table without distinguishing j. First, we train a Gaussian diffusion model Pa on child table row æ, with its reverse process modeled with at ~ N (x M )) Then, train classifier that classifies cluster labels based æ. The con- ditional reverse process can be approximated by æe C ~ N (x++1i M + + 9 Ed(41). where 9 V log (po (c ax++)) and v) is a scale parameter controlling the strength of conditioning. One can regard
hyper parameter measuring the trade-off between single-table
generation quality and tabee correlations, to be demonstrated by our ablation study in Section 5.3
4.3.3 Multi-parent dilemma: matching
Consider the case where some child table X has two parent tables Y1, Y2. Our parent-child syn- thesis modeling paradigm would lead two divergent synthetic child tables X, X YY,, and X2 X Y2 Each synthetic table encodes its own parent-child relationship, i.e. the foreign keys. Combining Xı and X2 so that the synthetic child table contains foreign keys from both parents p and is non con-trivial, and we call it multi-parent dilemma. One possible approach is to explicitly constrain the model sample space of X Y2 to be the synthetic data X1, as used in PrivLava (5). However, this approach not applicable to diffusion models that sample from continuous space.
We provide a universal solution for all generative models. Consider some real data point x with two parent rows yf and y². Ideally, some synthetic data point ă following the same distribution as real data point x should be sampled from æ l y This can be approximated by finding the intersection of two conditional distributions x y and x y22 Specifically, we estimate ž by finding two synthetic data points ū1 E X2 and 72 E X2, such that Ø1 ~ x y and Z2 ~ x 1 and the two points are close enough. We reason follows: although 21 sampled from V/ long as it is close enough to some other synthetic data point Zo sampled from x y2 then 2i will also be within in the high density region of the distribution x: y2, indicating a high probability that ø follows æ y V2- Symmetrically, the same reasoning also holds for Øz2-
Therefore, we can estimate the true sample data point by ã f (ã1, 22) if 5; is close to 22, where f can simply be an interpolation between two data points in practice. We call this a matching process between two divergent synthetic tables X, and X2. and this can be done efficiently using approximate nearest neighbor search. Although we call this a "matching", it does not require finding a one-to-one mapping. Note that this estimate can be further improved by resampling X and X₂ and estimate X with more data points rather than just a pair, and the trade-off is larger computational overhead, and we leave this for future research. Empirically, sampling X, and X2 only once is already strong, and an ablation study on the effectiveness of parent matching is in Section 5.3
7

---

5 Evaluation
We evaluate ClavaDDPM's 's performance in multi-relational data synthesis, using both single-table and multi-tables utility metrics (including the new long-range dependency). We present an end-to-end comparison of CIavaDDPM the SOTA baselines, followed by an ablation study for ClavaDDPM.
5.1 Experimental setup
Real-world datasets. We experiment with five real-world multi-relational datasets including Califor- nia [6]. Instacart 05 [23]. Berka [4, Movie Lens 39 and CCS [3]. These datasets vary in the number of tables, the maximum depth, the number of constraints, and complexity. Among all, Berka, Movie Lens, and CCS exhibits complex multi-parent and multi-children structures. We use Berka in our work for ablation study and model anatomy. Details can be found in Appendix C.1
Baselines. We adopt two multi-relational synthesis models in literature as our baselines: PrivLava 5 as a representative of state-of-the-ard marginal-based methods, and SDV [35] as a statistical method specially designed for multi-relational synthesis. We also introduce two multi-relational synthesis pipelines, SingleT(ST) and Denorm(D), as our additional baselines. SingleT learns and generates each table individually, but also assigns foreign keys to each synthetic child table accordingly to the real foreign key group size distribution such that the group size information is preserved. Denorm follows the baseline idea that joins table first, but it is hard to join all tables into a single table. Hence, Denorm first applies angle-table backbone model to generate the joined table between every parent-child table pair and then split it. For these two pipelines, we use CTGAN +7 and TabDDPM [25] as the single-table backbone models, representing the SOTA tabular synthesis algorithms with GAN-based models and diffusion-based models. The details can be found in Appendix C.2
Evaluation metries. We evaluate the quality of the synthetic data using: 1) cardinality to measure the foreign key group size distribution for the intra-group correlations: 2) column-wise density estimation (I-way) to estimate the density of every single column for all tables; 3) pair-wise column correlation (k-hop) for the correlations of columns from tables at distance k. c.g., O-hop refers to columns within the same table and 1-hop refers to column and another column from its parent or child table; 4) average 2-way, which computes the average of all k-hop column- -airr correlations, taking into consideration of both short-range (k- 0) and longer-range (k > 0) dependencies. For each measure, we report the complement of Kolmogorov-Smmnng Smirnov (KS) statistic and total variation (TV) distance between the real data and the synthetic data, ranging from 0 (the worst utility) to I (the best utility). The reported results are averaged over 3 randomly sampled synthetic data.
We also consider higher-order single-table evaluation metrics for some representative tables as prior work (50). We include their details and experiemntal results in Appendix Duee to space constraints.
All experiments are conducted with an NVIDIA A6000 GPU and 32 CPU cores, with a time limit of 7 days. If an algorithm fails to complete within the time limit, we report TLE (time limit exceeded). Implementation details and hyperparameter specifics are in Appendix C33
5.2 End-to-end evaluation
We conducted multi-table synthesis experiments five multi-table datasets and report the averaged utility with standard deviation for all algorithms in Table First, the evaluation shows that ClavaD- DPM has overall advantage against all the baseline models in terms of correlation modeling, and is surpassing the baselines by larger margins for longer-rang raage dependencies. e.g. in Instacart 05, our model outperforms the best baseline by 58.29% on 2-hop correlations, and in Berka, model exceeds the best baseline by 20.24% on 3-hop correlations. For single-colum densities and cardinality distributions, ClavaDDPM exhibits a competitive result compared the state-of- -frt baseline models. We also evaluate ClavaDDPM against baselines on high-order single-table metrics (Appendix [.33). which shows that our model has advantages in preserving data fidelity, generating diverse data, and achieving high machine learning efficacy.
It is worth noting that ClavaDDPM, despite its complexity and capability, is more efficient and robust than some simpler baselines. PrivLava demonstrates strong performance on the California dataset
'The complement to KS/TV distance between two distributions / and Q is 1.0 - DKSeTV (PI Q). We use KS for numerical values and TV for categorical values.
8

---

Lid so end
PrivLava
SDV
ST
TANDPPM
ST ClavaDDMM
DCTGGN
TUNDDPM
D ClvaDDPM
ClavaDDMM
California
CARIIALITY
71.45
98.19
L-WAY
72.52 =800
#007
50:22 =.00
87.07
2:033
79.27 0:00s
91.17 2:000
97.65 2:105
1-10P
97:46
64.80
+8/0
$4.82 +0000
78.44 4:004
92.96 0005
89.72 0.300
84.43 +1.50
57.24 1:0.0
95.10 +033
AVO 2-WAY
97.97 20.20
52.56
=0000
56.25
=0.0
78.85 5000
95.83
=0007
84.85
20.36
57.78
90.71 2:04
98.41 21.30
Instacart 05
CARDINALITY
95:78 1.05
94.75 1.111
93.81 1030
94.98 1.084
96.30 1.77
L-WAY
79.85 2:0.0
89.90 =: 000
60.07 =0.77
71.53 5:0.0
80.84 00229
1-HOP
DNC
DNC
78.27
02.488 2:0.0
2:0.25
TLE
08.393
19..0 = 8000
Dx17
00,26 20.38
84.85
11.44
TLE
02.58 2:0.0
58.74 2:1.
98.62 11154
76.02 11139
2-100P
24.82 1.502
16.22
13341
0.00 1000
0.00
36,
3.35
AVG 2-WAY
60.05
66.06 #:307
56,10
58.52
76. +.078
Berka
CARMINALITY
96608
96.06
L-WAY
76.41
D-HOP
74.24
2:0.32
72,80
0:23
21.34
72.12 2:0.7
91.49 20.52
1-10P
DNC
DNC
1.0.44
64.01 1:3:35
81.77 1.19
1.555
1:2:3
55-77
1:300
86.86
2-HOP
75.83
=1.07
50.88
+.331
78.00 +0.33
72.40
20.45
63.04
1155
57.68
1.067
80,
2:2.7
3-HGP
72.58
2:0.80
65.29
1:305
75.56 0.34
71.74 20.00
02.67
2:2.3
50.09
1:148
87.27 1:1.
AVO 2-WAY
73.22
2:045
61.74
3:147
82.33 = 01-00
71.94 24.57
00.29
1:30
00.53
2.44
89.21
1:155
Movie Lems
CARDINALITY
98.91 2:000
98. 18.99 0.0.
98.71 70 10.40
98.87 2:0.00
98.07 20.35
L-WAY
95.58 =:0.0
99.19 5.000
68 68.35 20.56
78.00 #:0.17
90.34 =0.00
0-100
LNC
DNC
72.50 2:0.95
TLE
08.76 butt
31, 11:32
TLE
07:38 2:0.0
98. 1115
I-HOP
74.86
02:72 +0009
58.00
77:45
96. +0.3
AVG
74.30
94,87
70007
97.11
CCS
CARTINALITY
74.36
L-WAY
500
#2300
94.84
3,30
57.002
2:0.8
88.30
57.15 1:136
2:034
98.
I-HOP
21:74
+9/97
69.84
+:330
47.11
+:0.0
51.62 0:223
36 66.80 +0000
61. 61.53 +1000
57.77
+:000
83
AVO 2 WAY
41.88
6.73
59.98
#1-72
58.29
=0.00
64.53
63.64
20.77
68.51 11:11
63.64
2:000
87.33 #8.1
Table 1: End-to-end results. DNC denotes Did Not Converge, and TLE denotes Time Limit Exceeded.
ST stands for Single-T and D stands for Denorm. Statistical metrics described in Section 5.1 are reported.
Default
Verying&
Varying A
Verying
Derka
R 20, 155.1
k
== 1000
A
A= 10
A= 100
72
11 n=
No Matching
CARDINALITY
96:92 LuT:
97.05
1.:00
96.12
-035
97.21
1:0.0
97:22
1::31
97.17
L0.17
96.20
10:31
96.35
1:3.9
97.76 1::35
1-RAYY
04.20 =0.4
94.04
=0.00
50.13
=0
94.90
=0.5
01.04
05.82
=0.4
01.67
20.39
94.14
=3.40
91.21 =022
D-ROP
91 10
=0:3
81.96
=200
50.07
=0:57
88 18.00
-111
80-91
+0:00
90.54
-1.5
85.35
1:30
90:00
=030
88.75
+142
I-St0P
86 86.56
+2:11
77:51
+..55
51.97
-1:00
81.72
+511
61.02
1:65
$4.15
-156
81:63
1.33
85.10
+1:0
81.97 +.:0)
2 Boe
89.26 =2.27
80.78
=0.5
$8.18
110
81.24
443
57.79
2:34
55.54
=21
84.42
20:4
57.64
2:14
82.41
=1:00
3 BOR
87:27 1192
75.53
=:266
$6.10
11
75.15
2706
85 20
22.00
79.19
80.08
+3.1
52.58
2:300
74.78
=200
ANG 2-WAY
60.21 1156
81.64
1:00
57.17
00:0
64.01 5:356
57.52
113
56.36
=21
84:00
00.26
81.57
20350
63.59 =15
AVG AGREZ SATE
81.12
20:
100.00
21090
65.97
20.77
78.85 20.73
50.87
82.44
2037
81.32
20.30
51,37
2:10
80.8 21135
Table 2: Ablation studies number of clusters k, parent scale X, and classifier gradient scale 17. Note
that v) and matching have no effect on agree rates. Statistical metrics described in Section 5.1 are reported.
(the simplest data), but fails to converge on all the other datasets. SDV also tends to fail on complex datasets, and is limited to datasets with at most 5 tables and maximum depth of 2 [9. Although TabDDPM shares similar model backbone with ClavaDDPM, its synthesis fails to complete within 7 days on multiple datasets, while ClavaDDPM completes all experiments within 2 days.
5.3 Ablation study
Gaussian diffusion backbone. To decouple the effect of our Gaussian diffusion only backbone with the latent conditioning training paradigm, we also included two models in Table ST-ClavaDDPM and D-ClavaDDPM, which use the Gaussian diffusion model in ClavaDDPM as backbone model, but are trained and synthesized following Single-T and Denorm. Compared to other baselines, ST-ClavaDDPM exhibits superiority in modeling both single column densities and column-pare correlations. ST-ClavaDDPM significantly outperforms its sibling ST-TabDDPM, which proves the effectiveness of using Guassian diffusion for tabular data synthesis solely. On the other hand, ST-ClavaDDPM falls short on longer-range correlations when compared to the full ClavaDDPM model. This observation provides solid evidence to the efficacy of our multi-table training paradigm.
Besides the study of the single-table backbone models, we perform comprehensive ablation study using Berka (for has the most complex multi-table structure) on each component of ClavaDDPM and provide empirical tuning suggestions. The full results in Table 2

---

Number of clusters k. We study the necessity of using latent cluster conditioning: (i) no conditioning with & = 1; (ii) many clusters with h 1000 to approximate a direct conditioning on parent rows rather than latent variables. When k- = 1, the quality of long-range correlation degrades drastically. When k- = 1000, we still get reasonably strong performance, which showcases ClavaDDPM'ss robustness. Compared to the default setting (k: = 20). the metrics are lower in all of cardinality distribution, single column densities, and column correlations - proper latent variable learning leads to better results than direct conditioning on parent rows. We also report new metric avg agree-rate, the average of all per-table agree rates for the labels within each foreign key group (Section 1..11 This measure highly depends on k but higher rate does not always imply better performance (e.g.. k=1 achieves perfect rates). We provide more insights on how it varies with the next parameter. We also conducted finer-grained experiments examine the effect of & on model performance, as shown in Appendix D.2
Parent scale A. Varying the parent scale parameter À changes the agree-rates as shown in Table 2 but the downstream model performance does not vary too much. This result indicates that the relation-aware clustering process is robust against such factors, and the GMM model is capable of capturing nuances in data patterns. The detailed discussion is in Appendix D.I
Classifier gradient scale This parameter controls the magnitude of classifier gradients when performing guided sampling, and thus the trade-off between the sample quality and conditional sampling accuracy. shole that, when O. which essentially disables classifier conditioning, the single column densities (1-way) are slightly higher than the default setting. However, it falls short in capturing long-range correlations. When 1 = 2, the conditioning is emphasized with higher weight, which significantly improves the modeling of multi-hop correlations compared to rj = 0 case.
Comparing with no matching for multi-parent dilemma. Berka (Figure I suffers from the multi- parent dilemma, where the Disposition table has two parent tables, Account and Client. Our abalation study switch the table matching technique to a naive merging of two synthetic table (Appendix C.2) The experiment result show that even if trained with the same hyper parameters and model structures, ClavaDDPM with matching is significantly stronger than the no- matching setup in terms of long-range correlations, with 3- h-h correlations 16. 16.70% higher than no- no-matching.
Conclusion
We proposed ClavaDDPM as solution to the intricate problem of synthetic data generation for multi-relational data. CIavaDDPM utilizes clustering on a child table to learn the latent variable that connects the table to its parents, then feeding them to the diffusion models to synthesis the tables. We presented ClavaDDPM's seamless extension to multiple parents and children cases, and established comprehensive multi-relational benchmark for a through evaluation introducing a new holistic multi-table metric long-range dependency. We demonstrated ClavaDDPM not only competes closely with the existing work on single-table synthesis metrics, but also it outperforms them in ranged (inter-table) dependencies. We deliberately selected the more complex public databases to exhibit ClavaDDPM's scalability, and introduce it as a confident candidate for broader impact in industry.
We focused on foreign key constraints in this work, and made the assumption that child rows are conditionally independent given corresponding parent rows. This brings three natural follow-up research directions: i) extension to the scenarios where this prior information is not available and these relationships need to be discovered f2rst[29]. ii) further relaxing the assumptions, and iii) inspecting multi-relational data synthesis with other integrity constraints (c.g. denial constraints [5). general assertions for business rules). Furthermore, evaluated ClavaDDPM's privacy with the common (in tabular data literature) DCR metric. Nonetheless, we think it is worthwhile to: i) evaluate the resiliency of ClavaDDPM against stronger privacy attacks 43) and ii) investigate the efficacy of boosting ClavaDDPM with privacy guarantees such as differential privacy. Similarly, the impacts of our design faimess and bias removal, as another motivating pillar in synthetic data generation, is well worth exploring as future work. We believe the thorough multi-relational modeling formulation we presented in this work, can serve as a strong foundation build private and fair solutions upon.
10

---

Acknowledgments
This work was supported by NSERC through Discovery Grant, alliance grant, the Canada CIFAR Al Chairs program. Resources used in preparing this research were provided, in part, by the Province of Ontario, the Government of Canada through CIFAR, and companies sponsoring the Vector Institute. We thank the reviewers and program chairs for their detailed comments, which greatly improved our paper.
References
[1] A. Alaa, B. Van Breugel, E. S. Saveliev, and M. van der Schaar. How faithful is your synthetic data? sample-level metrics for evaluating and auditing generative models. In International Conference on Machine Learning, pages 290-306, PMLR, 2022.
[2] S. A. Assefa, D. Dervovic, M. Mahfouz, R. H. Tillman, P. Reddy, and M. Veloso. Generating synthetic data in finance: opportunities, challenges and pitfalls. In Proceedings of the First ACM International Conference on Al in Finance, pages 1-8, 2020.
[3] A. Atserias, M. Grohe, and D. Marx. Size bounds and query plans for relational joins. SIAM Journal Computing, 42(4):1737-1767, 2013.
[4] P. Berka et al. Guide to the financial data set. PKDD2000 discovery challenge, 2000.
[5] K. Cai, X. Xiao, and G. Cormode. Privlava: synthesizing relational data with foreign keys under differential privacy. Proceedings of the ACM Management of Data, 1(2):1-25, 2023.
[6] M. Center. Integrated public use microdata series, international: Version 7.3 [data set]. min- neapolis, mn: Ipums, 2020.
[7] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321-357, 2002.
[8] Chen and C. Guestrin. Xgboost: scalable tree boosting system. Proceedings of the 22nd acm sigkdd international conference knowledge discovery and data mining, pages 785-794, 2016.
[9] DataCebo. Hmasynthesizer- synthetic data vault, 2023. Accessed on: May 20, 2024.
[10] Dhariwal and A. Nichol. Diffusion models beat gans image synthesis. Advances in neural information processing systems, 14:8780-8794, 2021.
[11] T. Dockhorn, T. Cao, A. Vahdat, and K. Kreis, Differentially Private Diffusson Models. Transactions Machine Learning Research, 2023.
[12] W. Dong and K. Yi. Residual sensitivity for differentially private multi-way joins. pages 432 444, 06 2021.
[13] C. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating noise to sensitivity in private data analysis. In Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4-7, 2006. Proceedings 3, pages 265 -84. Springer, 2006.
[14] J. Fonseca and F. Bacao. Tabular and latent space synthetic data generation: a literature review. Journal of Big Data, 10(1):115, 2023.
[15] C. Ge, S. Mohapatra, X. He, and F. Ilyas. Kamino: constraint-awart differentially private data synthesis. Proc. VLDB Endow., (886)11886-1899.)) 2021.
[16] S. Ghalebikesabi, L. Berrada, S. Gowal, I. Ktena, R. Stanforth, J. Hayes, S. De, S. L. Smith, O. Wiles, and B. Balle. Differentially private diffusion models generate useful synthetic images. ArXiv, abs/2302. 13861, 2023.
[17] B. Ghazi, Hu, R. Kumar, and P. Manurangsi. Differentially private data release over multiple tables. In Proceedings of the 42nd ACM CTGMOD-SIGACT-S/CE Symposium Principles of Database Systems, PODS '23, page 207-219, New York, NY, USA, 2023, Association for Computing Machinery.

---

[18] A. Gonzales, G. Guruswamy, and S. R. Smith. Synthetic data in health care: A narrative review. PLOS Digital Health, 2(1):1-16, 01 2023.
[19] M. Hernandez, G. Epelde, A. Alberdi, R. Cilla, and D. Rankin. Synthetic data generation for tabular health records: A systematic review. Neurocomputing. 493:28 45, 2022.
[20] J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840 18501,66881661 2020.
[21] E. Hoogeboom, D. Nielsen, P. Jaini, P. Forré, and M. Welling, Argmax flows and multinomial diffusion: Learning categorical distributions. Advances in Neural Information Processing Systems, 34:124454111 12465, 2021.
[22] X. Hu, S. Sintos, J. Gao, P. K. Agarwal, and J. Yang. Computing complex temporal join queries efficiently. Proceedings of the 2022 International Conference on Management of Dula, SIGMOD '22, page 2076-2090, New York, NY, USA, 2022. Association for Computing Machinery.
[23] jeremy stanley, M. Risdal, sharathrao, and W. Cukierski. Instacart market basket analysis, 2017.
[24] Kim, C. Lee, and N. Park. Stasy: Score-based tabular data synthesis. arXiv preprint arXiv:2210.040168 2022.
[25] A. Kotelnikov, D. Baranchuk, I. Rubachev, and A. Babenko. Tabddpm: Modelling tabular data with diffusion models. In International Conference on Machine Learning, pages 17569-17579. PMLR, 2023.
[26] I. Ktena, O. Wiles, I. Albuquerque, S.-A. Rebuffi, R. Tanno, A. G. Roy, S. Azizi, D. Belgrave, P. Kohli, T. Cemgil, A. Karthikesalingam, and S. Gowal. Generative models improve fairness of medical classifiers under distribution shifts. Nature Medicine, 30(4): 166-1173, apr 2024.
[27] N. Lawrence and A. Hyvärinen. Probabilistic non-linear principal component analysis with gaussian process latent variable models. Journal of machine learning research, 6(11), 2005.
[28] C. Lee, J. Kim, and N. Park. Codi: co-evolving contrastive diffusion models for mixed-type tabular synthesis. ICML'23. JMLR.org, 2023.
[29] Li, B. Wu, K. Yi, and Z. Zhao. Wander join: Online aggregation via random walks. SIGMOD 16, page 615-629, New York, NY, USA, 2016. Association for Computing Machinery.
[30] Li and S. Chen. A review on gaussian process latent variable models. CAAI Transactions on Intelligence Technology, 1(4):366-376, 2016.
[31] C. A. Mami, A. Coser, E. Medvet,
Boudewijn, M. Volpe, M. Whitworth, B. Svara, G. Sgroi, D. Panfilo, and S. Saccani. Generating realistic synthetic relational data through graph variational autoencoders. arXiv preprint arXiv:2211.16883 2022.
[32] J. Motl and O. Schulte. The ctu prague relational learning repository. arXiv preprint vrxiv:/51/ 1.0086, 2015.
[33] H. Nickisch and C. E. Rasmussen. Gaussian mixture modeling with gaussian process latent variable models. In Joint Pattern Recognition Symposium, pages 272-282. Springer, 2010.
[34] NIST. 2018 differential privacy synthetic data challenge, 2018. Accessed: 2024-05-17.
[35] N. Patki, R. Wedge, and K. Veeramachaneni. The synthetic data vault. In 2016 IEEE inter- national conference on data science and advanced analytics (DSAA), pages 399-410. IEEE, 2016.
[36] V. K. Potluru, D. Borrajo, A. Coletta, N. Dalmasso, Y. El-Laham, E. Fons, M. Ghassemi, S. Gopalakrishnan, V. Gosai, E. Kreačić, G. Mani, S. Obitayo, D. Paramanand, N. Raman, M. Solonin, S. Sood, Vyetrenko, H. Zhu, Veloso, and Balch. Synthetic data applications in finance. arXiv preprint 240.:24000000088 2024.
12

---

[37] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[38] O. Schulte and Z. Qian. Factorbase: multi-relational structure learning with sql all the way. In International Journal of Data Science and Analytics, Int J Data Sci Anal 7, page 289-309. Springer International Publishing AG, 2019.
[39] O. Schulte, Z. Qian, A. E. Kirkpatrick, X. Yin, and Y. Sun. Fast learning of relational dependency networks. Machine Learning, 103:377-406, 2016.
[40] J. Sohl-Dickstin,, E. Weiss, N. Maheswaranathan, and S. Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. International conference on machine learning, pages 1:56-22655 PMLR, 2015.
[41] J. Song, C. Meng, and S. Ermon. Denoising diffusion implicit models. arXiv preprint wwXiv:2010002500 2020.
[42] Y. Song, Sohk-Dic,stein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020.
[43] T. Stadler, B. Oprisanu, and C. Troncoso. Synthetic data anonymisation groundhog day. In I USENIX Security Symposium (USENIX Security 22), pages 1451-1468, Boston, MA, Aug. 2022. USENIX Association.
[44] B. van Breugel, T. Kyono, J. Berrevoets, and M. van der Schaar. Decaf: Generating fair synthetic data using causally- generative networks. In Advances Neural Information Processing Systems, volume 34, pages 22221 -22233.. Curran Associates, Inc., 2021.
[45] B. van Breugel, N. Seedat, F. Imrie, and M. van der Schaar. Can you rely your model evaluation? improving model evaluation with synthetic test data. In Advances in Neural Information Processing Systems, 2023.
[46] B. van Breugel and M. der Schaar. Beyond privacy: Navigating the opportunities and challenges of synthetic data. arXiv preprint arXiv:2304.03722, 2023.
[47] L. Xu, M. Skoularidou, A. Cuesta- Infante, and K. Veeramachaneni. Modeling tabular data using conditional gan. Advances in neural information processing systems, 32, 2019.
[48] Young, P. Graham, and R. Penny. Using bayesian networks to create synthetic data. Journal of Official Statistics, 25(4):549-56)) Dec 2009.
[49] R. Zemel, R. Urtasun, and S. Fidler. Mixture of gaussians and em. edu/ "urtauuut/uuuuu wttpruuuttttuuree/////ooo
woronooo
mmoomog..... pdf
[50] H. Zhang, J. Zhang, B. Srinivasan, Z. Shen, X. Qin, C. Faloutsos, H. Rangwala, and G. Karypis. Mixed-type tabular data synthesis with acore-based diffusion in latent space. In The twelfth International Conference on Learning Representations, 2024.
[51] Z. Zhang, T. Wang, N. Li, J. Honorio, M. Backes, S. He, J. Chen, and Y. Zhang. PrivSyn: Differentially private data synthesis. In 30th USENIX Security Symposium (USENIX Security 2 21), pages 929 9969996...01 USENIX Association, Aug. 2021.
[52] 7.. Zhao, A. Kunar, R. Birke, and Y. Chen. Ctab-gan: Effective table data synthesizing. In Asian Conference on Machine Learning, pages 97- PMLR, 2021.
[53] S. Zheng and N. Charoenphakdee, Diffusion models for missing value imputation in tabular data. arXiv preprint arXiv:2210.17128 2022.
13

---

A Notation Summary
We use boldface represent random variables. e.g. Y Y, where Y is the data of R2, and Y is the random variable Y being sampled from. In addition, we use subscript to represent the parent row some data or random variable refers to. e.g., æy represents the child random variable that refers to parent y. The important notations used in the paper are summarized in Table 3
Relational database, relational table, synthetic table
R, R
Random variable and data for parent table
Y, Y
Random variable and data for child table
X.X
Random variable and data for grandchild table
2,2
Random variable and data for yth parent row
yj.
Random variable and data for foreign key group referring to jth parent row
93.95
Random variable and data for child rows in jth foreign key group
Random variable and data for jth foreign key group size
8,.8,
Latent cluster random variable and value
C,
Augmented table with a latent variable column
Ty (Y;C)
Directed acyclic graph, nodes, edges
G- (R,
Diffusion model for augmented table
pr
Diffusion model for child data paremeterized by ø
Ps (x).
Latent variable classifier parameterized by 1/
Ps (c x))
Classifier guided distribution, parameterized by 0.
Pe. (I)
Table 3: Notation summary
B Algorithm Details
B.1 Diagram for two-table relational databases
Figure 2 summarizes the generative process for two-table cases.
Account (chid)
Difkusion
Rolation-wware Clustering
X
y
(x. l)
(y. )
and tatenz value synthesis
Augmentad
Tabries
X
Xy
Conditional child row syrthisia
Cluster
p(c|x)
Clasetiier
geadlent
Canditianat-
p(slc)
Conditonal, group alzo sampling
Figure ClavaDDPM overview for a two-table relational database
B.2 End-to-end algorithms for more tables
We detail the end-to-end algorithms for the three phases of ClavaDDPM, including (i) latent learning and table augmentation, (ii) training, and (iii) synthesis.
Latent learning and table augmentation. As shown in Algorithm given database R. (Ri..... Rm and foreign key constraint graph g. we learn the set of latent variables Ct,y for every pair of parent-child (R; + R₁) G.E and augment all the latent variables to the parent table and the child table, denoted by T, and Ty, respectively. We initialize each augment table with its original table (line 1). This algorithm follows bottom-up topological order starting from the leaf child with its parent (line 2), ensuring each child table is already augmented by the time we learn the latent variable augment its parent. For each parent-child pair R, Ry, we join T, (not T() with R, into
14

---

single table (X;; Y) (line 3) and then run the clustering algorithm using GMM and maximum voting described in Section 4.3.I We append the corresponding clustering labels Ct to the augmented parent table 7 and augmented child table T; respectively.
Algorithm 1 ClavaDDPM: Latent learning and table augmentation.
Input: tables R = Ri...... Rm }, foreign key constraint graph g
Output: latent variables {C,,) R₂ + R₂) E G.E), augmented parent tables (Ti,..., Tm
augmented child tables (71.-.. T
1: Initialize augmented tables {Ti,.... TT + R. (Ty..... ...... T R
2: for (R; - -Ry) in bottom-up topological order of g do
3: Join parent and augmented child (X;Y) Ry)
4: Ci.j f- Clustering (X:Y)
b Relationship-aware clustering in Section 4.3.1
5: Augment parent T3 + (T C3ag)
6:
Augment child TV - (1.))
7:
end for
Training. shown Algorithm 2 the training phase takes in the augmented parent tables (T1...., Th. and the foreign key constraint graph G. For each augmented table T,, we train diffusion model pr, (lines 2-4). Then, for each parent-child pair R₃ + R2 (lines 5-7), we train a child classifier Po(Ci,)2z) (z) with R 's child augment table TV, where the latent column C.. is used as labels, and all remaining columns are used as training data (including the augmented latent columns corresponding to R2 's children). Using the same table, we also estimate the foreign key group size distribution conditioned on the latent variable p(s(ca,).
Algorithm 2 CIavaDDPM: Training
Input: augmented parent tables (Ti...., Tm augmented child tables (TY,.... Th foreign key constraint graph
Output: diffusion models D, classifiers C, group size distributions S
1: Initialize D, C, + 0
2: for Ry in G.R do
3: Train Pr, with Tj. and add to DD
4: end for
5: for (R4 (Ra+11 R2) in topological order of G do
6: Learn classifier Po(c,, (z) and p(s((1,) using with TV (ignoring irrelevant latent columns)
and add to C and S respectively
7: end for
Synthesis. Algorithm takes in learned diffusion models D, classifiess C, group size distri- butions S, and the DAG representation of the database G, and outputs the synthetic database R. [R, .....,mmm We first initialize the synthetic augmented tables be empty (line I). Then, for root augmented tables, since they have no parents to condition on, they can be directly synthesized from their diffusion models (line 2-4). Next, we traverse the database in topological order to synthesize the remaining augmented tables (line 5-16): If we have already synthesized T; before, which means we encounter the multi-parent dilemma, we just store the old version and continue to generate a new version (line 6-9). For each parent-child relationship RG - Ry, we must have already sampled the augmented parent table 7,. This is because we follow the topological order of a DAG, and all root augmented tables have been synthesized as base cases. Therefore, we can obtain the synthetic latent variables Či.j from the synthetic augmented parent T; (line 10). Then, iterate through each synthetic latent value شi,ة and perform a two-step sampling: (1) use the learned group size distribution conditionally sample a group size ğ (line 12); (2) sample á rows of data conditioned on č.jj using classifier guided sampling (line 13). We repeat this process until the augmented child table T; is fully synthesized. We simply obtain synthetic tables from synthetic augmented tables by removing all synthetic latent columns (line 17-19). Finally, for all the encountered multi-parent dilemmas, follow Section 4.3.3 to match the divergent versions.
15

---

Algorithm 3 ClavaDDPM: Synthesis
Input: diffusion models D, classifiers C, group size distributions S, foreign key constraint graph
g
Output: Synthetic tables [R...... Rm
1: Initialize fi,..., T. .,...,
2: for Ry root nodes do
3: Sample T3 PT;
4: end for
5: for (R, + R2) in topological order of G do
6: if T; already synthesized then
7: Store T; as Tik, where i is the parent that synthesized Tip
8: Reinitialize T; 0
9: end
Split Tý into (-:Cis)
10:
11:
for či, in Ci. do
12:
Sample 8 ~ p (s ())
13:
Classifier Slassifir-guidee sample 3 rows of data: & pt, (āg)
14:
Append & to T3
end for
16: end for
for T, in all synthetic augmented tables do
18: Rg + aal latent columns removed from T
19: end for
20: for R with multiple synthetic versions do
21: R2 + MATCHING (Ri........ Rz...)
22: end for
TABLES
FOREIGN KEY PAIRS
DEPTH
TOTAL # ATTRIBUTES
ROWS LARGEST TABLE
California Intacart 05
15
1. 1,690, 642
12
1,616,315
Berka Movie Lens
41
1,056,300
996, 159
CCS
2
11
383, 282
Table 4: Dataset Specifics
C Experimental Details
C.1 Datasets
Here describe the real-worke datasets used in evaluation in detail. The specifics of datasets in Table
California: The California dataset is real-world census database (()) on household information. It consists of two tables in the form of basic parent-child relationship.
Instacart 05: The Instacart 05 is created by downsampling 5-percent from the Kaggle competition dataset Instacart (23)), which is real-world transaction dataset of instacart orders. This dataset consists of 6 tables in total with maximum depth of 3.
Berka: The Berka dataset is real-worle financial transaction dataset (14), consisting of 8 tables with maximum depth of This will be the main dataset in our work for ablation study and model anatomy.
Movie Lens: The Movie Lens dataset (39] consists of 7 tables with maximum depth of This dataset exhibits complex multi-parent and multi-children structures.
16

---

CCS: The CCS dataset ((32)) is real-world transactional dataset Czech debit card company. It consists of tables with a maximum depth of 2, which exhibits complex multi-parent and multi- children patterns.
C.2 Baselines
We adopt two multi-relationa synthesis models in literature as our baselines: PrivLava [5] as a representative of state-of-the-ar marginal-based methods, and SDV [35] as statistical method specially designed for multi-relational synthesis. In addition, we introduce two types of multi- relational synthesis pipelines, SingleT and Denorm, as additional baselines. For the additional baselines, we use CTGAN (47) and TabDDPM ((5)) as backbone models, representing the state- of-the-art tabular synthesis algorithms with GAN-based models and diffusion-based models. In the following, we describe the high-level ideas of Single-T and Denorm.
Single-T: Given single-table backbone model, we first learn and synthesize each table individually, Then, for each parent-chile table pair (p, c), assign foreign keys the synthetic child table R. by randomly sampling group sizes in the real table Re, which enforces the synthetic group size distributions be similar to real ones.
Denorm: For each parent-child table pair (p. c), we join the table into Rp.es then use the single-table backbone model to synthesize the joint table Rp.c. Finally, we split &p into two synthetic tables Rp and R. as follows: (1) Lexicographically sort Rp.c. where the parent columns are prioritized. This guarantees that similar parent records are grouped together. (2) From the real table Rg, randomly sample group sizes 9 with replacement. Then, for each sampled g. the consecutive ğ rows in Rp.c will be taken as a synthetic foreign key group Pp.c- The child columns part of Pp.c will be assigned the same foreign key and appended to the child synthetic table R.. Then, randomly sample parent row in Pp.c and append to the parent synthetic table Rt. We follow the exact same way as in ClavaDDPM to extend 2-table Denorm to the entire database.
Random matching: We conduct ablation study by training a ClavaDDPM model with the same setup as the default setting, while instead of performing table matching to handle the multi-table dilemma, it performs naive merging of two synthetic tables. For the diverged synthetic tables RD.A and RD.C. where RD, A is the Disposition table synthesized conditioned on the Account table, and RD. is conditioned on the Client table, we simply keep RD, and randomly assign the (D, C) foreign keys from RD.C to RD.
C.3 Implementation Details
C.3.1 Classifier Training
We use an MLP for classifier with layers 128, 256, 512, 1024, 512,256, 128. The output layer size is adapted to the number of clusters k. We use learning rate of le - 4, and optimize with AdamW optimizer, and use cross entropy loss as objective. The overall training paradigm follows [10, where incorporate timestep information by encoding the timesteps into sinusoidal embeddings, which are then added to the data. For experiments on California, we train the classifier for 10000 iterations, and for all other datasets we train 20000 iterations.
C.3.2 Hyper Parameters
Baseline models. PrivLava was run under non-private setup by setting privacy budget 50, and the datasets are prepossessed spesifically for PrivLava to have domain sizes less than 200.
For all models with ClavaDDPM or TabDDPM backbones, we use the same set of hyper parameters.
We set diffusion timesteps to 2000, and use learning rate of Ge 4. In terms of model architecture, we use MLP with layer sizes 512, 1024, 1024, 1024, 1024, 512. The model architecture details are following the implementation of TabDDPM (25). All DDPM- models are trained 100, 000 iterations on California dataset, and 200, 000 on other datasets.
17

---

CALIFORNIA
INSTACART OS
BERKA
MOVIE LENS
CCS
Num Clusters
25
20
50
25
Parent Scale
1.5
1
1
Classifier Scale v
Table 5: Hyper parameters CIavaDDPM on each dataset.
We conducted CTGAN experiments using the interface from SDV library, and follows the default parameters, where the learning rates for the generator and discriminator are both 2e - 4, and is trained 300 epochs.
PrivLava's code is not publicly available, and we directly followed the authors' settings. Note that PrivLava requires privacy budget searching process, and 50 is the largest working privacy budget according to our experiments, where larger leads to failure. We consider this as large enough to resemble non-private setting.
For SDV, we used the default setting of their HMASynthesizer, which by default uses a Gaussian Copula synthesizer.
CIP settings. We list the major hyper parameters used by ClavaDDPM for each dataset in Table [ and we provide an empirical guidance for hyper parameter tuning: it is suggested to use number of clusters k to be least 20, and classifier scale 17 to be in [0.5, 2]. We empirically find ClavaDDPM consistently perform well in such range across all datasets. Parent scale À is a less sensitive factor, and with small hyper parameter space, and there is very little need for tuning.
stable starting point for tuning. In general, ClavaDDPM is robust,
C.3.3 Metries
C2ST. The Classifier Two Sample Test trains a logistic regression classifier to distinguish synthetic data from real data. We consider this metric as high-level reflection of data fidelity.
Machine Learning Efficacy. Different from prior works that evaluate MLE utilities who work on datasets with predefined machine learning tasks, the five real-world multi-relational datasets we use do not come with a designated downstream task. addition, the prior knowledge about which column will be used for downstream predictions will introduce significant inductive bias to the training process, especially for models capable of performing task-oriented training. To avoid such issue, we evaluate machine learning efficacy each of the columns. To be specific, each time we select a column as target, and train an XGBoost [8] model on remaining columns. For categorical target columns, we perform regression and evaluate R², and for categorical target columns we perform classification and evaluate Fi. The overall MIE is measured by average R² and average Fi across all columns.
To evaluate the single-table MLE on synthetic data generated from multi-tabi synthesis process, instead of performing independent train-test split each table, we split by foreign key relationship. e.g. for California dataset, we first perform a random 90%, 10% split on the parent table Household, and then we follow the foreign key constraints to assign corresponding child rows, i.e. Individuals to the corresponding buckets. Note that although this splitting method does not lead to the same train/test ratio on child table, consider such sampling to be foreign key relationship preserving, which is a more important property in the context of multi-table synthesis.
D Additional Experiments
D.1 Agree rate discussion
As introduced Section 4.3. within each foreign key group, we perform majority voting to synchronize the assigned cluster label among the group. To measure the consistency of such majority voting process, we introduce the measurement agree rute, which computes the average ratio of agrecing on the mode within each foreign key group, and the metric avg agree-rate is the average of
18

---

all per-table agree rates within a multi-table dataset.
mg
A(g)
(II)
lgl
A(g) represents the agree rate of some group g. where my represents the number of records that are assigned the mode cluster within g, and the avg agree-rate
AAVG
Σ A(g)
(12)
G
9EG
computed as the average of the agree rates across all groups.
However, our experiment results in Table abbleiindicate indicate that the relationship-aware clustering process is robust against such factors, and the GMM model is capable of capturing nuances in data patterns. In our experiments on Berka dataset, ClavaDDPM's clustering process achieves a consistent agree rate around 81%, which is practically high enough given we have 20 clusters. Intuitively, when parent scale approaches infinity, the clustering is performed completely on parent table, which will lead to a perfect agree rate. Also note that higher agree rate does not always imply a better performance, and the disagreement can potentially come from the intrinsic parent-child relationships. e.g. when child data intrinsically independent of parent data, it reasonable to have noisy learned latent variables, leading to low agree rates. However, in such cases the noisy latent variable would not degenerate model performance, because the best strategy will be a direct sampling of child table, rather than conditioning some enforced prior distribution. In addition, as shown in Table agree rates are highly affected by the number of clusters chosen, and there exists a trade-off between the granularity in clustering and consistency. In an extreme case where we have k = 1 cluster, indicating an infinitely coarse-grand graieedg latent learning, it trivially achieves perfect agree rates.
D.2 Selecting Number of Clusters k
We conducted finer-grained experiments to examine the effect of the number of clusters on model performance, as shown in Figure 3 which offer empirical insights for selecting k. Based on the results, (1) a binary search approach could be used to efficiently find a suitable k and (2) while may require more computational resources, opting for larger k is generally safe choice.
AVG 2eway [5mmmthedd]
I
Data Points
90
08 1.55
88
A3a
11.40
86
2
AVG
89
82 00044 1:
80
500
K.ooo scale)
Figure 3: Smoothed model performance on Berka dataset regarding different A (measured by AVG 2-way), where k oo represents assigning each row a unique class,
D.3 High-order Single-table Evaluation
We also consider higher-order single-table evaluation metrics for the quality of some representative tables as prior work [50]: 1) a--precision and 3-recall [ to measure fidelity and diversity of synthetic data; 2) Machine Learning Efficacy (MLE) to measure the downstream-task utility; 3) Classifier Two Sample Test (C2ST) to measure if the synthetic data is distinguishable from real data by machine learning models.
19

---

PavLava
SDV
STETIGA
STADDD
ClaaDDDMM
D
TUNDDPM
CloaaDDM
ClusaDPM
Houscheld
87.32
56.08 =200
2006
01.50 2.222
21121
AVG
18.21
63.03 1::00
65.96 1./13
1:0.7
64.07 1:0.0
1.100
06.33
06:71
1070
Individual
PRECISION
99.43
2006
55.80
20.6
$3.55 4:00
88.41
430L
99.74 2:006
87.52 =0.0
96.08
#:0.44
90.55 #..5
98.00 =4.00
3-RECALL
60.40
15.15
0.52 =0.11
46.65
#:668
31.00 +3.13
63.14 1.774
39.33
=0.21
$7.05
#2203
62.34 #20
665.1 #:477
C2ST
09.63
1:22
5.29 2:131
78.82
26.71
6X.25 5.0
99.76 1::17
76:44
2:.12
94.50
1:1.6
981.23
97.06
2.139
AVG
59.50
+:00
22.87 +.00
58.57 +0.16
50.19 +307
61:35 1
57.58 +.0.0
57.94
+:117
60.95
6.00
61.22 0..6
AVG R2
73.51
2030
-157.24 20.42
80,52 =6.00
81.33 2013
83.13 2:00
77.91 20.0
82.23 20.4
50.08 2023
80.12 =0007
Table High-level single-table metrics evaluated on the Household table and the Individual table
from the California dataset.
Houschold
Individual
Transaction
Order
DCR- SER-SMOTE
0.0295
0.0304
0.0082
0.0029
DCR-CLAVADDPM
0.0647
0.0407
0.0101
Table 7: Median DCR comparison between ClavaDDPM and SMOTE.
We evaluated high-ordet single-table metrics the California dataset across all baseline models and ClavaDDPM. Following [50], for the evaluation of MLE we perform a 90%, 10% train-test split, where the Fi and R² metrics evaluated the 10% holdout set. Note that although PrivLava has an advantage on the California dataset when evaluated with statistical tests (Table D.. ClavaDDPM stronger performance than PrivLava for MLE, ClavaDDPM surpasses PrivLava by 13.07% in terms of average R² Individual table, and also beats PrivLava on average Fi in both tables. Also notice that the baseline ST-CLAVADDPM dominates in high-order metric evaluations, demonstrating the strength of our Gaussian diffusion-only backbone model.
exhibits competitive,
higher-order metrics. Especially
ClavaDDPM achieves a second second-highest 8-recall on Household table and ranks first in B-recall on Individual table with large margin, gaining 7.65% advantage over the best baseline without ClavaDDPM backbone. This serves as strong evidence that ClavaDDPM is not only data fidelity preserving, but is also capable of generating highly diverse data.
D.4 Privacy Sanity Check
We follow TabDDPM 25 to perform a privacy sanity check against SMOTE 77, which is an data through convex combination of real data point
interpolation-batt method that generates with its nearest neighbors. We use the median Distance to Closest Record (DCR) [52] to quantify the privacy level. We compare the median DCR, as well as DCR distributions of ClavaDDPM against SMOTE on selected tables.
As shown in table 7 ClavaDDPM although neither specialized in privacy preserving. nor in single table synthesis, it still maintains a reasonable privacy level. The charts demmonstteessss the distributions of DCR scores, where ClavaDDPM is in blue. The overall distribution is more leaning to the right side, indicating an overall higher DCR distribution.
(a) Houschold
(b) Individual
Transaction
(d) Order
Figure 4: DCR distributions of four selected tables. The axis is log-scaled for better presentation.
20

---

E Complexity Analysis
Given a multi-rclational database g ER, with m tables, n foreign key constraints, and p rows per table. For p-row table, we denote the time complexity of performing GMM clustering as (p) training a diffusion model as Can (an(p), training classifier as Catss (p), synthesizing as (p)( ANN searching as (A))
Phase latent learning and table augmentation
COMM (p)
(13)
Phase 2: training
Celass (p) m (pit(()
(14)
Note that in practice this phase is dominated by diffusion training, primarily influenced by m.
Phase synthesis
Csyn
(15)
Additional step:
matching
(p)
(16)
Note that the runtime in this phase is negligible compared to the earlier phases, particularly with the FAISS implementation in the non-unique matching setup.
Total
(CoMMip)+ (p) + (y) CANN + mC)n(()
(17)
the overall runtime is dominated by Phase (training) and Phase 3 (synthesis), with the critical factors being m, n, and The model remains robust against the number of clusters in Phase 1, as the impact runtime is minimal due the dominance of the later phases.
21

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We provide empirical evidence in the evaluation section for the claims made in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: In the conclusion section (second paragraph), we discuss the limitations of our work and our interest in addressing them future work.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sellsspeciiiationn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
22

---

Answer: [Yes]
Justification: We list all the assumptions required to derive our generative process. We do not have any proof.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proof' in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We detail the datasets, baseline algorithms, implementation details of our algorithms, and evaluation metrics in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
23

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material? [Yes] Justification: reprodueibility.
Question:
We upload supplementary materials including code for
Guidelines:
The answer NA means that paper does not include experiments requiring code. Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips /ttpss.ccpppllieee ctc/public/ggudeen gewcc//ggge/////eeeeeooe ProdeSubmissiooPolltti
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/eetails
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide the basic experimental setting/details, ablation study in the evalua- tion section, and details in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification: We repeat experiments for each configuration three times and report their averaged performance with standard deviations.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative emrr rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We include computing specifics in the evaluation section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethic
Answer: [Yes]
Justification:
We carefully go through the NeurlPS Code of Ethic and ensure we follow it.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification: discuss the broader impacts of our work in the conclusion section.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
25

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We use public datasets for experiments.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We add citations and resources for the datasets and source code used in the evaluation.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
26

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We document new code and and provide references for all code adopted from public resources.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
27


Reviews:
[{"review_id": "diUAaK5zdv", "reviewer": "Reviewer_ok29", "summary": "This work addresses the challenges inherent in generating multi-relation tabular data and proposes a novel generation method based on hidden random variables.  The approach analysis correlations between primary and foreign keys between tables by predicting hidden variables associated with these keys within simultaneous tables.  These hidden random variables are inferred using Denoising Diffusion Probabilistic Models (DDPM) grounded in stochastic processes.  The method is elaborated from two-table linkage generation to multi-table linkage generation, with a step-by-step explanation of the approach.  Comprehensive experiments are conducted to demonstrate the practical effectiveness of the proposed method.", "questions": "Question 1: In line 79 to 85, the author mention table with R and relation with R. Is this means that a relation in Relational Database is a table in this paper? Since the relations are model as edges of graph.\n\nQuestion 2: According to the paper content, the author mention that “the primary key of table serves as the unique identifier for each row in the table”. And in section 4.1 the rows in parent table can suitable i.i.d assumption. If every row is unique, how rows can be suitable independent and identification distributions?\n\nQuestion 3: From the perspective of learning latent variable method alone, what are the advantages of CLavaDDPM proposed in this paper compared with other studies?", "limitations": "The authors adequately addressed the limitations.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "This paper highlights the extensive dependencies between different tables in the process of multi-relation tabular data synthesis and introduces an innovative approach by analyzing the relationships between primary keys and foreign keys using hidden variables.  The proposed ClavaDDPM process is well-articulated and has demonstrated promising results across several datasets.  Additionally, the subsequent analysis of hidden variables enhances the credibility of this work.", "weaknesses": "Although this work attempts to elaborate on its theoretical framework with substantial content, the notation system and descriptive symbols used do not conform to standard academic mathematics conventions (see sections 3. Background and 4.1 Notations). The marks in equation are not uniform (Eq.2). Furthermore, several key formulas lack thorough proofs (for example, Formula 8). The proof of this key formula would be best placed in the appendix. Additionally, Formula 3 from a previous study lacks supporting evidence; a brief explanation of its proof process or an indication of its rationale is necessary. The validity of the method “we introduce latent random variables c such that g is independent of y conditioned on c” also requires theoretical verification.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6PExBXtxqq", "reviewer": "Reviewer_5kZH", "summary": "This paper proposes ClavaDDPM to address two key deficiencies in multi-table data generation: scalability for larger datasets and capturing long-range dependencies, such as correlations between attributes across different tables. This approach utilizes cluster labels as intermediaries to model relationships between tables, paying special attention to foreign key constraints. The authors elaborated on the proposed method, made some assumptions, conducted some experiments for evaluation, and performed some analysis about the results.", "questions": "1. The paper assumes that there are parent-child relationships between different tables and the relationships are known. I However, in real life, the relationships may be very complex and may not be known in advance. In this case, how can ClavaDDPM handle it?\n\n2. The i.i.d. assumption in Section 4.1 seems too strong, because the child rows corresponding to different primary keys may not be independent in real-world scenarios. If this is the case, does the theoretical analysis in the paper still hold?\n\n3. Please clarify why you selected CTGAN over newer or state-of-the-art models like CTAB-GAN, TabSyn, and CoDi mentioned in the related work for your experiments.\n\n4. The reason for the absence of average agree rate in the last two columns of Table 2 remains unclear. Additionally, it is not specified what NA represents. Moreover, there is a lack of clarity regarding the calculation method and formula used to determine the agree rate in this experiment.\n\n5. I feel a bit confused about why TabDDPM converges so slowly on the Instacart05 and Movie Lens datasets. The authors of TabDDPM stated in their work that TabDDPM can converge quickly with a 2080Ti GPU. In addition, I noticed that the Berka dataset seems to be larger than those two datasets. Could you kindly provide an explanation as to why TabDDPM exhibits convergence on this larger dataset, while failing to do so on the two smaller datasets?\n\n6. Are the evaluation metrics used in the experiment widely used in the field or are they proposed by the authors? I haven't seen similar metrics in related works. If they are proposed by the authors, why don't you use metrics from other papers, such as the relative error used in PrivLava [1] or more comprehensive metrics used in TabSyn [2]?\n\n7. There are some typos in the paper: “LavaDDPM” in line 206, “Dnorm” in the caption of Table 1, “The experiment result show…” in line 355.\n\n8. Page 18's experimental results table lacks a caption.\n\n9. In Section 5.2, you referred to additional results in Appendix 5.3, which is missing from the paper. Please provide the mentioned experimental results.\n\n10. In the checklist, you said that you discussed the limitations and future work in the second paragraph of the conclusion section, but there is no relevant discussion in the conclusion of the paper, not even the second paragraph, which is confusing.\n\nReference:\n\n[1]\tK. Cai, X. Xiao, and G. Cormode. Privlava: synthesizing relational data with foreign keys under differential privacy. Proceedings of the ACM on Management of Data, 1(2):1–25, 2023.\n\n[2]\tH. Zhang, J. Zhang, B. Srinivasan, Z. Shen, X. Qin, C. Faloutsos, H. Rangwala, and G. Karypis. Mixed-type tabular data synthesis with score-based diffusion in latent space. arXiv preprint arXiv:2310.09656, 2023.", "limitations": "n/a", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper proposes an efficient framework to generate multi-relational data that preserves long-range dependencies between tables, and proposes relation-aware clustering for modeling parent-child constraints.\n\n2. The paper applies a matching technique based on approximate nearest neighbor search as a general solution to the problem of multi-parent relation synthesis for a child table with multiple parents.", "weaknesses": "1. I have some concerns about the assumptions made in the paper, which seem to be somewhat inconsistent with the scenarios in real world settings. Please refer to Questions 1 and 2 for details.\n\n2. There may be some clarifications needed in the experimental section. Meanwhile, the analysis of the experimental results does not cover some key points. Please refer to Questions 3-6 for details.\n\n3. There are some problems with the writing and formatting of this paper. Please refer to Questions 7-8 for details. Also, there are some confusing statements in the paper, which hinder the readability of the paper. Please refer to Questions 9-10 for details.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**A1: Known relationships**\n\nWe fully agree on the importance of realistic assumptions. This work tackles a prevalent challenge in the finance industry [in collaboration with them] on tabular synthesis. Synthesizing data for multiple interconnected tables, even with known foreign keys, has been entirely overlooked in the literature, with only two notable exceptions offering inefficient solutions [1,2]. Our work presents a practical approach that surpasses the state of the art (SOTA) with known foreign keys. We will discuss addressing unknown constraints in future work in the revision.\n\n$\\quad$\n\n**A2: I.i.d assumptions**\n\nWe appreciate your perspective and agree that the assumption in Sec 4.1 is strong. While previous works TabSyn and TabDDPM [6,7] made even stronger assumptions (each row is i.i.d), we introduced a weaker assumption: in a parent-child two-table scenario (in Sec 4.1), each parent row is i.i.d., while the child rows are dependent on its corresponding parent row. In fact, we are making a Bayesian modeling assumption that each child row, although not independent by itself, is conditionally independent of other child rows given its parent. \n\nOur evaluation in Sec 5.1 demonstrates the effectiveness of our weaker assumption (ClavaDDPM) on real-world datasets compared to that of prior works (ST-ClavaDDM with strong i.i.d assumptions for child rows). \nWhile our assumption may not be perfect (modeling all dependencies with perfect accuracy is hard), our empirical results support our theoretical analysis and outperform previous works' assumptions. We will clarify these and discuss weaker and more realistic assumptions as future work.\n\n$\\quad$\n\n**A3: Choice of baselines (CTGAN and TabDDPM over CTABGAN, TabSyn, and CoDi)**\n\nOur work focuses on a multi-table synthesis with a new paradigm. Thus, the strength of single-table baseline models was not our only consideration. Our goal was to pick one diffusion model and one GAN-based model, each being *SOTA* and *representative* in their domain for a diverse comparison.\nFirstly, we excluded CTABGAN(+) or CoDi because they were shown inferior to TabDDPM and TabSyn [6,7]. Since both TabDDPM and TabSyn are SOTA, we chose between them. Given our ClavaDDPM employs a similar model to TabDDPM, comparing against baselines using TabDDPM offers a fair evaluation, highlighting the effect of our Clava framework while disregarding the inherent advantages of TabSyn over TabDDPM. Additionally, our preliminary results (with slightly different metrics) at Clava’s early development indicated TabDDPM’s superior performance over TabSyn as the backbone on real-world datasets. **(Table 2 in comments)**\n\nThus, we chose TabDDPM as the backbone for both our model and the two baselines, ST- and Dnorm-. Even with other baselines, ClavaDDPM's advantage would remain evident.\n\nThough CTGAN is weaker than TabDDPM,  we included it because it is a representative GAN-based tabular synthesis model, aligning with the TabSyn paper’s inclusion of CTGAN over CTABGAN.\nWe will add the clarification and include experiments on other baseline models, as you suggested in the revised version.\n\n$\\quad$\n\n**A4: Clarification on agree rate**\n\nThanks for the suggestion. We will detail the agree rate formulation in the appendix. The agree rate is only influenced by k and lambda, but not eta or “no matching” (the last two columns that are not involved in the clustering process). We will include the agree rates in those columns for completeness and clarity.\n\n$\\quad$\n\n**A5: Clarification on convergence**\n\nWe will add a complexity analysis in the revised paper. Convergence depends on both the data size and the domain size. Even though TabDDPM can handle all datasets on a 2080Ti GPU (A6000 is overkill), its multinomial diffusion for categorical features is a bottleneck with large domain sizes, as exemplified by  Instacart05 and Movie Lens datasets. For instance, Instacart05 has a categorical column with 49,688 values, requiring one-hot encoding and expensive multinomial diffusion. In TabDDPM's single-table datasets, categorical columns have relatively few values(e.g., the largest category count in the adult dataset is 42). The Berka dataset, despite having more tables, has the largest categorical domain size of 77, allowing faster convergence for TabDDPM.\n\nReal-world data often contain categorical columns with numerous distinct values. ClavaDDPM accelerates training by applying a unified Gaussian diffusion to both categorical and numerical columns, leading to faster convergence.\n\nAdditionally, for fair comparison, we aligned TabDDPM's hyperparameters with ClavaDDPM's, which are larger than the original TabDDPM settings, and thus will be slower than the original TabDDPM (e.g., diffusion timesteps = 2000 vs. 1000).\n\nOur TabDDPM experiments used the TabDDPM library within the Synthcity framework. We will release the baseline code for reproducibility.\n\n$\\quad$\n\n**A6: The choice of evaluation metrics**\n\nWe used single-table data quality metrics from TabSyn and reported KS/TV metrics in the main paper, and the rest (alpha precision, beta recall, classifier detection, machine learning efficacy) in the appendix (section D.2) due to space constraints. \n\nWe did not use the multi-table metric like relative error from PrivLava for\n- It relies on human-designed queries, not applicable to all datasets.\n- No open-source evaluation code, hard to replicate.\n- No theoretical validation for these hand-designed queries.\n\nThese limitations highlighted a gap in multi-table synthesis evaluation. Therefore, we proposed the long-range dependency metric,  a statistical measure applicable to all multi-table datasets without needing hand-designed queries.\n\n$\\quad$\n\n**A 7,8,9: Formatting issues**\n\nThank you for your feedback. We will address them in our revision.\n\n$\\quad$\n\n**A10: Conclusion**\n\nWe apologize for the omission of limitations and future work due to page constraints. These sections will be included in the revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yIfvnjkWha", "reviewer": "Reviewer_bZ6q", "summary": "This paper proposes the new ClavaDDPM approach to address the scalability and long-range dependency challenges in tabular data synthesis. ClavaDDPM uses clustering labels to model inter-table relationships, particularly focusing on foreign key constraints, and employs diffusion models' robust generation capabilities along with efficient algorithms to propagate learned latent variables across tables. Evaluations demonstrate ClavaDDPM's superior performance in capturing long-range dependencies and competitive utility metrics for single-table data.", "questions": "See the above weaknesses.", "limitations": "No.", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "$\\bullet$ The manuscript is commendably well-structured, with a logical flow that effectively guides the reader through the paper. The clarity in writing and the organization of content significantly enhance the reader's comprehension and the overall quality of the presentation.\n\n$\\bullet$ The authors have conducted a thorough experimental validation, which is a significant strength of the paper. The meticulous detailing of experimental procedures, including data handling, model training, and evaluation metrics, greatly contributes to the reproducibility and credibility of the study.\n\n$\\bullet$ The topic of the paper addresses a highly relevant issue in the field, which is a strong point. The research aligns well with current trends and challenges in the domain, ensuring that the findings will be of interest to a broad audience and have potential applications in real-world scenarios.", "weaknesses": "$\\bullet$ The manuscript exhibits a lack of punctuality and standardization in the presentation of mathematical formulas, leading to potential ambiguity in interpretation. Key terms are not consistently defined upon first use, and there is a misuse of symbols that may confuse readers. Additionally, the placement of table names appears to be incorrect in some instances, further detracting from the paper's clarity. It is essential to standardize the notation and ensure consistent definitions and proper placement of table names for better readability.\n\n$\\bullet$ The assumptions made in the model section require more rigorous justification. The authors should provide a more detailed explanation of why these assumptions are reasonable and how they contribute to the validity of the model. This will strengthen the theoretical foundation of the work and enhance the credibility of the proposed approach.\n\n$\\bullet$ The paper would benefit from an analysis of the model's complexity, particularly in the context of large-scale databases. Given the increasing size and complexity of real-world databases, understanding how the proposed model scales and performs under such conditions is crucial. Including a complexity analysis will provide insights into the model's practicality and efficiency. \n\n$\\bullet$ The ablation study regarding the cluster number 'k' lacks necessary granularity. A more fine-grained investigation is needed to understand the impact of varying 'k' on the model's performance and to identify the optimal settings for different scenarios. This will provide a clearer understanding of the model's behavior and its adaptability to various contexts.\n\n$\\bullet$ The section on the multi-relational synthesis problem, currently placed in the appendix, should be relocated to the main body of the paper. This will improve the flow of the paper and make it easier for readers to follow the synthesis process and understand its relevance to the overall work. Integrating this section into the main text will enhance the coherence and comprehensiveness of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**A1: Presentation improvements**\n\nThank you for your valuable feedback. We will take steps to improve the writing and overall presentation of the paper, and have fixed the table names issue, and will have a better written revised version.\n\nTo further enhance our manuscript, we would greatly appreciate it if you could point out any specific issues with the mathematical formulations or inconsistency in definitions, so we can address them in detail.\n\n$\\quad$\n\n**A2: Justification of assumptions**\n\nThank you for highlighting the need for more rigorous justification of our assumptions. We appreciate your feedback and would like to provide a detailed explanation:\nWe understand the importance of validating assumptions in tabular data synthesis. When reviewing the literature, such as TabSyn and TabDDPM, we observed that they implicitly rely on the naive row-wise i.i.d. assumptions, which lack validity when extended to multi-table scenarios. To address this, we reexamined the problem and adopted a weaker and more realistic assumption: that child rows are conditioned on foreign key constraints. We provided a mathematical deduction to strengthen the theoretical foundation of our approach, aiming to address tabular data synthesis more rigorously than simply applying generative models as done in previous works.\nWhile it is challenging to justify these assumptions perfectly in alignment with real-world scenarios, we designed experiments to empirically validate them. In section 5.1, Table 1, we compared ST-ClavaDDPM with ClavaDDPM. ST-ClavaDDPM utilizes the same model backbone as ClavaDDPM but makes the i.i.d. assumption on all child table rows. In contrast, ClavaDDPM assumes that child rows are conditioned on parent rows. The experimental results in Table 1 indicate that ClavaDDPM significantly outperforms ST-ClavaDDPM, empirically demonstrating that our assumption is more effective on real-world datasets compared to the naive row-wise i.i.d. assumption.\nAlthough our assumptions may not be perfect, our empirical results show that the theoretical analysis in our paper is robust and surpasses the assumptions made in previous works. Additionally, we consider exploring even weaker and more realistic assumptions as a direction for future work.\n\n$\\quad$\n\n**A3: Complexity analysis**\n\nThank you for your suggestion. We agree that a complexity analysis is essential for understanding the scalability and efficiency of our model, especially for large-scale databases. Here is a preliminary analysis, which we will expand upon with detailed results in the revised version.\n\nGiven a multi-relational database $G = (R, E)$ with $m$ tables, $n$ foreign key constraints, and $p$ rows per table. Given a p-row table, we denote the time complexity of performing GMM clustering $c_{GMM}(p)$, training a diffusion model as $c_{diff}(p)$, training a classifier as $c_{class}(p)$, synthesizing as $c_{syn}(p)$, ANN searching complexity as $c_{ann}(p)$.\n\n*Phase 1*: Latent Learning and Table Augmentation:\n- Runtime: $n \\cdot c_{GMM}(p)$.\n\n*Phase 2*: Training:\n- Runtime: $n \\cdot c_{class}(p) + m \\cdot c_{diff}(p)$.\n- This phase is dominated by diffusion training, primarily influenced by $m$.\n\n*Phase 3*: Synthesis:\n- Runtime: $n \\cdot c_{syn}(p)$.\n\n*Additional Step*: Matching:\n- Runtime: $n \\cdot c_{ann}(p)$.\n- Negligible runtime with FAISS implementation.\n\n*Summary*:\n- Dominated by Phase 2 (training) and Phase 3 (synthesis).\n- Critical factors: $m, n$, and $p$.\n- Robust against the number of clusters $k$ in Phase 1 due to the dominance of later phases.\nOur model shows significant scalability and practicality compared to existing methods like SDV, which is limited to synthesizing at most 5 tables with a depth of 2 (section 5.2). We will provide empirical runtime measurements and baseline comparisons in the revised version.\n\n$\\quad$\n\n**A4: Fine-grained investigation of $k$**\n\nThank you for your valuable suggestion. We agree that a more fine-grained analysis of the cluster number $k$ is essential for understanding its impact on the model's performance and identifying optimal settings for different scenarios.\nOur preliminary results show an interesting trend: the model quality initially performs lower when $k$ is small, peaks at $k=25$, and then decreases. When $k$ is very large, the model performance converges to a local optimum. This suggests that a technique similar to binary search could be applied empirically to find the optimal $k$.\n\n$\\quad$\n\n$\\textbf{Table 1: Model Performance for Different k}$\n|Berka|$k=1$|$k=10$|$k=25$|$k=50$|$k=100$|$k=500$|$k=1000$|$k=\\infty$|\n|-|-|-|-|-|-|-|-|-|\n|AVG 2-way|81.64 $\\pm$ 1.09|86.87 $\\pm$ 3.51|89.21 $\\pm$ 1.95|87.43 $\\pm$ 1.34|84.7 $\\pm$ 2.27|86.4 $\\pm$ 1.56|87.77 $\\pm$ 0.83|87.81 $\\pm$ 1.70|\n\nThe results of these experiments will be included in the revised paper. This will provide a clearer understanding of the model's behavior and its adaptability to various contexts.\n\n$\\quad$\n\n**A5: Relocation of research problem**\n\nThank you for pointing this out. We agree that moving the section on the multi-relational synthesis problem to the main body of the paper would improve the flow and make it easier for readers to follow the synthesis process and understand its relevance to the overall work.\nHowever, due to the NeurIPS page limit, we had to place the definition in the appendix. We will integrate this section into the main text in the revised version to enhance the coherence and comprehensiveness of the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MUguiGfLjW", "reviewer": "Reviewer_sDU8", "summary": "The paper tackles the problem of synthetic data generation in tabular formats and particularly focuses on data generation in multi-relational (multi-table) setups like relational databases. Whereas existing tabular diffusion models work only on single tables, the proposed ClavaDDPM extends the DDPM framework to support many relational tables linked via primary-foreign keys. In the hierarchy of parent-child tables, child tables contain keys from parent tables, and ClavDDPM conditions the generative process of child entries on their parent entries (grouped by the foreign key). To enable this, the model learns latent variables (expressed as GMMs) corresponding to the groups of foreign keys. Capturing the hierarchy of parent-child relations enables learning quite deep, long-range dependencies between k-hop neighbors in the relational schema. Experimentally, ClavaDDPM outperforms existing non-diffusion baselines both in terms of effectiveness and efficiency.", "questions": "**Q1.** Most of the reported numbers (total variation distance between synthetic and real data) are in the higher 95%+ range which makes it hard to distinguish the quality of new models. Are there any other metrics close to real-world tasks that could be of use? For instance, TabDDPM and others report the quality of CatBoost/XGBoost and other standard tabular ML algorithms trained on the synthetic data.\n\nComments:   \n* Line 206: Typo in LavaDDPM    \n* Captions of Tables 1 and 2 could also mention what the reported metrics are.", "limitations": "There is no discussion of the limitations.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "**S1.** The problem of synthetic data generation is of growing importance in the era of large, data-hungry models whose quality depends on the quality of training data. Besides, synthetic generation enables upsampling of a small custom dataset (with possible private and sensitive data) for fine-tuning backbone models. In the world of relational databases, most databases consist of multiple linked tables that have to be modeled jointly to capture data dependencies. ClavaDDPM is the first tabular DDPM that supports generation conditioned on multiple tables (organized in several hierarchy levels) and can be used on real-world datasets.\n\n**S2.** The idea of learning latent variables with classifier guidance for grouped keys is rather clever. In addition, the model supports both numerical and categorical data as well as attempts to model the generative process for tables with several parents. Conditioning on several parent tables, the authors propose soft matching of latents (conditioned on each parent separately) via approximate nearest neighbor search.\n\n**S3.** Compelling experimental results - ClavaDDPM was compared against existing non-diffusion multi-table models (PrivLava and Synthetic Dava Vault), the authors also adapted single-table baselines CTGAN and TabDDPM in several versions as baselines. Ablation studies are informative.\n\n**S4.** The paper is well-written and structured, I enjoyed reading the manuscript.", "weaknesses": "**W1.** The idea of long-range dependencies in relational tables has been introduced in the beginning and highlighted in the experiments, but the main part (Section 4) does not go into the details. Perhaps Section 4.2 could expand on long-range modeling mapping to the example hierarchy from Figure 1.\n\n**W2.** No discussion of the limitations, e.g., how long the training took; inference on the datasets takes several days. The checklist mentions “the second paragraph of the Conclusion” but this section in the manuscript has only one paragraph.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**A-W1 Metric Details:**\n\nThank you for your suggestion. We agree that expanding on the details about long-range dependencies in Section 4.2 and mapping them to the example hierarchy from Figure 1 would greatly enhance the clarity. We will make this change in the revised version.\n\n$\\quad$\n\n**A-W2 Limitations and Future Works:**\n\nThank you for identifying this issue, and we apologize for the confusion.To respect NeurIPS page limit, we submitted a shortened version of our paper. However, after the submission deadline, we realized that the limitations and future work that we referred to were unfortunately part of what was cut . Here is the original second paragraph of the conclusion, we will make sure to add this to the main paper in the revised version:\n\n>\"We focused on foreign key constraints in this work, and made the assumption that child rows are conditionally independent given corresponding parent rows. This brings three natural follow-up research directions: i) extension to the scenarios where this prior information is not available and these relationships need to be discovered first[3], ii) further relaxing the assumptions, and iii) inspecting multi-relational data synthesis with other integrity constraints (e.g, denial constraints[4], general assertions for business rules). \n\n>Furthermore, we evaluated ClavaDDPM's privacy with the common (in tabular data literature) DCR metric. Nonetheless, we think it is worthwhile to: i) evaluate the resiliency of ClavaDDPM against stronger privacy attacks[5], and ii) investigate the efficacy of boosting ClavaDDPM with privacy guarantees such as differential privacy. Similarly,  the impacts of our design on fairness and bias removal, as another motivating pillar in synthetic data generation, is well worth exploring as future work. We believe the thorough multi-relational modeling formulation we presented in this work, can serve as a strong foundation to build private and fair solutions upon. \n\"\n\n$\\quad$\n\n**A-Q1 Real-world Metrics:**\n\nYes, there are other metrics being used for evaluation. We conducted single-table machine learning efficacy (MLE) experiments using the same settings as in TabSyn, which employs an XGBoost model to predict one column given the remaining columns. The results of these experiments are presented in the appendix (section D.2). These results demonstrate that, in terms of high-order or real-world metrics, ClavaDDPM achieves state-of-the-art performance, even though it was designed for multi-table synthesis. In the revised version of our paper, we will include comments about the MLE results in the main paper.\n\nAdditionally, our primary focus in this work is on multi-table quality. As shown in Table 1, ClavaDDPM exhibits significant advantages over other baselines regarding long-range dependencies. This advantage becomes more pronounced as the number of hops increases, making the quality of ClavaDDPM more distinguishable.\n\nFor multi-table MLE, this remains an unexplored area, and we plan to address it in our future work.\n\n$\\quad$\n\n**Reply to comments:**\n\nThank you for pointing out the typo, and suggestions about captions. We will make these changes in the revised version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We greatly appreciate the reviewers' efforts and constructive feedback. We humbly accept their suggestions and will make improvements accordingly. These insights will help make the paper more solid and better organized. Here, we address some general questions and misunderstandings and outline the steps we will take to address the issues in each aspect. 

$\quad$

## Assumptions and Theoretical Foundations

>Previous works like TabDDPM and TabSyn on tabular data synthesis for single tables have implicitly assumed that each row in the table is i.i.d. (independently and identically distributed). This assumption: (1) has not been specifically analyzed or validated, and (2) is unsuitable for multi-table synthesis. Therefore, our work aims to address this by extending the problem to multi-table synthesis. Our improvements include: 
>1. Explicitly stating the assumptions and deriving mathematical deductions based on them.
>2. Introducing a weaker, more realistic assumption for multi-table synthesis: 
>    - In a multi-table database forming a DAG, each row in a root table (i.e., tables without parent tables) is i.i.d.
>    - Each child table row is dependent on its corresponding parent row.
>
>We acknowledge that misunderstandings occurred, and we will enhance clarity and organization in the revision.

$\quad$

## General Writing and Formatting

>Reviewers identified several typos and incorrect table references. Additionally, we received valuable suggestions to move some sections from the appendix to the main paper and to add discussions of limitations and future work. These issues arose primarily due to NeurIPS page limits, and we appreciate the feedback. All these points will be addressed in the revision.

$\quad$

## Complexity analysis
>We appreciate reviewer bZ6q’s constructive suggestion about having a complexity analysis. We summarize the analysis here and will add it in the revised version.
Given a multi-relational database $G = (R, E)$ with $m$ tables, $n$ foreign key constraints, and $p$ rows per table. Given a p-row table, we denote the time complexity of performing GMM clustering $c_{GMM}(p)$, training a diffusion model as $c_{diff}(p)$, training a classifier as $c_{class}(p)$, synthesizing as $c_{syn}(p)$, ANN searching complexity as $c_{ann}(p)$.

>**Phase 1: Latent Learning and Table Augmentation:**
>- Runtime: $n \cdot c_{GMM}(p)$.

>**Phase 2: Training:**
>- Runtime: $n \cdot c_{class}(p) + m \cdot c_{diff}(p)$.
>- This phase is dominated by diffusion training, primarily influenced by $m$.

>**Phase 3: Synthesis:**
>- Runtime: $n \cdot c_{syn}(p)$.

>**Additional Step: Matching:**
>- Runtime: $n \cdot c_{ann}(p)$.
>- Negligible runtime with FAISS implementation.

>**Summary:**
>- Dominated by Phase 2 (training) and Phase 3 (synthesis).
>- Critical factors: $m, n$, and $p$.
>- Robust against the number of clusters $k$ in Phase 1 due to the dominance of later phases.
Our model shows significant scalability and practicality compared to existing methods like SDV, which is limited to synthesizing at most 5 tables with a depth of 2 (section 5.2). We will provide empirical runtime measurements and baseline comparisons in the revised version.

$\quad$

## Experiments

>Multi-table synthesis, as a superset of single-table synthesis, necessitates more comprehensive evaluations, especially to assess how well the synthetic dataset captures inter-table correlations. Current metrics (e.g., machine learning efficacy, statistical metrics used in TabSyn and TabDDPM) are insufficient. In addition to including experiment results for the previous metrics, we have proposed a novel multi-table quality measure: long-term dependency, which evaluates the capture of long-range correlations in a multi-table database. Reviewers noted the lack of clear demonstration and formulation of this metric. We agree and will include an additional section detailing this metric.

>Furthermore, reviewers suggested adding a finer-grained ablation study on the number of clusters k. We conducted these experiments and discovered meaningful trends that can guide practical hyperparameter searches. We appreciate this suggestion and will incorporate the findings in the revised version.

$\quad$

## Limitation and future work

>As pointed out by reviewers, we are missing a discussion about limitations and future works. This is because of the page limit. We are going to add the following discussion in the revised version:

>“We focused on foreign key constraints in this work, and made the assumption that child rows are conditionally independent given corresponding parent rows. This brings three natural follow-up research directions: i) extension to the scenarios where this prior information is not available and these relationships need to be discovered first[3], ii) further relaxing the assumptions, and iii) inspecting multi-relational data synthesis with other integrity constraints (e.g, denial constraints[4], general assertions for business rules). 
Furthermore, we evaluated ClavaDDPM's privacy with the common (in tabular data literature) DCR metric. Nonetheless, we think it is worthwhile to: i) evaluate the resiliency of ClavaDDPM against stronger privacy attacks[5], and ii) investigate the efficacy of boosting ClavaDDPM with privacy guarantees such as differential privacy. Similarly,  the impacts of our design on fairness and bias removal, as another motivating pillar in synthetic data generation, is well worth exploring as future work. We believe the thorough multi-relational modeling formulation we presented in this work, can serve as a strong foundation to build private and fair solutions upon. ”

Meta-review: Despite the relatively low mean score of the paper, I recommend the paper for acceptance, albeit with less certainty. The reason for this decision is that one of the reviewers was very positive about the paper and provided a set of strong arguments in favor of the proposed method:

- Important problem: Synthetic data generation is growing in the era of large, data-hungry models whose quality depends on the quality of training data. Moreover, an application of this problem to relational databases.
- Novelty: ClavaDDPM is the first tabular DDPM that supports generation conditioned on multiple tables (organized in several hierarchy levels) and can be used on real-world datasets.
- Innovative: The idea of learning latent variables with classifier guidance for grouped keys is interesting. In addition, the model supports both numerical and categorical data and includes parent-child relationships. 
- Strong empirical results: ClavaDDPM was compared against existing non-diffusion multi-table models (PrivLava and Synthetic Dava Vault), the authors also adapted single-table baselines CTGAN and TabDDPM in several versions as baselines. High-quality ablation studies.
- Well-written paper.

One reviewer (bZ6q) provided a rather shallow review, likely written with the help of an LLM, and responded to the rebuttal with comments that I could not comprehend. The other negative reviewer (5kZH) ghosted the authors after the rebuttal was submitted despite several attempts to obtain a response. In my opinion, the authors provided reasonable responses in their rebuttal to said reviewer. For instance, the reviewer simply mentioned other methods to compare to without explaining why comparing to these methods adds any value to the paper. Hence, I think that these two reviews do not meet the quality standards of NeurIPS. Hence, not taking these into account, one reviewer is very positive about the paper, and one gives it a score of 5. Overall, this points to accepting the paper. I also think that his paper falls into the "new work" category (a new perspective on an important problem), which often receives harsher criticism than papers that fall into more established "mainstream" areas. For all of these reasons, I recommend to accept the paper.
Decision: Accept (poster)