A Bayesian Nonparametric Framework for Private, Fair, and Balanced Tabular Data Synthesis
Authors: Forough Fazeli-Asl
Michael Minyi Zhang
Linglong Kong
Bei Jiang
Venue: ICLR 2026 Poster (2026)
Keywords: Bayesian nonparametric
Dirichlet process
Differential privacy
Tabular data generation
Primary area: generative models
TL;DR: Incorporating privacy as well as fairness through Bayesian nonparametric learning

Abstract: A fundamental challenge in data synthesis is protecting the fairness and privacy of
the individual, particularly in data-scarce environments where underrepresented
groups are at risk of further marginalization by reproducing the biases inherent in
the data modeling process. We introduce a privacy- and fairness-aware for a class
of generative models, which fuses the conditional generator within the framework
of Bayesian nonparametric learning (BNPL). This conditional structure imposes
fairness constraints in our generative model by minimizing the mutual information
between generated outcomes and protected attributes. Unlike existing methods
that primarily focus on sensitive binary-valued attributes, our framework extends
seamlessly to non-binary attributes. Moreover, our method provides a systematic
solution to class imbalance, ensuring adequate representation of underrepresented
protected groups. Our proposed approach offers a scalable, privacy-preserving
framework for ethical and equitable data generation, which we demonstrate by
theoretical guarantees and extensive experiments on sensitive empirical examples.

Published as a conference paper at ICLR 2026
A BAYESIAN NONPARAMETRIC FRAMEWORK FOR PRI- VATE, FAIR, AND BALANCED TABULAR DATA SYNTHE-
SIS
Forough Fazeliasl, Linglong Kong & Bei Jiang Department of Mathematical and Statistical Sciences University of Alberta, AB, Canada [fazelin ias, lkong, be 1 Qua lberta.ca
Michael Minyi Zhang
Department of Statistics and Actuarial Science University of Hong Kong Pok Fu Lam, Hong Kong mzhangl 8@hku. hk
ABSTRACT
A fundamental challenge in data synthesis is protecting the faimess and privacy of the individual, particularly in data-scarce environments where underrepresented groups risk of further marginalization by reproducing the biases inherent in the data modeling process. We introduce a privacy- and fairness-aware generative model, which fuses the conditional generator within the framework of Bayesian nonparametric learning (BNPL). This conditional structure imposes fairness con- straints in our generative model by minimizing the mutual information between generated outcomes and protected attributes. Unlike existing methods that primar- ily focus on sensitive binary-valuce attributes, our framework extends seamlessly to non- non-binary attributes. Moreover, our method provides a systematic solution to class imbalance, ensuring adequate representation of underrepresented protected groups. Our proposed approach offers scalable, vivaacy-reseervng framework for ethical and equitable data generation, which we demonstrate by thcoretical guarantees and extensive experiments on sensitive empirical examples.
INTRODUCTION
Tabular data, which are structured collections of continuous and categorical variables, critical for building multimodal artificial intelligence (AI) systems such as generative image and language models that integrate structured data with text and images. Synthesizing tabular data is partieu- larly challenging when fairness (equal treatment across protected groups), privacy (individual-level protection, typically through differential privacy), and class balance (uniform representation across categorical attributes) constraints must be jointly ensured while preserving utility. Violations in any of these aspects reinforce bias, leak sensitive information, or distort learning through imbalance (Shaham et al. 2025).
Although several works separately address these concerns (Chen et 2024). unifying them remains an open challenge. Among existing methods, most rely solely on generative adversarial networks (GANs) model the data, which risk mode collapse and can thereby exacerbate dlss-diversiity issues. Others rely on variational autoencoders (VAEs), which can better promote diversity but often produce outputs that miss fine-grained patterns. Combining VAEs and GANs could jointly leverage the diversity and fidelity of each method for generating data, yel its application remains underexplored for tabular data.
To unify faimess and privacy within single framework for tabular data generation, we propose a novel supervised Bayesian nonparametric method conditioned on protected attributes, which is flexible enough be applied any model in the class of generator- models (GDMs). We demonstrate its implementation by applying the proposed scheme to generative model which combines VAEs and conditional GANs.
Our model enforces fairness by using a BNP-based mutual information (MI) regularization term, which offers computationally efficient procedure for training the generative model. The MIl term

---

Published as a conference paper at ICLR 2026
facilitates fairness by conditioning on protected attributes and further allows direct control group proportions to ensure balanced representation of minority classes. In particular, a notable advantage our ability to support fair data augmentation for non-binary valued protected attributes.
We enforce privacy protection at two levels using BNPL: (1.) Globally, by using a Dirichlet process (DirP) mechanism, which injects structured uncertainty into the sampling process. (2.) Locally, by constructing a copula-based base measure for the DirP to ensure that continuous and categorical features receive tailored privacy protections at the marginal level while preserving feature dependen- cies. The limitations of standard training methods in privacy preservation make BNPL a promising alternative, as we will discuss in later sections.
Our paper is structured as follows: In Section 2 we briefly provide an overview of fair and private generative networks. In Section 3] we define fairness and privacy in the context of this paper. Later, we present the technical details for our proposed idea in Section 4 Then, we provide experimental results in Section[]Jwhen we will validate our proposed model on real datasets which exhibit sensitive features and demand privacy protection. Finally, we conclude our paper in Section and provide directions for future work. A notational glossary, definition of terms, proofs, additional experiments, and algorithms are provided in the appendix.
BACKGROUND WORK
We position our work in the intersection of privacy preservation and fairness-aware data synthesis. While others have focused on, at most, one aspect of this problem, we seek to fill critical gap in tabular generative models to address both of these issues. Previous work in synthetic generation of tabular data is primarily based on generative adversarial networks (Goodfellow et al., 2014). One notable example the CTGAN, which aims to preserve the modes of the population to ensure high utility by conditioning on one discrete attribute per step (Xu et al.) (019)). However, CTGAN lacks the ability to set category ratios arbitrarily at generation which hinders our ability to fairly balance classes. Moreover, GANs in general do not inherently have mechanism to protect privacy.
In Privacy-preserving tabular data synthesis (Truda 2023] for example), most approaches build on either the differentially private stochastic gradient descent (Song et al., 2013), which can exacerbate class imbalance by disproportionately affecting underrepresented groups Or private aggregation of teacher ensembles methods (Papermot et al. 2017). which faces scalability issues due to its reliance on multiple teacher discriminators. VAE-based methods, like OVAE, introduce a framework with differentiable oblivious decision trees to embed inductive bias (Vardhan & Kok 2020), but also lacks formal privacy guarantees and does not support a tunable privacy budget.
Fairness-awart models like TabFairGAN, DECAF, and FairGAN, for example (Rajabi & Garibay 2022] Van Breugel et al. 2221; Xu et al. 2018), impose fairness constraints through modifications of the GAN architecture. However, these models struggle to handle non-binary protected attributes during training. Currently, there exists some related works which use MI for classifier based fair- ness (Cho et al., 2020; Roh al. 2020), though these models ignore the marginal entropy of the protected attributes, which reduces its capacity to represent the data.
Privacy and fairness interact in ways that make the joint problem more challenging than enforcing either one alone. Fairness relies on accurate group-level statistics, yet privacy noise can distort these estimates, especially for small sensitive groups. the same time, fairness constraints further restrict an optimization process already affected by noise. Consequently, the two constraints each other's 's difficulty. Existing approaches at this intersection include PF-WGAN and PreFair for differentially private fair synthetic data generation (Sarmin et al. 2025
reinforce
Pujol et al. 2023).
Previous methods to remedy unbalanced classes rely on techniques like oversampling of the minority
class (Chawla al., 2002). generating synthetic examples through linear interpolations and generative networks. (He et al. 22088; Chen et al., (024) However, many of these methods are rather simplistic in their design to handle unbalanced data are complex train.
2

---

Published as a conference paper at ICLR 2026
3 FAIRNESS AND PRIVACY
Consider dataset D = (X, Y,S) RR, where X contains unprotected attributes (both continuous and categorical), S E {0, 1} is a binary protected attribute, and Y E {0, 1} is a binary outcome variable. A positive outcome is denoted by Y = 1, and S = 1 indicates membership in protected group (e.g., gender or ethnicity). this paper, we focus on the following concepts of fairness and privacy:
Fairness via Statistical Parity (SP): SP requires the decision Y to be independent of the protected attribute S, i.c., the probability of positive outcome must be the same actoss groups:
Pr(Y 1 S 0) Pr(Y 1 1| 1).
Equivalence via Mutual Information: While SP extends categorical S [K] with K >>
it becomes complex in many generative models due to pairwise comparisons. Instead, SP can be formulated via MI, holding if and only if
MI(Y, S) DKL (Fy,s, Fy @ Fs) H(Y) H(Y 5) O,
with DKL denoting the KL divergence, @ the product of marginals, and H()), (-1)))11 the Shannon and conditional entropy, respectively.
(e, Diffferrenta Privacy (DP) Mechanism: DP ensures that the inclusion or exclusion a single data point does not significantly alter the output distribution of randomized algorithm. A mechanism M x - RT satisfies (e, () diifffrrrell privacy if for any neighboring datasets X, e R" and measurable set E' C RTm
Pr[M(X;e, E E e e Pr[MM(X :, e E] 5
where controls the privacy-utility trade-off, and ō allows small failure probability. M satisfies (e, O)ddifferential privacy, it is referred to as e-differential privacy.
Classical Gaussian Mechanism (CGM): mechanism that satisfies (c, 5)-DP defines the CGM as McGM (x c, 6) = s(X) + Z where s X + R is deterministic function, and ~ NO0 o² ) The noise variance is calibrated using tail bound approximation, leading to the variance o² 24, mm1225/8)) which tends to overestimate the noise, especially in high and low privacy regimes. 2 Here, A, supx.x! [xx s(X')l the global sensitivity of measuring the maximum change in the output of between neighboring inputs.
The AGM improves the CGM by tightening the noise vari- ance calibration for continuous datasets (Balle & Wang 2018). The AGM, denoted by MAGMM ()) improves the CGM by computing the noise variance that satisfies the exact privacy condition:
Analytic Gaussian Mechanism (AGM):
A
< 6, where + is the standard normal cumulative distribution
20
function (CDF). AGM reduces variance, improving utility while maintaining privacy.
mecha- nism used to privatize categorical data (Wang et al. 2016). Let X be a categorical random variable taking values from a discrete set [K]. where K the number of categories. The RRM, denoted by MARM (X;c), perturbs the original value X according to privacy budget which controls the trade-off between privacy and accuracy. The c-differential privacy mechanism given as
Randomized Response Mechanism (RRM): Randomized response
if the true value of X, otherwise.
Pr(MRRM((X;; =x)
PRIVACY AND FAIRNESS PRESERVATION WITH BAYESIAN NONPARAMETRIC LEARNING
Our proposed generative model uses BNPL Fong et al. 2019) method of ensuring privacy and fairness protection by resampling the data from a Dirichlet process (DirP) posterior, which we will first introduce in this section.
3

---

Published as a conference paper at ICLR 2026
4.1 THE DIRICHLET PROCESS
The Dirichlet process, denoted by F ~ (a H), is a distribution over probability mea- sures on measurable space A), where for any measurable partition , the vector (F(At)..... F.......F(.AA). F(Ax)) ) Dir(aH(AA) (.1)).......AAA (Ax)) (Ferguson 1973). The base measure H en- controls strength. As conjugate prior, the posterior again DirP: F (a n1, Hf",. where =1=
codes prior knowledge, and
... P.... and PD.... is the empirical CDF of the sample, D 1:n (X 1.z. I.n. S.,a) To facilitate sampling, we use the finite approximation of the posterior DirP from Ishwaran & Zarepour (2002), avoiding the need truncate the infinite ttick-breaking process (Sethuraman 1994 Zarepour & Al-Labadi 2012
N
F
Din I:N
(
=
flat iN
1 ())
(3)
where (J..--....., ((atm))) Dir((a + n)/N, (a + n))N), and D. IID H Here, )
and t represent weights and locations, respectively. In the subsequent sections, investigate the efficacy of this approximation within a regularization method in BNP generative model. To
ID generate (J(atz) JISiSN, we first sample Ti,N Gamma((a + n)/N,1) independently from D.
and then set M = P E
N Ti,.
Remark To balance accuracy and efficiency in approximating Eq. 3 we adopt the adaptive Pas
truncation rule from Zarepour & Al-Labadi (212)), stopping at inf threshold E (0, 1), when the incremental weight becomes negligible.
{i
<p}
<
for
El Fig
4.2 THE DIRICHLET PROCESS SATISFIES (e, DIFFEREREII PRIVACY
The next proposition shows how the randomized weights in Eq. contribute to the privatization of the distribution; see Appendix G for further clarification. The top panel of Fig. D illlustrates its effect in perturbing the shape of the empirical distribution around the basc measure.
Proposition 1 (Global Privacy Guarantee) For a fixed E (0, and parameters T), 7j (0, 1), let W [N 1] and define min{1,a ²J. Consider two b-adjacent empirical distributions of the resampled dataset D PI:N. given by FDi. -E= PI:N, PLENN E An . N 2 2/b, and there exist indices i, W such that p-(1.) = =P--(a)) L/N, with p/- = -1 (1+e)/N, p/J
and F'D,x =N Ei=1,, where
e)/N, and ((P())) - -( 11 S b. Then, given regularity assumptions (Eg. 3-Eq. l.9.5y 5), the DirP weights, M (pLN) Egio, 6gio) = (Jlat ...... J Jlatt)) ). define a Dirichlet
N........N
mechanism with parameter a a n, meaning that for all measurable sets E c A it satisfies (Egio 8gto)= differential privacy:
Pr[Mattr (atn)) (pi.) Cigtn 801) E EE "' PE(MoNt (a+t))
(PiNiC 80m)) E EJ + dyhn-
The privacy parameters are given by
B(aa + n)n, m)(1 7- 11))
b(a +
In 1-
(1- ([I -1)7
1)y
In
+
B ( 3. (a n)(1 fj 11 2))
2
and
801 1- min Pr (J(otn)
(atn) 1.N- N.N- N.N-1 )
E =2
PHNNPLN
where 22 = fx E 84 2 7, Vi W),
(x RA
EN = 2,,; 0,Vi E [N]]
A 9.5 {PI.N Σvew Pi < 1 T.P. 2 n. Vi W). and B(..) the beta function.
Corollary (Global Perfect Privacy) Under the conditions of Proposition ! as a - cc, we have (i) Egla 0; moreaver, (ii) Egia 2g 0 for fixed [W| 1.
4

---

Published as a conference paper at ICLR 2026
4.3 LOCALIZED PRIVACY VIA COPULA-BASED BASE MEASURE
Proposition (innerent implies a default level of privacy that is always obtained from the weights of the DirP approximation in Eq. 3 However, the locations in the DirP approximation are not inherently protected by this mechanism. To enhance privacy while incorporating regularization, we model the base measure of DirP prior as a copula-based measure. The copula applies suitable privacy mechanism separately to cach continuous and categorical column of the tabular dataset while preserving the dependency structure among variables.
Specifically, let the tabular dataset D Imn have Ne continuous columns and ND discrete columns, with d = Nc + ND. Let D (C)
limit and (DD) Pim,ip be the icth continuous and inth discrete columns with samples, where ic E [No] and iD [ND]] Consider the representation of the dataset as
D 1 ling ERuXNO
E
D
( 1:n.1:
D PR
D I.m.ND -2
((D)
D DR L.n.ND
(4)
Lo
Lm..N-11
Unprotected altributes: X1:0
Outcomes: Yi:
Protected altributes: Si:n
For each ic. consider the sample mean and variance given by d/ n "E= D(C) and oi(( n - (D
Marginal CDF
v-Giobal Privacy
(C)
Pic
HPtt
respectively. Likewise, consider the estimated probability of observ- ing category in D ismiip Pipj "ips, where nipj is the number of samples in category of D ((D) imm.ip- j [o. with Kip being the 'immip" The estimated probability vector Pip (Pip1....., Pip K. with Ej= Pips 1. We then construct the base measure of DirP prior through the following steps:
DirP
a
number of categories in D (b)
Localized Privacy
Gaukkan Conda
RRM
Step 1. Regularizing marginals: We first assign a normal dis- (0)
AGM
tribution (C)
1e = =N o o2(")) and a categorical distribution H!! = Catl(K. Pip) as marginal priors to each (C))) and (!
N DDP Prior Locations
ip respectively, where F((() and (DD) represent the true distributions Figure 1: Privacy Scheme.
of the icth continuous and ipth discrete columns in Eq. 4 These
marginal priors will be used to construct the joint base measure of DirP prior. This step anchors
the sampled distribution around the marginal prior and acts a regularizer on the marginals, thereby robustifying model training.
Furthermore, it prevents prior data conflict (Evans & Moshonov 2006 Mutsvari al. 2016 6 when
the prior contradicts the data- by employing the empirical Bayes framework (Maritz 2018 where
the prior's hyperparameters are estimated from the data and treated separately in inference to stabilize the model. In the BNP framework, empirical Bayes estimates the base measure of the DirP, which serves as its hyperparameter, using data (McAuliffe et al., 2006).
Step 2. Perturbing marginals:
For given flocio" locio" <ivcip let Pri(C)
(C)
((C) (D)
H!C and D Pri(D)
!PP) Using the identity function for s(-) in the AGM, we perturb each prior marginal by (AG Pri(C) (C)
Oeocep FAGM,CC and MRIM (D) "Rucagy
"'uenp ; )
FRRM, respectively.
Step 3. Capturing dependencies between marginals: Then, we estimate the correlation between continuous features using Pearson correlation, denoted by Rcc- For dependencies involving discrete features, use Spearman rank correlation for both discrete-discrete and discrete-continuon pairs, resulting in the total estimated correlation matrix R R RDC, RDD). This estimation process follows the procedure described in Mendes et al. 2023) for tabular dataset. Given t E R" apply R, FAGM.iC FAGMAD in in Eq. 10 (Appendix D to model the joint perturbed base measure using copula-based model Eq. 5
Port (t) C (FAGM, (11), (11),
(IGM,No(NN.).. FRIM, (INe FR (I2))-
(5)
The final two steps ensure that both continuous and categorical attributes receive tailored privacy treatment through the prior marginals while capturing the covariance structure between the variables.
5

---

Published as a conference paper at ICLR 2026
This framework enables localized privacy by allowing marginal privatization only selected continuous and discrete columns indexed by 7(0) S [[Nc] and I(D) S [ND]]
This is achieved by assigning large enough values to "locue" "oocip : and setting ( for all
(C) (D)
èc & I(0), iD & I()), See the bottom panel of Fig. [ for an illustration, and Appendix for the posterior privacy scheme. However, generating samples from H PP is challenging, as It jointly models dependencies across both continuous and discrete marginals. Algorithm I in the Appendix outlines sampling procedure inspired by the latent Gaussian copula approach (Fan et al. 2017).
Proposition 2 Consider Per defined by Eq. 5 and let H be the joint CDF modeled by a semi- Gaussian copula with estimated correlation matrix it and marginals H:()) H IIf 1
and + for all E [Nc]V iD [[ND] [NDD\/ then lim Pet((())) (t) Ht)t] s
(D) (C) + 00, Stocp élocee
Eideries FAAMMsc (tie) - !
(H( (tic) + EiDEID, FRRM, (tNo+in) - H1 (tNotio) More-
over, the unperturbed case where o and Floci Stocp + 30 for all ig E [Nc] iD E [ND]. we have Peert 4 H.
(D)
Proposition (ii) confirms that, when 6/0) 0 and Goe.g Stocip the copula-based base measure imposes no perturbation and solely enhances generalization.
(D)
for all relevant data columns,
Remark 2 Following Steinke (2022) Theorem 3.22). for any > O, the composition of differentially private mechanisms per-continuous attribute privatization (e, 8 ôgto + +0/0)) )-differential private,
where e - min Exto + 'loroo ( + (<lor, )²) + 2l((1))) (22 + (ciocde } A similar
(C)
expression applies per-discrete attribute privatization.
4.4 TABULAR DATA SYNTHESIS
To train our generative model, we first must preprocess the posterior samples drawn from the DirP. Given these samples, they are fed into BNP conditional VAE-GAN network whose details we will describe in this section.
4.4.1 CONDITIONING LEARNING ON PROTECTED ATTRIBUTES
Preprocessing: Given tabular dataset 1:n F with F (a Pert), ). let Deo (XPes I.N YP SPax denote N realizations from the DirP approximation in Eq. corresponding to 1:N DPm pPss The structure P follows the same representation as in Eq. 4 except that N re- places n, and each term carries the superscript "Pos" to indicate that it the posterior distribution. To preprocess the posterior sample for the subsequent network architecture, the continuous columns are
coamiile-rrnsffommme via D 1:N.ic = - (EPox Pox(C)) ). where Pu is the CDF of the standard uniform distribution. This step addresses the multi modal numerical features that hinder GAN training, im- proving stability and sample quality without relying on mixture-based techniques (Rajabi & Garibay
Pox(C)
(ox(D) E [Kip] is encoded as a one-hot vector D P.N..ip E Kippp 2022). Each discrete variable 1-N,ip with at the observed category and 0 elsewhere. The complete transformed dataset is shown in Eq. Hereafter, one-hot representations are marked by a "superscript dot".
Pos(D)
L.N- Volee with total feature dimensicn of =1= No+ E KiD
d'=KND
Pos D
I.N
D Po(()
D
PoN(C) D Brar I.N.No:
DINI..... ..... D D
D Pow(D)
DIN.NN-
D Pow(D) 1:N.ND
(6)
R
Let Gw be neural network generator the class of GMDs, parameterized by w Ω. Given a distance A, robust posterior approximation of w can be obtained by optimizing Eq. 7 to enforce Go to generate realistic tabular samples (Dellaporta et al. 2022):
BNPL:
w" Pos) ) arg min A Pus
Poi FG.).
welt
D DIN

---

Published as a conference paper at ICLR 2026
BNP- VAECAAN::: We use generative model that learns a latent representation by minimizing reconstruction and regularization errors in a VAE composed of an encoder mapping the data to the latent space and a decoder which reconstructs the data (Fazeli-As) & Zhang 2023). The C denotes code-GAN that includes only code-generator and uses MMD as a discriminator to explore underrepresented code regions. The GAN comprises of generator that maps latent variables E IP to synthetic samples. The decoder is then shared with the generator to enable joint optimization.
Architecture: We define our NN-based generator, encoder, code-generator, and discriminator functions conditioned on STA G [Rpxd' R En [R;; xd'2 IP, CG, R9xd2 + RP,
and Dg R dd + IR, with parameters w E Ω 17 E H. T E T. and 0 E O respectively. We train the CBNP- VAECGAN by minimizing the composite objective (Eq. 8) with respect its respective network parameters:
Lumin (Gw, En,, Do. CG+) L(Gw, En Stm)) L(Da Stm)) L(CG.|
(
Pos
The conditional generator is fed with (f1.N. SP to produce synthetic samples X 1.N- Since sensitive attributes are never generated, the framework allows omitting their privacy constraint (eNo x)) in equation 5 preventing the removal of minority sensitive groups during perturbation.
4.4.2 EMPLOYING DIRPMINE AS A FAIRNESS REGULARIZER
The structure of CBNP- VAECGAN flexibly enforces SP fairness by regularizing the loss function
in Eq. 8 with an MI penalty, MI(Ý " to reduce dependence this remains challenging due intractable likelihoods and the curse of dimensionality high- dimensional spaces. The following definition shows that MI can be approximated by maximizing variational form of the DV lower bound (DVLB. Donsker (DVLkB,Donsker Varadhan 1975), derived from the KL representation of MI and claimed as scalable estimator (Fazeliasl et al. 2025 see Figs. 3-4 and Table 1). used here as a proxy for fairness training in high-dimensional settings.
protected attributes. However,
Definition (DirPMINE (Fazeliasl al. 2025)) Let Te Rdd + IR be an NN over compact
Y. Then MI(Y Por S is approximated as follows using random permutation [T[[]] of [N].
(Gar(C
Approsinaaion
Por
L:N, SPN,)
SPu
max VET
jlatal CNN
Pas S
In
jlatn) ( Tolk, ster))
the
The framework manages the once before fairness optimization, and its manually set per-attribute budgets, light for outcomes and none for sensitive attributes, help prevent localized noise from distorting the group-conditioned statistics needed for faimess, especially for small sensitive groups.
fairness tension by applying the localized privacy mechanism
4.4.3 CLASS BALANCING: A POTENTIAL DRIVER OF FAIRNESS
ensures class balance across protected attributes via generator conditioning. To extend balancing to unprotected discrete features ip E WVD=2], we define U.p ~ Cat(Ki,, 1/Kip) Kip
and minimize DKL Pos (D) ,UiD) = Σ pPus In (F Kin)), where p is the generated class proportion. This divergence is estimated via semi-BNP DVLB with one-hot samples ÚI:Na and auxiliary functions To.p KR + R:
Lidase (Gw)
DiPDV Pas(D) DKL (DINN.I ÚBND)) = max
flatn) F.N
Toip
Mos(D) (Dipir
N
Tw a
U.p..
Vip ETip
The final loss, scaled by )r, Apip E [0. 1J. is then given by Eq. eventually reverted by inverse QT for continuous and argmax decoding for categorical columns.
The generator's outputs are
ND-2
Lumiay (Gw, En, Do, CG+) + AeLarr(Gu)
Aud Lisancee (Gw)-
(9)
For CDFs Fx, Fz, and T. Rdim(X)
DVLB F F (x( In FnFFrle Fr To(20) 5 (Fx, Fz).
7

---

Published as a conference paper at ICLR 2026
We provide strong theoretical guarantee that the key Sesull-showing that utility, faimess, and class imbalance are jointly addressed by the method is preserved, as stated Theorem 2 in the Appendix G This guarantee is further validated by the results in Appendix [ Together, these credible findings make the proposed framework clearly distinguishable from other works that lack truly joint treatment of these aspects.
5 EXPERIMENTAL RESULTS
We assess the utility of our proposed model by using MMD and three classifiers (DTS, LR, MLP) (Hastie et al. 2209), each trained on 80% of the original data and validated on the remaining 20% as true baseline, with utility measured using accuracy and Fl scores (Rainio et al., 2024). The classifiers then trained on synthetic data generated by the learned conditional generator with real sensitive ratio inputs (S,r) and tested on the same 20% original, with each score averaged over 10 runs.
Adult Dataset: It includes 6 continuous attributes and discrete attributes with Si consisting of 67.5% males and 32.5% females. (Appendix I.3.Ip. is known for its income (I) bias (>50K vs. <0KK) across sex (S). Here, Y "i "income" and "sex"
0.25
Case 1
Egbs 7,54, Sylo
80%
Case 2
Ept 0.348, 6g0
Case 3
Egi 0.101, 630
Case 4
* Egt 0,008, 690
70%
MMD
0.15
Egts 550.74, Gg
548.6, dgto 0.06
Eglo 0,002, 0ge
Eglo 0,003, 8g0
Score
60%
0.10
Egto = 29.35, 650 0.01
50%
Case I Acc
Case Acc
0.05
Case 1 F1
Case
Original
2 Acc
Case Acc
0.00
30%
Original
Case 2
10
10-2
10-4
10a
10a
DTC
MLP
(a) a 10-6 ,,,,, 10, 50, 100, 500, 1000, 1300
(b) a 100
Figure 2: Privacy investigation under different global and local budgets when Ap 0.
Privacy-Preserving: Four designs considered for different values: (1) No local privacy (Gre = -00,, (2) (2.5, -o0a) privacy on continuous attributes 'A' and 'HPW', (3) the same budget on discrete attributes MS and Ra in addition to the two continuous attributes in Case (four attributes in total), and (4) the same budget on all attributes. Fig. 2a shows global privacy budgets from Proposition 1 Jusing W = 3, r7 = 7 = 0.2, = 0.01 as recommended in Gohari et al. (2021), illustrating trade-off where increasing and the number of privatized attributes reduces utility, validating Corollary [ and Proposition 2 We also note that global privacy based on parameter a refers the proportion of posterior samples generated from Pert because of the mixture form of the posterior base measure H", whereas local privacy refers to the intensity of noise injected into the marginals of the samples generated from Pert Hence, for sufficiently small values of (e.g., a = 10 10-"), even strong local privacy budget does not lead to meaningful perturbation of the marginals, effectively leading a clean model (see Appendix HH. This clearly shows how our privacy mechanism maintains good utility for small values of even under strong local privacy budget. Fig. 2b that adding more privatized attributes under high global privacy (a 100) leads to worse accuracy and Fl.
Fairness-Aware: Given a = 5 for light global privacy budget (29.35, 0.01), experiments now pro-
ceed with Ap for Case 4, comparing ours with FairGAN (Xu et al. 2018). DECAF and Tab
GAN. Fairness in ours is evaluated using SP Pr(Y > 50K S Male) Pr(Y 50K S
Female)], and MI Spa)). In others, S is replaced with the corresponding synthesis S. Fig. 3 shows that ours outperforms others in faithfulness and fairness, particularly the sfate-of-the. DECAF, while being a strong competitor with imposed uncertainty.
8

---

Published as a conference paper at ICLR 2026
MMD
SP
80%
DECAF FairGAN TabFair ours
0.0012
0.0014
0.2080
0.0010
0.0258
0.0000
70%
50%
0.0023
0.0051
0.1832
0.3730
0.0214
0.1022
Original
0.0301
0.0510
0.1780
0.0421
0.0201
0.0093
20%
Original
FaiGGAN Acc
Ours Acc
- FairGGN F1
0%
Ours F1
TabFairGAN ACc
DECAF Acc
DECAF F1
0.0011
0.0016
0.2000
0.0010
0.0217
0.0006
DTC
MLP
--
Ax-1
(a)
(b) MlTuee (Y, S)=0.0256, SP True SY S),.)_01198
Figure Adult: Fairness-Awar investigation for Case 4 given a over 10 runs.
05
Balanced
60%
ACC 79.8%
FI 58, 58
Fake Unbalanced
Trairing
Male
Percentage 0
0.0027
Balanced
Female
Workclass
Percentage
40% Default-Setting
0%
80. 0%.0%
SENI
SG
SEI
WP
80
30%
20%
50 0%
50.0 50.0%
NM
W MSA
MAS
Marital-Status
10%-
32,5% 72.5%
20. .%.0%
20.0%
MCS
Cat
Cat
Cat
Cat
Cat
Cat
7
0%
Setting 1
Setting 2 Setting 3 Setting
(a)
(b) Sex ratio settings
Figure 4: Adult: Class balanced investigation for Case given and AF = over 10 runs.
Bass-Balnn:ig:
Completing our investigation, Fig. 4a shows category frequencies in real and
synthetic data, averaged across sex ratio settings from Fig. 4b input to the learned conditional generator and balanced on 'W' and 'MS' with 7 and a for Case 4. Despite this balancing. utility and fairness remain strong.
COMPAS Dataset It includes 3 continuous and 15 discrete attributes. As shown in Table 13 and
Eq. 35 (Appendix I.4.3), the score. text attribute exhibits disparities in risk classification across ethnic groups The corresponding MT is 0.0259, underscoring the disparity. Any generative model should aim to minimize these disparities. Here, Y "score_te textt"text"" and "ethnic" have 3 and 8 categories, respectively, making standard fairness baselines inapplicable. This highlights limitation of existing methods and motivates our approach. As shown Table 14 (Appendix I.4.3). several groups (e.g., the Oriental category) appear minorities in the original dataset, increasing the risk of mode mixing. To address this, we set the protected attribute to have equal proportions across all ethnic groups and use it as input to the conditional generator for supervised learning.
For privacy, we jointly apply a (2.5, 0.05)-local (across all attributes) together with a (29.35, 0.01)- global privacy budget, allowing us to evaluate performance under combined privacy constraints.
The conditional distribution of score text across different ethnic categories for the generated samples is shown in Table with orange indicating generation without fairness constraints and blue indicating generation with fairness constraints applied. Comparing the orange part with corresponding values from the original dataset in Table 13 confirms that the model preserves the underlying distribution (i.e., perfect utility).

---

Published as a conference paper at ICLR 2026
Table Pr(Y
"Score Text" S "Ethnic") for the generated samples.
Ethnic Group
Score Text (Ar = 0)
Score Text (Ap = 1)
Low
Medium
High
Low
Medium
High
African-American
0.603
0.251
0.146
0.621
0.276
0.103
Arabic
0.759
0.164
0.077
0,631
0.230
0.139
Asian
0.826
0.116
0.058
0,635
0.264
0.101
Caucasian
0.734
0.185
0.081
0.629
0.234
0.137
Hispanic
0.773
0.162
0.065
0.624
0.253
0.123
Native American
0.673
0.212
0.115
0.618
0.211
0.171
Oriental
0.817
0.121
0.062
0.676
0.214
0.110
Other
0.809
0.143
0.048
0.683
0.202
0.115
As shown red, the generated data exhibits disparities in score text across ethnic groups without fairness policy. For instance, African-Americans are classified as "High" risk with probabil- ity 0.146, compared to only 0.058 for Asians. These probabilities become nearly equal in the blue portions.
Table 2: COMPAS: Ml (Y,S) 0.0259,
Fl 0.913, and AcCTrue 0.901 over 10 runs.
An =
Ap
MI
MMD
F1 (DTC)
Ace (DTC)
None
0
0.0263
0.0011
0.921 0.903
0.894
0.0018
0.886
Sex
Marital
1
le -
5
0.0024
0.917
0.883
Language
Table 2 analyzes model performance under fairness policies. The first block reports the proposed model without fairness and its improvement when faimess drives MI near zero. The second block activates three categorical variables (Sex, Marital, Language) to balance generation across all their categories. In all cases, MMD between generated and original samples remains negligible, indicating strong data utility, and F1 and Accuracy on the generated data stay close to their values on the original data, confirming that predictive performance is preserved.
Despite the widespread use of the Adult and COMPAS datasets in algorithmic fairness research. recent work has noted limitations in relying on them for evaluating faimesss guarantees (Bao et al. 2021: Ding et al., 202T); see the Appendix for brief review. To address this, we include additional experiments in the Appendix using the Bank Marketing dataset. We also provide controlled toy example that enables a more isolated analysis of specific components of the proposed approach.
6 CONCLUDING REMARKS
We propose a conditional BNPL framework for private, fair, and balanced tabular data synthesis. By conditioning on sensitive attributes, we can effectively address challenges in training generative models, like preserving faithfulness and preventing mode collapse, with theoretical guarantees validated through numerical results. Our fairness term is based on regular MI, the same structure can be extended to conditional MI, allowing compatibility with fairesss notions such as equalized odds, equality of opportunity, predictive equality, and conditional SP, though several these are inherently classifier-oriented rather than generator-oriented. Although the basic structure of the GAN different from large language models (LLMs) like ChatGPT, we believe that our proposed approach can still inspire private and fair language models. In future work, we plan extend this framework by incorporating Dirichlet processes into LLMs, as a mechanism to provide stronger privacy and fairness guarantees.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
Bei Jiang and Linglong Kong were partially supported by grants from the Canada CIFAR AI Chairs program, the Alberta Machine Intelligence Institute (AMII), and Natural Sciences and Engineering Council of Canada (NSERC), and Linglong Kong was also partially supported by grants from the Canada Research Chair program from NSERC. Michael Zhang was supported by the University of Hong Kong Seed Fund for PI Research #2402101367. The authors would also like to thank the anonymous reviewers for their constructive comments that improved the quality of this article.
REFERENCES
Borja Balle and Yu-Xiang Wang. Improving the Gaussian mechanism for differential privacy: Analytical calibration and optimal denoising. In International Conference on Machine Learming, pp. 394-403. PMLR, 2018.
Michelle Bao, Angela Zhou, Samantha Zottola, Brian Brubach, Sarah Desmarais, Aaron Horowitz, Kristian Lum, and Suresh Venkatasubramanian. It's COMPASlicated: The messy relationship between RAI datasets and algorithmic fairness benchmarks. arXiv preprint arXiv:2106.05498, 2021.
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. Mutual information neural estimation. In International Conference on Machine Learning. pp. 531-540. PMLR, 2018.
Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Inteligence Research, 16:321-357, 2002.
Qiuling Chen, Ayong Ye, Yuexin Zhang, Jianwei Chen, and Chuan Huang. An intra-class distribution- focused generative adversarial network approach for imbalanced tabular data learning. International Journal of Machine Learning and Cybernetics, 15(7):2551-22 2572, 2024.
Xiaohong Chen, Yanqin Fan, and Viktor Tsyrennikov. Efficient estimation of semiparametric multivariate copula models. Journal of the American Statistical Association, 101(475):1228 1240, 2006.
Jaewoong Cho, Gyeongjo Hwang, and Changho Suh. A fair classifier using mutual information. In 2020 IEEE International Symposium on Information Theory, pp. 2521-2526. IEEE, 2020.
Gabriel Della Maggiora, Luis Alberto Croquevielle, Nikita Deshpande, Harry Horsley, Thomas Heinis, and Artur Yakimovich. Conditional variational diffusion models. The Twelfth International Conference Learning Representations, 2024.
Charita Dellaporta, Jeremias Knoblauch, Theodoros Damoulas, and Trançois-Xavier Briol. Robust Bayesian inference for simulator-basee models via the MMD posterior bootstrap. International Conference Artificial Intelligence and Statistics, pp. 943-970. PMLR, 2022.
Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt, Retiring adult: New datasets for fair machine learning. Advances in Neural Information Processing Systems, 34:6478-6490. 2021.
Monroe D Donsker and SR Srinivasa Varadhan. Asymptotic evaluation of certain Markov process expectations for large time, Communications on Pure and Applied Mathematics, 28(1):1-47, 1975.
Michael Evans and Hadas Moshonov. Checking for prior-data conflict. 2006.
Jianqing Fan, Han Liu, Yang Ning, and Hui Zou. High dimensional semiparametric latent graphical model for mixed data. Journal of the Royal Statisticul Society Series B: Statistical Methodology, 79(2):405-421, 2017.
Tom Fawcett. An introduction to ROC analysis. Pattern Recognition Letters, 27(8):861-874 2006.
11

---

Published as a conference paper at ICLR 2026
Forough Fazeli-Ast and Michael Minyi Zhang. A Bayesian non-parametric approach to generative models: Integrating variational autoencoder and generative adversarial networks using Wasserstein and maximum mean discrepancy. arXiv preprint arXiv:2308. 14048, 2023.
Forough Fazeli-Asl, Michael Minyi Zhang, and Lizhen Lin. semi-Bayesian nonparametric hypoth- esis test using maximum mean discrepancy with applications in generative adversarial networks. arXiv preprint arXiv:2303.02637, 2023.
Forough Fazeliasl, Michael Minyi Zhang, Bei Jiang, and Linglong Kong. A deep Bayesian nonpara- metric framework for robust mutual information estimation. arXiv preprint (xXiv:250300800) 2025.
Thomas S Ferguson. A Bayesian analysis of some nonparametric problems. Annals of Statistics, 1 (2):209-230, 1973.
Edwin Fong, Simon Lyddon, and Chris Holmes. Scalable nonparametric sampling from multimodal posteriors with the posterior bootstrap. In International Conference on Machine Learning, pp. 1952-1962. PMLR, 2019.
Parham Gohari, Bo Wu, Calvin Hawkins, Matthew Hale, and Ufuk Topeu. Differential privacy on the unit simplex via the Dirichlet mechanism. IEEE Transactions Information Forensics and Security, 16:2326-23444 2021.
Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in Neural Information Processing Systems, 27:2672 26800, 2014.
Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf. and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13(1):723-773,22 2012.
Trevor Hastie, Robert Tibshirani, and Jerome H Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, volume Springer, 2009.
Haibo He, Yang Bai, Edwardo Garcia, and Shutao Li. ADASYN: Adaptive synthetic sampling approach for imbalanced learning. In 2008 IEEE International Joint Conference Neural Networks, pp. 1322-1328. IEEE, 2008
Hemant Ishwaran and Mahmoud Zarepour. Exact and approximate sum representations for the Dirichlet process. Canadian Journal of Statistics, 30(2):269-283, 2002.
Harry Joe. Multivariate Models and Multivariate Dependence Concepts. CRC press, 1997.
Diederik P Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, and Max Welling. Improved variational inference with inverse autoregressive flow. Advances in Neural Information Processing Systems, 29, 2016.
Diederik P Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. Advances in Neural Information Processing Systems, 34:21696-21707, 2021.
Gihyun Kwon, Chihye Han, and Dae-Shit Kim. Generation of 3D brain MRI using auto auto-encoding generative adversarial networks. In Medical Image Computing and Computer Assisted Intervention- MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13-17, 2019, Proceed- ings, Part III 22, pp. 118-126. Springer, 2019.
Yann LeCun. The MNIST database of handwritten digits. http://yanm. lecun. com/exdb/nmist/ 1998.
Xinjian Luo, Yangfan Jiang, Fei Wei, Yuncheng Wu, Xiaokui Xiao, and Beng Chin Ooi. Explor- ing privacy and fairness risks in sharing diffusion models: An adversarial perspective. IEEE Transactions on Information Forensics and Security, 2024.
Johannes S Maritz, Empirical Bayes Methods with Applications. Chapman and Hall/CRC, 2018.
Jon D McAuliffe, David M Blei, and Michael Jordan. Nonparametric empirical Bayes for the Dirichlet process mixture model. Statistics and Computing, 16:5-14, 2006.
12

---

Published as a conference paper at ICLR 2026
José Mendes, Tania Pereira, Francisco Silva, Julieta Frade, Joana Morgado, Cláudia Freitas, Eduardo Negrão, Beatriz Flor de Lima, Miguel Correia da Silva, António J Madureira, et al. Lung CT image synthesis using GANs. Expert Systems with Applications, 215:1 115:119350, 2023.
Timothy Mutsvari, Dominique Tytgat, and Rosalind Walley, Addressing potential prior-data conflict when using informative priors in proof-of-conee studies. Pharmaceutical statistics, 15(1):28 36, 2016.
Roger B Nelsen. An Introduction to Copulas. Springer, 2006.
Nicolas Papernot, Martín Abadi, Ülfar Erlingsson, Ian Goodfellow, and Kunal Talwar. Semi- supervised knowledge transfer for deep learning from private training data. International Conference Learning Representations, 2017.
David Pujol, Amir Gilad, and Ashwin Machanavajjhala PreFair: Privately generating justifiably fair synthetic data. Proceedings of the VLDB Endowment, 16(6):1573-1586 2023.
Oona Rainio, Jarmo Teuho, and Riku Klén. Evaluation metrics and statistical tests for machine learning. Scientific Reports, 14(1):6086, 2024.
Amirarsalan Rajabi and Ozlem Ozmen Garibay, TabFairGAN: Fair tabular data generation with generative adversarial networks. Machine Learning and Knowledge Extraction, 4(2):488-501, 2022.
Yuji Roh, Kangwook Lee, Steven Whang, and Changho Suh. FR-Train: A mutual information- based approach fair and robust training. In International Conference Machine Learning, pp. 8147-8157. PMLR, 2020.
Fatima J Sarmin, Atiquer R Rahman, Christopher J Henry, and Noman Mohammed. Privacy- preserving fair synthetic tabular data, arXiv preprint arXiv:2503..m 02968, 2025.
Jayaram Sethuraman. A constructive definition Dirichlet priors. Statistica Sinica, pp. 639-650,
1994.
Sina Shaham, Arash Hajisafi, Minh K Quan, Dinh C Nguyen, Bhaskar Krishnamachari, Charith Peris, Gabriel Ghinita, Cyrus Shahabi, and Pubudu N Pathirana. Privacy and fairness in machine learning: A survey. IEEE Transactions on Artificial Intelligence, 2025.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy, pp. 3-18.
IEEE, 2017.
M Sklar. Fonctions de répartition à n dimensions et leurs marges. In Annales de I'ISUP, volume pp. 229-231, 1959.
Shuang Song, Kamalika Chaudhuri, and Anand Sarwate. Stochastic gradient descent with differen- tially private updates. In 2013 IEEE Global Conference on Signal and Information Processing, pp. 245-248. IEEE, 2013.
Thomas Steinke. Composition differential privacy privacy amplification by subsampling. arXiv preprint arXiv:2210.00597, 2022.
Gianluca Truda. Generating tabular datasets under differential privacy. arXiv preprint arXiv:2308. 14784, 2023.
Boris Van Breugel, Trent Kyono, Jeroen Berrevoets, and Mihaela Van der Schaar. DECAF: Generating fair synthetic data using causally-aware generative networks. Advances Neural Information Processing Systems, 34:22221 22222222112233,,,.22222222222333 2021.
I. Vivek Harsha Vardhan and Stanley Kok. Generating oblivious variational autoencoders. Proceedings of the Workshop on Economics of Privacy and Data Labor at the 37th International Conference on Machine Learning, 2020.
synthetic tabular data using
Cédric Villani. Optimal transport: Old and new. 338, 2009.
13

---

Published as a conference paper at ICLR 2026
Yue Wang. Xintao Wu, and Donghui Hu. Using randomized response for differential privacy preserving data collection. In EDBT/ICTT Workshops, volume 1558, pp. 0090-6778, 2016.
Depeng Xu, Shuhan Yuan, Lu Zhang, and Xintao Wu. FairGAN: Fairness-aware generative adversar- ial networks. In IEEE International Conference Big Data, pp. 570-575, IEEE, 2018.
Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni. Modeling tabular data using conditional GAN. Advances in Neural Information Processing Systems, 32, 2019.
Mahmoud Zarepour and Luai Al-Labadi. On rapid simulation of the Dirichlet process. Statistics & Probability Letters, 82(5):916- 9-244, 2012.
Jingwei Zhang. Han Shi, Jincheng Yu, Enze Xie, and Zhenguo Li. DiffFlow: A unified SDE framework for score-based diffusion models and generative adversarial networks. arXiv preprint arXiv:2307.02159. 2023.
A NOTATION
Numbers and Arrays
a
A scalar (integer or real)
a
A vector
In.
Identity matrix with rows and columns
scalar random variable
X
A vector-valued random variable
X1.7
A matrix containing realizations of the random vector
X (for a d-dimensional X, the matrix has n rows and d columns).
Ximni
Column X
Columns through m of X1:n-
Sets and Graphs
I
The set of real numbers
Rd
The set of all d-dimensional real- valued vectors.
RaxNe
The set of all n x Ne matrices with entries in
Z+
The set of non-negative integers.
2nxND
ND
The set of all Np matrices with entries in Z..
[
The cardinality of the set W. representing the number of elements in W.
(1,..., N)
The containing I,...,N. N.
[N]
The set of all integers between 1 and N, i.e., (1,..., N).
la, 6]
The real interval including a and b
(a, b)
The real interval excluding but including
A/B
Set subtraction, i.e., the set containing the elements of that are not in B
Indexing
14

---

Published as a conference paper at ICLR 2026
as
Element of vector a, with indexing starting at
Aij
Element i, of matrix A
dim(A)
The dimensions of the matrix A, typically given as (number of rows x number of columns).
(D)
The categorical-valued columns of A.
(C)
The continuous-valun columns A.
ND
The number of categorical-valued columns matrix A.
Nc
The number of continuous-valuee columns in matrix A.
Kip
The number of categories in the ip-th categorical-valued column of A, where iD E (I,...,ND
Kig
The number of categories in the ic-th continuous-valure column of A, where ic E (1....,Nch.
(Eglo, dglo)
Global privacy parameters of the Dirichlet Process mecha- nism applied to the entire underlying data distribution.
(loci bloe)
(C)
Localized privacy parameters of a differentially private mechanism applied to the ic-th continuous column.
Goopd
Localized privacy parameters of a differentially private mechanism applied to the in-th continuous column.
Á
One-hot matrix. For categorical-valud vector a (a1...., an). the one-hot representation is matrix whose i- th row is one-hot vector the form (o..... O, 1, 0,... 0]. with single indicating the active category of a, and Os elsewhere.
Ä
The one-hot-encoded columns of the matrix generated match the one-hot training matrix Å, produced by genera- tor function.
The transfored version of the continuous-valued matrix/vector A.
A tabular- valued matrix containing mix of categorical and continuous variables, where indicates the one-hot encoded categorical columns and """ iindicates the quantile- transformed continuous columns.
The generated tabular matrix, produced by a generator func- tion, matching the tabular training matrix Å.
Ab
The column-wise concatenation of the tabular matrix Á with the one-hot matrix B.
S
Randomized sensitive attribute with a sample size equal to the mini-batch, constructed so that proportion of each label matches that of the true sensitive attribute S. Used as the conditional input to the generator in the experimental section.
Calculus
15

---

Published as a conference paper at ICLR 2026
V
Gradient with respect to æ
fx)
Definite integral over the entire domain of a:
[ [x))d
Definite integral with respect to æ over the set S
Probability and Information Theory
F(A)
A probability set A under F. variable Random variable X has distribution F
X~ F
(x(A)
Indicator function taking value 1 if X A and otherwise Empirical distribution based on the sample X Dirichlet Process prior, with a > 0 the concentration param- and H the base measure.
Fx
DP(a, H)
pPr
A random probability measure drawn from a Dirichlet Pro- prior, i.e., Fpri == (F (a H)).
Pos
A random probability measure drawn from a Dirichlet Pro- cess posterior, i.e., Pos = (F DP + H²)).
H*
Mixture probability measure combining the prior base mea- sure H and the empirical distribution Fx.... given by H a H a + Expectation of h(x) with respect to F(x)
a a+10 Fx...
71
[r[h(x]] or Eh(x)
Varp(h(2))
Variance of h(x) under F(x)
dw(P.Q)
Wasserstein distance between two probability measures and Q
(P,
Maximum mean discrepancy distance between proba- bility measures P and Q
(P,(P,
Kullback-Leibler divergence between two probability mea- sures P and Q
A(P,Q)
A statistical distance between P and Q.
Dir(an,...,,aw aNN)
Dirichlet distribution with parameter (an, -........ O.N)
N(,, E)
Gaussian distribution with mean μa and covariance Σ
Cat(K,p)
Categorical distribution over K categories with probability vector p (PI..... Pj. P pP)) Each component Py de- notes the probability of observing category j among the K possible categories of the categorical variable
Almost sure convergence
Convergence in probability
4
Convergence in distribution
Functions
F:A+B
The function f with domain & and range
f(x: 0)
A function of æ parametrized by 0. (Sometimes we write f(x) and omit the argument 0 to lighten notation)
16

---

Published as a conference paper at ICLR 2026
In æ
Natural logarithm of æ:
||>||p ||=||
1/ norm of a L norm of
KRBF (X, Y)
Radial Basis Function (RBF) kernel: ARBF(X, Y)
exp( - - - where y is the kernel bandwidth parameter.
Network Architecture
G
A generator function parameterized by a neural network
with parameters E
g
A generator input (latent noise).
T- Pos
XY
A generator output (generated posterior sample).
En
An encoder function parameterized by neural network with parameters E H.
An encoder output.
CG+
A code-generator function parameterized by neural net-
work with parameters 7 E T.
A code-generator input (sub-laten) noise).
ē
code-generator output (generated code).
De
A discriminator function parameterized by neural network with parameters O
Tu
A continuous function parameterized by neural network with parameters Y.
Acronyms
AGM
Analytic Gaussian Mechanism
BNPL
Bayesian Nonparametric Learning
CBNP-VAECGAN
Conditional Bayesian Nonparametric framework with triple generative model including a VAE, Code-GAN, and GAN
CDF
Cumulated Distribution Function
CGM
Classical Gaussian Mechanism
DirP
Dirichlet Process
DirPDV
Dirichlet Process approximation of Donsker- Varadhan rep- resentation of Kullback-Leibler Divergence
DirMINE
Dirichlet Process Mutual Information Neural Estimation
DTS
Decision Trees classifier
DV
Donsker- Varadhan representation of Kullback-Leible Di- vergence
DVLB
Donsker- Varadhan Lower Bound of Kullback-Leibler Di- vergence
GAN
Generative Adversarial Network
17

---

Published as a conference paper at ICLR 2026
GDM
Generator--eccodrr-aase Models
II
Independent and identically distributed. Indicates that ran- dom variables share the same distribution and are mutually independent.
KL
Kullback-Leibler Divergence
LLM
Large Language Model
LR
Logistic Regression classifier
MI
Mutual Information
MLP
Multilayer Perceptron classifier
NN
Neural Network
QT
Quantile- Trantill-TTansfommed representation of continuous vari- ables (maps data to a uniform or normal distribution using their empirical quantiles).
Semi-BNP DVLB
A semi Bayesian nonparametric representation the DVLB of the KL divergence, where one distribution in the KL is unknown and the other is known
SP
Statistical Parity
VAE
Variational Autoencoder
WMMD
Combined Wasserstein and Maximum Mean Discrepancy (MMD) distance
B REVIEW OF KEY CONTRIBUTIONS
We begin our appendix discussion by noting that the contributions of this pa- per are novel and not derived from Fazeli-Ast & Zhang (2023). which pro- posed VAECGAN under BNPL framework focused solely on improving gen- eration accuracy, without addressing privacy or fairness.
contrast, our work
introduces a private-fuir--allneed generative framework that is broadly applicable to any model
within the class of Generator-Decodder- Decoder-based Models (GDMs) is, generative models that
include at least generator or decoder component-within the BNPL setting by conditioning
sensitive attributes
This class encompasses wide range of models, including GANs (Dellaporta
et al.
wwon et al. Fazeli-Asl & Zhang 2023). Diffusion- GAN hybrids (DiffGANs) (Zhang al. 2023). and
2022 Fazeli-Asl et al. 2023). VAEs (Kingma et al. 2016), VAEGANs
2019
Diffusion VAE models (Della Maggiora et al. 2024 Kingma et al. 2021). Over this class of GDMs, our contributions be summarized follows:
1. Privacy via DirP:
We are the first to introduce the Dirichlet Process (DirP) (e, 6)- differential privacy mechanism within the BNPL setting. We provide theoretical guarantees for privacy budget control through perturbation of DirP samples. In addition to global privacy, novel local privacy mechanism is proposed by modeling the base measure of the DirP using copula models, allowing structured perturbation the distributional level. None of these privacy mechanisms were introduced or discussed in Fazeli-Asi & Zhang (2023)
2. Fairness via Conditional Generator: We propose a conditional generation strategy that minimizes the DirPMINE between generated outputs and sensitive attributes. Using DirP. MINE provides scalable and effective faimess mechanism applicable to any sensitive attribute with more than two groups, aspect not addressed in the existing literature.
18

---

Published as a conference paper at ICLR 2026
3. Class Balancing via DirPKL:
We introduce class-balancing procedure using DirP-based KL estimator. This supports both fairness and distributional privacy by preventing dispro- portionate representation across groups, another contribution not present in the literature.
We emphasize that the main contribution of this paper is not to provide a comprehensive comparison across members of GDMs, as such comparisons are already widely addressed in the literature, primarily for utility benchmarking, and could be revisited focus of this work is developing a principled approach to jointly enforce privacy categorical and continuous features in tabular data while simultaneously preserving fairness and class balance. Here, class balance is treated as complementary notion of fairesss, reinforcing the fairesss achieved through conditional generation with respect to sensitive attributes.
dedicated survey study. Instead, the
a We chose implement this triple policy approach using the VAECGAN architecture from Fazeli-Asi & Zhang (0233) to demonstrate improved utility and reduced mode collapse, but the methodology itself is general and original.
the proposed triple-policy (fair privatppivteeeee pwiiaat--allanneeee balanced) framework succeeds, practitioners could flexibly adopt any desired differential privacy mechanism for the categorical and continuous features of tabular dataset though their relevant generative model GDMs within our general framework.
We also note that the experimental results reported in this paper include only those baselines most closely aligned with our objectives-namely methods for tabular data generation that incorporate statistical parity-based fairness within VAE-, GAN-, hy hybrid-based architectures. Among these, FairGAN (Xu et al. 2018), DECAF (Van Breugel et al. 2021), and TabFairGAN (Rajabi & Garibay 2022) are the most representative of this intersection, and their inclusion underscores our positioning as next-step contribution in this line of research.
C CBNP-VAECGAN Loss TERMS
Given di= No + Eip and dz KND the NN-based generator, encoder, code-generator, and discriminator functions conditioned on I:N as Go Rpxd' - Rd,, Fry R x - RP, CG, [Rxxd; + R?, and Dg Rdi + R, with parameters Sn n E H. T E T, and 0 E O, respectively.
The code space is defined C1.N = F.,, Y Pos I:N, $ (:N): and the generated codes denoted as
ČI.N := CG+(S > with sub-latent noise ÇI.N (O, Ig)- Given latent vectors ÉI.N ~ (O,, Ip), where p and considering L1N = (ÉL:N, CL:N, CI:N) ), train CBNP- NAMP-VA by minimizing the composite objective Eq. over relevant parameter spaces.
Recall the general utility loss function of VARCGAN in 4.4.1
Laumy (Gu, En, Do, CG+) L(Gw, E.j ) L(Do (0)) + LICG+15
Pio XY
L(Gw, Fry |
1
Do Do (C, APr ). SP
MMD PPus to Foo 8p
N
Posterice reconstracion T2.2
I1.1
Loss com- ponents are defined as:
+ MMD(F FeLNY
Regularization erior: T5
C(Do (tm) =
1
(1, (e., S. - 30(()))) (XY 1IN,, 3Par]] LeuL
N
N
[(CG+I$ = MMD Fe..).
The terms Z1,1 + Z1,2 and J correspond to minimizing the combined distance
dwMMDD
Pos
-FGu )
Xy 1S
19

---

Published as a conference paper at ICLR 2026
The gradient penalty, scaled by Aop is defined
LG
-. m Dg XY, Ill 1)²
XY,
and enforces the I-Lipschitz constraint stabilize discriminator training, where
Pos
Pos
+
with u E [0, 1]. See Diagram 21 in Appendix K for the model structure.
D COPULA-BASED MODEL
Copula models are widely used to describe multivariate distributions Sklar 1959). A d-dimensional
copula is nondecreasing, right-continoos function [0, [] [0. that is grounded, has
margins, and is d-increasing, making it uniformly continuous on [0. 1 Nelsen (2006). Sklar's
theorem establishes the link between copulas and multivariate distributions:
Theorem (Sklar's theorem (Sklar 1959)) Let F be a d-variate distribution function with marginals Fi,....FF Fd. There exists d-copula C such that for all t R", F(t1...... (Fiitt)....,Fa(tt))
If the marginals are continuous, is unique and given by C(u)= = (
( (ua)).
where u, E [0, (u,) = inf{t E IR (,(t) 2 u. and IR" is d product of the extended real line [o). Otherwise, C uniquely determined on the range of the marginals.
1997 A well- Copulas capture the dependence structure of multivariate distributions Joe
known example is the Gaussian copula. Let & R be the CDF of the multivariate normal distri- bution with zero mean and correlation matrix R. The family of Gaussian copulas is CR(U) *R( (u)),.. & 2- (ua)), for u E [o, 1jd. Following Chen et al. Chen et al. (206)). the multivari- ate distribution F is semi-parametric Gaussian copula model with unknown correlation matrix R, taking the form
F(h,...., = (
(10)
E DEFINITION OF MMD AND WASSERSTEIN DISTANCES: DIRP AND EMPIRICAL REPRESENTATIONS
Selecting an appropriate statistical distance crucial for effective generative model training. Here, we focus on the DirP representation of two popular distances used BNP deep learning and we will briefly mention their frequentist counterparts.
E. MAXIMUM MEAN DISCREPANCY DISTANCE (FEATURE-MATCHING COMPARISON)
The MMD distance was initially introduced in Gretton et al. comparisons. Recently, a DirP- based version of this distance has been proposed for BNP hypothesis testing Fazeli-Asl Asl et al. (2023). Consider set of functions (Gw, Twen parameterized by an NN that can generate n IID random variables (Yım), where the likelihood function is intractable and not
(201)) for frequentist two-sample
IID accessible. Given kernel function k(:-)) satisfying Assumption and sample (Di.n) F, the MMD distance between Poss and Fy,.. is approximated as:
MMD (
Pos DRoy PY..))
Σ
(lann) ((atn) N-N 1
N
2
flatal D(D Y.) + n2 rit=l
Σ NY Y.). (II)
T.N-
20

---

Published as a conference paper at ICLR 2026
In the frequentist version of the MMD distance, defined in Gretton al. (2012). the empirical
distribution F.D... is considered. This is denoted as MMD' (F F Y...> which is obtained by replacing N, J x(a+n)) I.N.N-1- and D
1:N with n, 1/n, and D I: respectively, in Eq.
Assumption 1. The kernel k(:-)) continuous and defined on a compact metric space with feature space in universal reproducing kernel Hilbert space (RKHS). Additionally, [k((, < K for all t, E R
Remark The Radial Basis Function (RBF) kernel, denoted by (RBP(;)), satisfies Assumption and is therefore used throughout the implementation in this paper, with the same parameter settings as given in Fazeli-Asl & Zhang (2023).
E.2 WASSERSTEIN DISTANCE (OVERALL DISTRIBUTION COMPARISON)
The frequentist version of the Wasserstein distance is completely discussed in Viillanj 2009 Part 16). Fazeli-Asi et al. Fazeli-Asl & Zhang (0223) proposed BNP version of this distance through Rubinstein its Kaatorooich-Ruumnseenn dual representation. Let (Dee be parametrized family of contin- uous functions that all are I-Lipschitz. Then the Wasserstein distance between PPos and Fy, km is approximated as:
N Σ
(D (Y.).)
W M DiN
FY max
(12)
By utilizing modifications in Eq. 12 similar those described for the MMD distance, the empirical representation of the Wasserstein distance can also be obtained.
Assumption 2. There exists a constant M such that for any n, E I.
max (W x F XY...IS... iS... ) W
,H",,, W H²)) < M,
(13)
XY 6m
where H" denotes the conditional df- dd-variate CDF derived from the full dj dz -ariate base measure associated with D Pos' conditioning on the last d'2 marginals. Moreover,
lim W(F s F X XxYI. S1.. )=
lim W(F WOP N.N-o0 YYINI
,HH)) lim W(F x H*1) 0. =20
10000
F DIRICHLET MECHANISM: ASSUMPTIONS AND DEFINITION GOHARI ET AL. (0211)
REGULARITY ASSUMPTION
Assumption 3. In A a. > 0, f > 0, and + < 1/2.
Assumption 4. For the Dirichlet mechanism M (k) the parameter satisfies
+ n max 1-
Assumption 5. Fix W C [N 11, and, then 7 S W
The assumption in Eq. 5 ensures that the argument of the final logarithm term in the global privacy budget Eglo Proposition M is positive.
Definition 2 A Dirichlet mechanism with parameter (a (a+ +
n)
R++ denoted by
M(atr) (pi:N 6gto), oulputs x E AN according to the Dirichlet probability distribution function centered on PL.N, i.e.,
(a+n)pp
N-1 (a-n)p.-1
N-1
(a+n) Pr [Mote (ptons 6gto) = BEN)]
1
Σ
x;
B(la + n)
i=
21

---

Published as a conference paper at ICLR 2026
where
B((a + n)PI:))= = Il, I((a + n)pp)
T T( n) Ei=1 Pi)
is the multi-variate beta fanction.
G THEORETICAL PROOFS
Proof of Proposition 1 Gohuri el al. introduced general mechanism that guarantees differential privacy for vectors on the unit simplex. Proposition identifies Eidenfffes practically relevant realization of that mechanism by showing how it naturally arises in the context of resampling within Bayesian nonparametric frrmaaaoetrccrammeooot.
Let the fixed dataset be
D D= ( Dn).
and consider two size-N resampled vectors,
D.NN
Di+1. D j+h 1...... Dx):
D I:N DID.....DDII,.... Di+ D D;, ",t,,...,,,DD. DN),
where in D'un the observation D; appears twice, while Dj (j # is omitted. The corresponding empirical distributions are
N FDIN = Σ Pe Pel(DD
N Σ t=1
FD
pee (pib
t=1
where the weights satisfy
Pe = for all E [N].
= 3. p==0,
p== tv for $ {t.j}.
These two empirical distributions are defined to be neighbors, as they differ in exactly one resampled observation. For N: 2 2/b, the corresponding weight vectors satisfy
lp - p' = lp: - p9l lpg PD
N < b.
which matches the b-adjacency condition required by the Dirichlet mechanism in Gohari et al. (2027)
Therefore, the randomized weights in equation 3 coincide exactly with the Dirichlet mechanism of Gohari et al. (2027) when applied to the classical resampling weights. This establishes a direct theoretical connection between classical resampling and the DirP resampling scheme in equation In particular, the additional uncertainty introduced by the randomized weights constitutes a differentially private perturbation of standard resampling, with an explicit privacy budget and formal privacy guarantees.
The remaining argument follows directly from (Gohari et al. 2021 Theorem 1) and therefore omitted.
Proof of Corollary Recall that Eglo in Proposition 7 is given by
B ((a + n) (a + n1(1 - fj 1))
b(a +a n)
- ([W) - 1)
Eglo = In
In
((a n 2), (a n)(1 7j 1) 2)
2
?
Iz(a)
To prove (i), utilize the Pata-Gamma relation to rewrite (a)
r [( + n)n))
r [((a + n)) -1)
(a) = In
+In
T((a n)n Iz(a))
[((a+m)( n1 17 11) Iz (a))
(1,1(0)
(1,)(0)
22

---

Published as a conference paper at ICLR 2026
Applying the fundamental theorem of calculus, obtain
(a) In ((a m)n Iz(a)) In r ((a + n)n)
(o+m))+Ta(a)
(t) dt,
(a+n)n
where (t) 25 In Int is the Digamma function.
Since (-) is continuous on {(a + n)n, (a + +n)n + Iz(a]], the mean value theorem for integrals ensures the existence of E {a + nnn, (a + +0))) such that
II, (t (Iz(a).
(2)().))()..
For sufficiently large a, we approximate
1
4 ((a + m)y + I2aa) ~ In ((a + n)m I2(a))
(14)
2(a + n)n + I2(a)
1
In ((( + In 1
(15)
2a²n1
2(a n1) [2(a)
1
In ((a++)))) +
+
(16)
(a + n2)
20
2(a + n)n + I2(a)
c ((a m)m)
+ 2a²n 2(a n)n + I2(a)
(17)
+ nn+
h(a)
Eq. 14follows from the asymptotic expansion of the Digamma function. The value of chosen based on the cutoff a = 1, and to analyze the asymptotic behavior of Eglo as - 0O, we consider
2 1. leading to the substitution b = - in (a) across all equations. Eq. 15 is derived by factorizing (a + n)y inside the logarithms and applying the logarithum product rule. Eq. T6 uses the approximation In(1 + z) se for small x, obtained from the Maclaurin series expansion. Finally, using the asymptotic expansion of the Digamma function again concludes Eq. 17
Considering Eq. 14 Eq. 17 and the continuity of the Digamma function, we obtain
v ((a + m) (a) < (I) 4 ((a n)n) I2(a) h(a)Iz(a).
(18)
Since lim ((a) 0. letting + in all terms of inequality in Eq. 18 yields
lim (a) lim + + ()) a+
(19)
Applying the asymptotic expansion of v-.) in Eq. 19 and simplifying, we obtain
lim [a) x lim [2(a) ((a + n)n)
lim
400
a0+00
a 202n
H lim
a +
lim 1
(Applying I. L'Hôôittll's rule)
2a 4n
2a2n
0.
Similarly, we find that lim (1) 0. Since it evident that lim Iz(a) = 0. the proof of (i) is complete.
To establish (ii), we first demonstrate that
1,N l...., ((atm) P4
1
" N
N
as a oc.
Since the expectation under PPss satisfies Epau (ernaJlla Chebyshev's inequality:
(,N-1)
x for all f1...., N}, apply
23

---

Published as a conference paper at ICLR 2026
Var ( ((atn)
((att) i,N-1
Pr
2
(20)
N
22
for any e > 0. Substituting Var ( (F(ott))
N-1 Na
into Eq. 20
we obtain
( INN
N -
Pr
20
(21)
N
(2 n 1
Taking the limit in Eq.
27 implies that
f(atn) NN
P
N
Since convergence in probability implies convergence in distribution, it follows that
(((+t) INN-
3
(
as + 00. N
By the definition of convergence in distribution, for any Borel set A C A whose boundary (denoted by OA) satisfies
Pr ((1 ( N 5AA) -,
(22)
it holds that
EA) EA)= =Pr ((+ 1 E EA)
lim Pr a0200
flatz) ( -
To clarify the boundary condition, note that 0A is the boundary of the set A, defined as
0A = An
where A is the closure of and A is the closure of the complement of A. The boundary consists of points that be approached both from inside and outside of A. The condition in Eq. 22 ensures that the limiting measure does not concentrate on the boundary of A. This requirement is necessary for the definition of convergence in distribution hold, convergence in distribution is defined in terms of the measure of sets where the limit point lies away from the boundary.
Since the limiting distribution is degenerate at (N.....N)) ). the probability measure is concentrated at this single point, meaning the distribution becomes deterministic in the limit. Therefore, the boundary condition is trivially satisfied. Consequently, the convergence holds for any Borel A C A.
Furthermore, since S2 c. C and, by Assumption S 1. the set Sh., a Borel set whose boundary does not contain the limiting point ( )) with positive probability. Therefore, the convergence equation holds for Sry implying that
Pr ()
1 11 N
E ESN))
1.
Finally, the continuity of the min function completes the proof.
24

---

Published as a conference paper at ICLR 2026
Proof of Proposition 2 Given Patt and H modeled by semi- copula with the estimated correlation matrix R, for t E R with Ne + ND. we have
PH H C&(FAGM1(11)) ..... NA (INc), FRRM,((NN+t)) ND (fa)) C No (tNc), (tNo+1)...., (D) (ta))
1- Lipachitz S
No
ND
FAGMiC - (tio)
+
Σ
(tNo+to))) H:D (tNo-wr)
10=1
in=1
Σ
FAGM.io(tic) 11 (lic) +
Σ
FR (tNe+io) - (t (tNc+iD)
11
icEnC))
iDE
E ioE[NeIVNO))
FAGMicctio)) - (tic))
Σ
PR (tNc+io) - Hi! (tNotio)
iDE[ND]VI
The last inequality follows from the I-Lipschitz, property of copula functions, which ensures that the difference between two copula values is bounded by the sum of the differences between their marginal inputs.
0, FAGM, corresponds to the CDF of an e-differential privacy mechanism. For two differential privacy mechanisms, AGM and RRM, we have
Given Dicce =
FAGMAC S (1((((to) and FRRMAD 4 (P
(C) as Cloii DC, flrcp + oo since increasing the privacy budget reduces the added noise, causing the perturbed marginals to converge to the original CDFs. This completes the proof.
(D)
The following theorem establishes the asymptotic guarantees of the proposed framework. shows that, under mild regularity conditions, the generator converges in probability to the true conditional distribution in the utility-only case and jointly achieves utility, fairness, and balance in the full-loss setting. This result provides rigorous theoretical foundation for the method and highlights its ability to simultaneously address multiple objectives.
Theorem (Consistency and Joint Guarantees) Given choosing ((::)) as dwMaa((;-) in Eq. define
Mdwnaan (Rd) = (P(R"), dwMMD).
Under assumptions in Eg. Eq. as n, N + oo:
(i) If X >BB,, for iD E [ND 2]. and w minimizes the utility-only loss from Eq. 8 then in the well-specified case where
dwMMD F XVIs' FoJO) for some true minimizer w
we have
FG E, F P
(perfect utility),
where X (X.Y Y).
(ūi If Xp ARIP = and の minimizes the full loss in Eq. all target discrepancies are zero for some wa
then in the well-specified case where (utility),
F
2, F P
An((Gwo (L, (e,5))
)) $) S0 (fairness),
(CN,-
DKL (Gwo (e, (2) (()
Uno) 4 (balance),
where
O; = ip,
ot Cip + Kip
25

---

Published as a conference paper at ICLR 2026
denote the column indices (Cip Cipy in the generator output corresponding to the one-hot- representation of the ip-th umprotected attribute in the original data. This block spans Kip = C Cop columns, each being except for a single indicating the active category.
Proof of Theorem 2 When AP AB., 0, the loss function in Eq. reduces the utility loss Luilay defined in Eq. 8 In this case, the optimized value w is determined solely by minimizing Luitty. the fairness and balance constraints have effect. For such w and >> 0, Markov's inequality implies
dwwMDD (fr (F x Fawe))
Pr (WWMMD(F&xe FG..) 2 2()=
(23)
The triangular inequality implies:
dWMMD F "YIs FG S (WMMMD(F&x s D
dwMMD Diu
FG
IT.
(24)
For any w E Ω2 within the context minimizing Lussiity applying the triangular inequality yields:
dwMMD (F& s Fo
S dwMMD( (F
Poo Dijn
) + dwMMD F
FGya)
(Triangle inequality)
s dwMMD
xvis.
Diu ) dwMMD
(PPo Dir
FGw)
(Definition of w
< dwMMD (F &
F F ) dWMMD (Fow
F F
dwMMD F
&
EPos
(Triangle inequality)
22 dWMMD & XY BI. ) dwMMDD (F
XYI. $
F PPs D
}
+ dwMMD F
(Triangle inequality)
< [dWwMDD( s &
F & 8 ISm. + dwwMD (H" F + dwMMD F ''vss)
dWMMD 8 18..." _!!1)
(Triangle inequality)
= ddMMD ( & 1.. + dMMD (F XV i:n IS......
HH))
(dMMD(H" FOs)
+ 2[dw( (Fdw(F&vis) F F& (s.. dw(F is...... HH))
+ H p.oo))
+ dWMMDD (F F F
(Definition of WMMD)
(25)
Since the inequality in Eq. 25 holds for all w E Ω2 it follows that
dwMMDD(F ( FG..) ) S 2 dddMMD (F& & : F& + dMMDD(F (F& .... "..."..... HH')
+ dMmD (H FPos) + 2I + min dwMMMD F F (26)
wEst
26

---

Published as a conference paper at ICLR 2026
Taking the expectation of both sides of Eq. 26 we get:
E (ddWMMD( F & Fo.e)) E ERRVIS
(EFRON (FTMMMD(&&YC)) Fawel))
< 2F xvis
(W(P&VIs) PXY 18.m..11mm )) +2EFRViS (Epro
(W( 11
DiS + +EPRvIS (EpPOr (W
Fxvss)
(27a)
I.N
+ Vn + 4ak
2K
(a+n+ N)K
+21
(27b)
a n
(a 1)N
+ d 2x1s)
(27c)
where Eq. refers to (T), directly derived from the proof of (Fazeli-Asl et al. 2023 and Eq. 27c due to assuming w the true parameter minimizing dWMMD (G '
Lemma 4),
To complete the proof of (i), need to analyze the behavior of the expressions in Eq 27a and Eq. (77 as n and approach infimity. Considering Assumption 2 and then applying the dominated convergence theorem, conclude that all expectations in Eq. 27a converge
n, N co.
Finally, taking the limit as n and N approach infinity on both sides of Eq. 27 yields:
lim IF (dwaaaD( F & FG_s)) < dWMMD dWMMM(Fa... F& 8): n.N-o
which establishes the utility guarantee in the well-specified case.
For proving (ii), when dp λB., 1, the objective Eq. becomes the full loss function:
Np
Luility + Faiir
Lalunnce
(28)
where the learned parameter w minimizes three terms simultaneously. In this case, the well- specified scenario corresponds to the setting where the optimal minimizer w" satisfies the following:
dwMMM (P Po.)
(29a)
MI((G (e.))) 2:0 .)) $) S)
(29b)
DRI((G (L,)) UiD)
(29c)
As result, the utility guarantee
dWMMD(FG_.o F ( 2 0,
(30)
directly follows from Eq. 29a similar to the reasoning in part (i), now in the context of the full loss formulation Eq. 28
To establish the fairness guarantee, we begin with the following inequality:
27

---

Published as a conference paper at ICLR 2026
o (CnD N. ,), (e,(G=-(e,)))) 200 ).
Mi[(G (e, (CN 2)1 MI ((G (e, ((xxx(1))) $1
+ +|MIPV (e, (e, (CND
2) S) -M((M (e, (C (CMD S1
MI (e, (CN) 5) MIDV (L. (e,5)) (CND -
$)
MIP ((G (e, (L, 2: x 5) MIPY (L, 5)) (CCN 2:C ) $)
+ +|MTPV (e, 2.O 2) 5) MI((G (L. ()) -
2.D 2)² $)
< MIDPDV MIIPDYY((GWO (LIN Sp..
(LNN))(CDD ( -2 $1:N
Sos
MIIV (L, ()) ((n 2): $)
(31a)
+ MI((Gwo ((,
-MDDiPPD -MIDiPDN (lIN (f.N. SPos (CND -:p
StPs SI:N
(31b)
+ MIPV MI (C, s dotS) - MIDV (e, (OND
$)
No-
(31c)
+ ((w,)
)) (I(( 2),
(31d)
To conclude the proof it is enough to assess the asymptotical behavior of the right hand side of the inequality in Eq. 31 Applying (Fazeliasl et al. 2025 Theorem 5(i,ii)) yields:
Eq.
31a 0.9. 0, Eq. 31b a.e. O. as,
N - oc.
(32)
To handle Eq. 31e consider triangle inequality:
dwMMD (FMMD(FFG..) Fo Fw.) -S dwMMD (Fa F FXYIs)
+1 dwwMD (FaaDFFxxxx) FxIs FGJ.);
JT
By Eq. 30 and Eq. 29a, we have J1 E. 0 and J2 O. hence FGJ 1 Fw as as,, N + co, which
further implies that Gwo 4 Gws. Applying the continuous mapping theorem:
(G (e, (C))) a0 -:: S') 1 ((G.- 2t 2) $),
and again via continuity of the max(.) function:
Eq. 31c 0.
(33)
Additionally, by applying Belghazi et al. (Belghazi et al. 2018 Lemma 1), for any e >> 0, there exists of neural networks (yyrer on some compact domain I such that:
Eg.
31d
a.s..
(34)
Finally, combining Eq. 32 Eq. 33 and Eq
34
then applying Eq. 29b within Eq.
34
completes the
proof of the fairness guarantee.
The proof of the balance guarantee follows similarly, as it also relies on the DirPDV representation of the KI. divergence, and is thus omitted.
28

---

Published as a conference paper at ICLR 2026
Generated from
Generated from fee
RRM
RRM
RRM
AGM
AGM
AGM
N DirP Posterior Locations
N DirP Posterior Locations
N DirP Posterior Locations
(a) Da + n. H*)
(b) (a + n, H*)
(c) (ag + n, H*)
Figure 5: Privacy scheme by DirP posterior sampling with d₁ < 02 < a3. Denser patterns indicate
stronger privacy budgets chosen by the operator.
H FURTHER DISCUSSION ON PRIVACY PRESERVING:
We clarify that the impact of marginal perturbation on data utility is modulated by two key factors: the localized privacy budget and the concentration parameter in the DirP. Specifically, the posterior base measure is given by: H
H Pert + ata F........
where is the sample size and will be replaced with mini batch size nmm when implementing Algorithm 2 larger a increases the influence of the perturbed base measure, enforeing stronger local privacy but potentially reducing utility, while smaller shifts weight toward the data, improving utility but reducing local privacy protection. See Figure 5 for visualization. We also note that directly determines the global privacy budget, shown in Proposition 1. Thus, higher values of simultaneously enhance both local and global privacy guarantees.
Eventually, our numerical results highlight the importance of supporting flexible marginal structures and demonstrate that the BNP formulation offers a principled mechanism for balancing privacy and utility. The framework introduces novel privacy amplification effect, which is beneficial for privacy (Steinke 2022 Section 6). In this setting, DirP posterior samples are drawn from a mixture of clean samples and perturbed samples generated from Pert. In contrast, traditional privacy amplification operates by subsampling the data and then applying a Differential Privacy mechanism that perturbs only the selected subset. The proposed framework retains all samples while injecting perturbed samples through posterior sampling, thereby achieving amplification without discarding data. discarding data.
ADDITIONAL EXPERIMENTS
All experiments conducted using PyTorrch Lightning with mini-batch size of 256, running
for 1,000 or 1,500 training epochs on the NVIDIA GPU described in Table [T Further details of the model configurations and dataset-specific training settings are provided in Table [2] This table uses consistent training sample size across datasets to ensure that running times remain comparable across different dimensional settings. We also note that the runtime convolutional neural network (CNN) differs fundamentally from that of two-dimensional (2D) CNN due to architectural and computational differences.
one-dimensional (1D)
Table 11: Hardware specifications of the GPU server used all experiments.
GPU Model
NVIDIA A100-SXM4-40GB
CUDA Version
12.8
Total GPU Memory
40,960 MiB
29

---

Published as a conference paper at ICLR 2026
Table 12: Convolutional Neural Network (CNN) architectures and training configurations across all
datasets Based on 10,000 Training Samples (equal sample sizes) with Mini-Batch Size 256.
Dataset
Adult
ProPublica COMPAS
Bank
Toy Example
MNIST
Marketing
(SLV)
of Dimension
15
18
21
x
Runtime
50 min
50 min
50 min
50 min
400 min
Epoch
1000
1000
1000
1000
300
Type of CNN
ID
ID
ID
ID
2D
# of CNN's parameters
1.8M
1.8M
1.8M
1.8M
302M
The implementation codes for this paper are publicly available at: https://github. com/ ias gitt
1.1
SIMULATED TOY EXAMPLE BASED SHARED LATENT VARIABLE (SLV)
For comprehensive evaluation of the proposed method, we begin this section by constructing four-dimensional toy example designed to analyse the method's behaviour under isolated mechanisms.
1.1.1 SIMULATING DATA CONSTRUCTION
We simulate data in which one continuous covariate, one multicategory covariate, binary outcome, and a binary sensitive attribute are all correlated through a shared latent structure. This controlled setup provides clean environment for examining the factors of interest discussed in this paper, particularly in fairness-related scenarios where dependency exists between the outcome and the sensitive attribute.
Let
Z Zn lid N 1)
denote a latent variable that governs the dependence among all observable coordinates. For each observation [m]], the four attributes
(X1., X2x. Ye, St)
constructed as functions of the shared latent factor Z,, ensuring that the components remain mutually dependent while retaining distinct marginal distributions.
The continuous covariate X1; E IR is constructed through smooth nonlinear transformation of the latent factor,
X1x tanh( (Z.),
which produces bounded continuous attribute while preserving strictly monotone relationship with the latent variable.
The categorical covariate X2 with three categories is obtained by thresholding the latent variable as
1, 2, < 003,
X
2, 0.. Z, 0.6,
3, Z, > 0.6,
which induces an ordinal categorical variable whose distribution reflects the ordering of the latent variable.
The binary outcome Y, is obtained using a logistic link:
Pr(Y, 2 2.) 1 2.52,
30

---

Published as a conference paper at ICLR 2026
so that the probability of assigning category 2 increases sharply the latent variable Z becomes larger.
The sensitive attribute S, is designed exhibit very strong dependence with the outcome. After sampling Y,, the sensitive variable defined by
Y, S
with probability 0.95,
3 Yi,
with probability 0.05.
This maintains realistic binary marginal distribution while ensuring that the mutual dependency between Y; and S; is large.
The final toy dataset
{(ii, X2i, Y,, 1 n = 10000,
provides a continuous attribute, a multicategory attribute, and two binary attributes that are all coupled through a single latent factor.
In the following subsections, we use these simulated datasets with a mini-batch size of 256 (i.e., setting Rub 256 in Algorithm ( as the training data to evaluate the performance of the proposed approach through three below sections
1.1.2 ISOLATING PRIVACY
We first isolate the privacy component our conditional generative approach by setting AF = (no fairness) and XB = 0 (no class balance[3 and by using Riter 1500 in Algorithm 2 to evaluate how effectively the proposed method privatizes the dataset. For this purpose, we employ ceabbbrshii-inferenne attacks (Shokri al. 2017).
This allows us
ceabbershi--iffereee attacker aims to determine whether a specific sample was used in training classifier, referred to here as the reference model. Given input data point, the attacker queries the trained reference model and observes its prediction confidence. Reference models typically assign higher confidence to samples seen during training (members) compared to unseen samples (non-members). The attacker exploits this discrepancy to infer membership. In our setting, the members are the synthetic samples generated by the conditional generative model, which are treated as the training dataset for the reference model during the privacy-evaluation phase.
Differential privacy mechanisms are designed to make the reference model's behavior insensitive to the inclusion or exclusion of any single individual. If the mechanism is effective, the reference model's outputs on members and non-members should be statistically indistinguishable. Thus, membership-intee infeien-efeeneee attacks serve as a practical privacy evaluation tool: failure of the attack indicates reduced training-date leakage, whereas success indicates that membership information is still exposed.
We report the effect of our privacy mechanism using two standard evaluation measures, AUC and test accuracy. with definitions following Fawcett (2006). Together, these two measures characterize the privacy-utility trade-off and reveal how the privacy mechanism influences both Icakage risk and downstream predictive performance.
Attack AUC. The cewtbeershh--iffeeeeeee attack produces confidence score for each sample. When the reference model leaks information, the score distributions members and non- members differ. The Area Under the ROC Curve (AUC) quantifies the attacker's ability to distinguish these two groups. A high AUC indicates strong privacy leakage, whereas an AUC near 0.5 indicates that members and non non-members are indistinguishable. Thus, the AUC directly reflects the privacy provided by the mechanism and captures the trade-off between utility and privacy. For example, across each local privacy budget in Figure(gja)-Ce increasing the hyperparameter strengthens privacy while reducing utility. We also note that, for fixed value of a, stronger local privacy budget results in higher privacy.
Test accuracy. Test accuracy measures the predictive performance of the reference model on a clean, unseen test set. It reflects the utility of the model after privacy noise is applied.
In this example, for simplicity of notation, write Xy to refer the coefficient of the KI. loss, given by Section 4.4.3 associated with both categorical covariates, X21 and Y,.
31

---

Published as a conference paper at ICLR 2026
As stronger noise is introduced (smaller E), the training data become less informative, often reducing test accuracy. Thus, test accuracy captures the utility cost of the privacy mechanism. Figure f)) shows accuracy results across different local privacy budgets and confirms the effect of increasing a in reducing utility. We also draw attention to Figure Ga), which illustrates the role of global privacy in the weakly localized privacy regime (i.e., almost no local noise), demonstrating that the generated data can still achieve meaningful privacy protection while preserving strong utility, as supported by the corresponding accuracy results in Figure f
To complement the ROC-based evaluation, we also visualize the distributions of prediction confi- dences for members and non-members using the histograms in Figure 7 Because a membership- inference attacker exploits differences in the reference model's 's prediction confidence, these histograms provide an intuitive view of the puivacy-utility trade-off.
When the model leaks membership information, the histogram for member points shifts to the right, indicating consistently higher confidence, while the histogram for non-members shifts left. Conversely, when the privacy mechanism is effective, the two histograms overlap substantially, and both distributions move left, indicating that the reference model assigns similarly low confidence levels to members and non-members. In this regime, membership inference approaches random guessing, with attack AUC trending toward 0.50. Thus, the histogram offers a direct visual summary of how the proposed mechanism trades privacy protection against predictive utility.
1.1.3 ISOLATING PRIVACY AND FAIRNESS
To understand how global privacy simultaneously affects fairness and utility, Figure S illustrates the variation of mutual information (MT) against the variation of MMD across different values of a, all with the same (2.5, 0.05) local privacy budget and with 95% confidence intervals. These results are obtained using the full optimization schedule of Algorithm 2 with ÄB O. comparing AF = 1 against Ar 0, and using Piter = 1500 total iterations with Maair = 800 faimess-optimizaaionn steps. Together, these results form
fairness-utte utility trade-off curve.
Figure g () shows that, in the absence of an explicit fairnsss constraint, increasing a raises the MMD (indicating reduced utility), while the MI remains close to the true MI. This demonstrates that stronger privacy distorts the utility of the generated samples but leaves the MI largely unchanged. This behaviour expected due to the copula-based construction of Pertt where the correlation parameter of the Gaussian copula is estimated from the original dataset. As increases and the posterior base measure H" converges to HPrrt, the conditional generator produces samples whose distribution approaches that of Peet Consequently, the MI between the generated outcomes and the sensitive attributes approaches the MI obtained from samples drawn directly from Pert (denoted as "Prior MI" in the plot). The slight discrepancy between the true MI and the Prior MI arises from estimation error.
Figure bb) shows that, although utility decreases as increases, the MI between the sensitive attributes and the generated outcomes remains zero when the model is trained with the fairesss constraint.
Finally, to further explore whether privacy creates any inherent tension with fairness in the optimiza- tion process, we examine how the MI evolves across different epochs through providing Figure 9 Consider the model when it begins by training only for utility during the initial 700 epochs under different values of a. As discussed earlier, increasing causes the MI approach the Prior MI, which explains why, at the end of utility training, the mean MI estimate is noticeably different from the true value (the navy tilde-pattern line in Figure Dh. The MI values are then tracked over the remaining 800 epochs (Niter = 1500, Raarr 800 in Algorithm(2) (2). During the first phase, the model naturally learns the unfair dependence present in the perturbed data. Once the fairness constraint is applied, this dependence is removed across all privacy levels, confirming that faimesss can still be achieved even when the data have been perturbed, as already concluded from Figure b).
However, the amount of perturbation affects the behaviour in faimess optimization. Stronger privacy leads to slower MI reduction and causes the MI values to vary more during training. In other words, higher privacy does not prevent fairness, but it makes convergence in fairness optimization slower.
32

---

Published as a conference paper at ICLR 2026
1.0
1.0
True Positive Rate
0.6
Thue Positive
0.0
Ele (AUC=0.867)
Eloc
(AUC-D.848)
(AUC-0.884)
100 (AUC=00828)
== 100 (AUC=0.808)
0.2
500 (AUC-0.798)
0.2
a 500 (AUC=0.762)
(AUC-0.742)
1000 (AUC-0.719)
10000 (AUC-0.644)
(AUC-0.601)
0.0
False Positive Rate
False Positive Rate
(a) (200, 0.0c)- privacy
(b) (100, 0.05)-local privacy
1.0
1.0
True Positive Rate
0.6
True Posttive
0.6
Eloc (AUC=0.867)
Ebxc
2.5 (AUC-0.853)
(AUC-0.756)
100 (AUC-0.753)
100 (AUC-0.732)
0.2
500 (AUC=0.729)
0.2
a 500 (AUC=0.656)
1000 (AUC=0.678)
3= 1000 (AUC=0.623)
(AUC=D.590)
(AUC-0.574)
0.0
0.0
0.2
9/4
04
False Positive Rate
False Positive Rate
(c) (50, 0.05)- privacy
(d) (30, 0.05)-local privacy
True Positive Rate
0.8
0.8
Élgc 2.5
(AUC
2.5 (AUC=0.848)
(AUC-D.684)
0.5
100 (AUC=0.621)
Alcc 200 (AUC)
Koc - 50 (Acc]
500 (AUC=0.529)
900
(Acc)
1000 (AUCC-.5116
0.8
Exe 100 (AUC)
100 (Acr)
Goc (Ace)
2.5 (AUC)
10000 (AUC-0.502)
Kuc (AUC)
2.5 (Aic)
0.0
62
0.1
1.0
False Positive Rate
(e) (2.5, 0.05)-local privacy
(f) Various local and global privacy
Figure 6: Membership-Infereee Attack ROC. AUC values approaching 0.5 indicate stronger privacy.
1.1.4 ISOLATING CLASS BALANCE AND FAIRNESS
As final investigation, consider two isolated settings in the absence of privacy (a 10 10-6);
(i) Isolating class balance by setting λg = and Af O (no fairness). This experiment examines whether enforcing class balance affects the underlying relationship between the sensitive attributes and the generated outcomes. To this end, the model is first trained solely for utility during the initial 700 epochs, after which the class-balance constraint is activated
33

---

Published as a conference paper at ICLR 2026
200
members
175
non-members
150
Requency
25
Prediction confidence the reference model
(a) Clean model (a = 10-9)
members non-members
members
175
120
non-members
150
100
Frequency
125-
Frequency
0
0.5
Prediction confidence of the reference model
Prediction confidence the reference model
(b) Perturbed model under 2.5
(c) Perturbed model under 50
120
members non-members
members
non-members
1.0
Frequency
80
Frequency
60
60
60
20
0.7
0
0. Prediction confidence the reference model
Prediction confidence of the reference madel
(d) Perturbed model under a = 100
(e) Perturbed model under a = 500
members non-members
members
100
200
non-members
80
Frequency
Frequency
150
100
40
0
Prediction confidence of the reference model
Prediction confidence the reference model
(f) Perturbed model under a = 1000
(g) Perturbed model under a = 10,000
Figure 7: Histograms of prediction confidences for members versus non-members under different choices of all with the same (2.5, 0.05)-local privacy budget.
34

---

Published as a conference paper at ICLR 2026
230
2.0
3.4
1.5
2A2
Affected by global parameter:
2.0
à 2.5
1000
Affected by global parameter:
à 2.5
500
330
à
50
1000
336
500
Prior Ml 0.426
à 100
10000
5.2
0.2
0.525
0.000
0.055
0.000
0.153
0.75
MMD
MMD
(a)
Ar =
(b) AF =
Figure 8: Privacy-fainen farees--iimes-uttiiit trade-off under different choices of all with the same (2.5, 0.05)- local privacy budget.
Mutual Information
Mutual Information
2.5
10
Estimated
0.3
Estimated
Mean MI Utility end=0.48
Mean MI at utiliiyy end=0.467
0.2
Desired
Desired
True MI=0.495
311
135o
Epoch
Epoch
(a) Perturbed model under 2.5
(b) Perturbed model under 100
10
Estimated ML.
6.4
Mean MI utility end 426
Mutual Information
Mutual Information
a 500
True Mi- 0.495
0.3
Mean utility 0.35
Desired MD
02
MI-0.495
6.1
01
0.0
100
2007
3000
1000
1200
1200
1200
1470
1300
/00
520
5.0
-000
22000
2207
2200
1402
2500
Epoch
Epoch
(c) Perturbed model under a 500
(d) Perturbed model under a 10,000
Figure 9: Isolated effects of privacy fairness optimization (Ap 1) under different choices of all with the same (2.5, 0.05)-local privacy budget.
for the remaining 300 epochs (Aitter = 1000, Tibalance = 300 in Algorithm 2). while utility updates continue throughout all 1,000 epochs. We record the MI and SP at each epoch during these final 300 epochs. As shown in Figure a)) the MI and SP remain concentrated around the corresponding true values learned during the first 700 epochs of utility training, indicating that applying class balance alone does not alter the underlying unfaireess present in the generated data.
35

---

Published as a conference paper at ICLR 2026
(ii) Isolating fairness by setting Ap and Äy (no class balance). Figure 10 b) illustrates the effect of enforcing faimess when the model is first trained for utility during the initial 700 epochs. and the fairness constraint is activated only in the remaining 300 epochs (Miter = 1000, Maair = 300 in Algorithm experiment highlights the central role of the fairness constraint in removing the dependence between the sensitive attributes and the generated outcomes. After the first 700 epochs, during which the model learns the underlying unfair relationship present in the training data, the faimess constraint effectively eliminates this dependence, even in the absence of adjustments.
contrast to the class-balane paass-halance-olly setting, this
As 1, ds= Estimated MI 0.495
6.94
Mutuel Information
0.52
5.00
0.00
0.30
Stant toa Pariy
An. Ap =0
Estimated SP
SP-0.9
0.85
Epech
Epech
(a) Class balance only (no fairness)
Statienical Party JRgY 1-1-111-11----- nV-115-71]
0.6-
Mutual Information
λr Ae
Ag. A-
Estimated MI
Estimated SP True -0.
Desired
0.2
True 0.455
230 Epoch
220
Epach
(b) Fairness only (no class balance)
Figure 10: Isolated effects of class balance and fairness on MI and SP.
I.2 MNIST TOY EXAMPLE WITH MAXIMAL DEPENDENCE.
The MNIST dataset consists 60,000 handwritten digit images spanning 10 classes (0-9), with each image represented 28 x 28 pixel grid (784 dimensions) (LeCun 1998). As additional illustrative toy experiment, the conditional generative model is trained on the MNIST dataset, where the class label is used the conditioning attribute. In this setting, the image vectors are treated the outcomes Y and the corresponding digit labels as S. In the baseline configuration, the generator is trained so that S matches Y with probability creating between the outcomes and the digit attributes. In this configuration, the model simply takes digits S and produces the corresponding images; see Figure [I]
extreme case of maximal dependence
The proposed fairness mechanism is then applied to gradually reduce this dependence, with the goal of driving the MI between S and the generated outcomes Y (i.e., Y) toward zero. In this setting, the generator receives the digit labels S but is expected to produce images that are independent of S. Figure 12 shows synthetic outputs for the fixed input sequence S = (2, 0, 3, 8, 7, 4,6, 7, 1,9) under different values a. As expected, there is almost no meaningful correspondence between the inputs and outputs, indicating that the fairness mechanism successfully enforces independence
36

---

Published as a conference paper at ICLR 2026
between S and Y. Moreover, for each fixed local privacy budget (isolating the effect of global privacy), increasing a increases the proportion of perturbed outputs. Conversely, when a is fixed (isolating the effect of local privacy), the severity of the distortions becomes more pronounced as the global privacy budget grows. This example illustrates how small values of a can preserve good utility even under strict local privacy budget, whereas larger values of induce increasingly strong perturbations.
is emphasised that this experiment digit labels do not constitute a meaningful or interpretable sensitive attribute. This is precisely the reason is referred as toy example. Its purpose to illustrate that the proposed method can eliminate even perfectly enforced dependence between S and generated Y and to provide intuition about the behaviour of the local and global privacy mechanisms, This toy example simply the visual realisation of the posterior sampling diagram shown in Figure 5
intended represent realistic fairness scenario, because
S
8 7
5
4
6
7
19
Y 20 5
4
0 9 1
Figure 11: MNIST generation without the fairness mechanism (Ar 0) and without privacy (a 10-6), using input vector S.
5
8
7
5
9
S 2
0
3
7
1
9
0
5
0
2
5
J
3
9
7
2.5
8 4
4
3
7
5
5
9
2.5
f
9
0
9
3
2
5
6
2
2
50
7
7
à
3
4
2
5
6
6
2
3
0
50
5
3
1
7
4
R
J
2
%
}
102
Y
5 3
4
/
5
f
9
4
8
6
102
5
2
0
R
7
2
3
3
/ 5
3
4 2
/
500
6 / 4 0 9 7
7 3
1
9
5
500
9
O
5
6
3
5
2
2
5
I
10
O 7
5
3
9
5
5
a 10
4
2
6
5
3
2 7
10
8 O
4
2 I
8 / 7
/
6
2
7
10
(a)
(200, 0.05) local privacy
(b) (100, 0.05)-local privacy
S
2
5
8
7
5
7
1
9
S
0
3
5
8
5
4
6
7
9
7
4
7
f
5
7
7
9
2
2.5
4 3 4
4
P
8
3
9
4
0
6
a = 2.5
9
S
2
9
3
O
4
8
5
50
6
5
2
6
5
1
7
50
7
2
2
6
4
2
102
Y
3 & 7
Y
K
9
7
Q
4
a 10
7
7
6
/ R
8 7 0 5
3
5
500
5
6
$
S
a 500
9
/ 2
O
6
10
10
6
7
6
7 6 0 4
1
a 10
(c) (30, 0.05)-local privacy
(d) (2.5, 0.05) privacy
Figure 12: MNIST generation in the presence of the fairness mechanism (Ap 1) with input vector
S.
I.3 ADULT DATASET
I.3.1 DATA DISCRIPTION
The Adult dataset is a well-known benchmark dataset derived from the 1994 US Census Bureau database. It contains demographic and employment information from sample of 48,842 adults with 15 columns in total. This dataset widely used for income prediction and for evaluating fairness-aware machine learning methods.
37

---

Published as a conference paper at ICLR 2026
Continuous Attributes:
age (A): Age of the individual (in years).
fnlwgt (F): Final weight, representing the number of people the census believes the entry represents.
education_num (EN): Number of years of education.
capital.gain (CG): Capital gain, income from investments.
capital loss (CL): Capital loss, loss from investments.
hours.per.week (HPW): Hours worked per week.
Discrete Attributes:
workclass (W): Employment status, with categories:
-'Private', 'Self-emp-not-inee, 'Self-emp-inc', 'Federal-gov', 'Local-gov', 'State-gov', 'Without-pay', 'Never-worked'.
education (B): Level education, with categories:
'Bachelors', 'Some-college', '11th', 'HS-grad', 'Prof-school', 'Assoc-addm', "Assoc- voc', '9th', '7th-8th', '12th', 'Masters', 'Ist-4th', '10th', 'Doctorate', '5th-6th', 'Preschool".
marital.status (MS): Marital status, with categories:
"Marriddciv-spooue'' 'Divorced', 'Never-married', 'Separated", 'Widowed', 'Marred- spouse-absent', Married-AF-spouss
occupation (O): Type of occupation, with categories:
"Tech-support', 'Craft-repair', 'Other-service', 'Sales', 'Exec-managerial, 'Prof- specialty', Handders-cleanerr', 'Machine-op-inspet, "Adm-clerical', 'Farming- fishing', "Transpor-mmovnng" 'Priv-house-sev'' 'Protective-serv' 'Armed-Forces'.
relationship (Re): Relationship within family, with categories:
''Wife', 'Own-child', "Husband', 'Not-yn-family', 'Other-relative', 'Unmarried',
race (Ra): Self-identified race, with categories:
'White', 'Asian-Pac-Islamee Pac-Islander',
'Other', 'Black'.
sex (S): Biological sex, with categories:
'Male'. 'Female'.
mative_countyy (NC): Country of origin, with categories:
'United-States', 'Cambodia', 'England', 'Puerto-Rico', Rico', 'Canada', 'Germany', 'Outlying-UU Cuaiingg-SSuuammmm --GGaa--UVV-tte) 'India', 'Japan', 'Greeee', 'South', 'China', 'Cuba', 'Iran', 'Honduras', 'Philippines', 'Italy', 'Poland', 'Jamaica', 'Vietnam', 'Mexico', 'Portugal', 'Ireland', 'France', Domminican-Rpubllee 'Laos', 'Ecuador', "Taiwan', 'Haiti", 'Columbia', 'Hungary', 'Guatemala', 'Nicaragua', 'Scotland', 'Thailand', 'Yu- goslavia', 'El-Salvador', "Trinadad&Tolee Tobago', 'Peru', 'Hong', "Holand-Netherland:::
income (I): Binary income class, with categories:
50K', 50K'.
I.3.2 PREPARING DATASET
The sensitive column is moved to the last position to maintain consistent structure.
Rows with missing values (NaN) and incorrect header rows (where sensitive is mistak- enly used as a value) are removed.
Continuous attributes are converted to numeric types to ensure proper mathematical opera- tions during training.
38

---

Published as a conference paper at ICLR 2026
1.3.3 DATASET LIMITATIONS
Ding et al. (2021) showed that the Adult dataset suffers from a critical limitation arising from its binary target label defined by the $50,000 income threshold. Because this threshold corresponds to the 76th income quantile overall, and the 88th and 89th quantiles for Black individuals and women, respectively. it induces highly unbalanced label distribution across groups. As demonstrated in Ding et al. (021)) empirical fairness findings become highly sensitive to this extreme threshold: the magnitude of fairness violations, the trade-off's between fairness criteria, and the apparent effective- ness of algorithmic interventions all change substantially as the threshold varies. Consequently, the Adult dataset may not reliably indicate which fairness algorithm is genuinely better or worse, since both similar and contrasting results can arise from this breshold-inducee imbalance rather than from true differences algorithmic performance.
1.3.4 EVALUATION RESULTS
The trade-off discussed in Appendix H illustrated in Fig. 2a particularly in Case 4, where all attributes receive a localized privacy budget of (2.5, 0.05). When the concentration parameter is set to a = 10 and a = 5, this leads to global budgets of (7.54, 0) and (29.35, 0.01), respectively. The MMD values in these settings illustrate that meaningful utility can be preserved despite significant local perturbation, through appropriate tuning of a.
Fairness Evaluation through Correlation Heatmaps: To further investigate faimess, we provide heatmaps the correlation matrices for real and generated samples, considering the sensitive attributes sex (S) and relationship (Re), as shown in Figures [3] and [4] respectively. In each case, the block representing the correlation between the sensitive attribute and income (I) is highlighted by a green frame, indicating values close zero when Ap- 1. This reflects the direct effect of minimizing the mutual information (MI) between the generated I and the sensitive attributes, effectively reducing unwanted dependencies.
Fairness Flexibility for Multi-Categon Sensitive Attributes: The example in Figure 14 which includes 6 categories for the sensitive attribute relationship. demonstrates a significant advantage of our model over baselines designed primarily to minimize SP for binary sensitive attributes. This broader applicability highlights the flexibility our approach addressing fairness data synthesis for sensitive attributes with more than two categories.
Notion on Confounded Dependencies: Additionally, some correlations between and unpro- tected attributes (last row of the correlation matrices) are also reduced in the presence of fairness. This effect is particularly pronounced for the correlations between and S as well as and Re, given the strong dependency between S and Re with a true correlation of -0.62. The direct path given by Figure T5a should not lead to the mistaken expectation that no reduction occurs in other attributes, as it can overlook the impact of confounded relationships. For example, when the sensitive attribute is S. undirected connections between I and other attributes, mediated through S, are shown in Figure [5b] This indicates that minimizing the MI between and S during training reduces not only their direct dependence but also the indirect correlations with other attributes influenced by these confounded relationships, as illustrated in Figures 15d and
15e
Figure 16 illustrates the consistency of our proposed approach as the sample size approaches infinity. The results elearly demonstrate that utility. fairness. and balance are strongly supported with increasing sample size, providing numerical validation for the theoretical guarantees established Theorem 2
Consistency Investigation:
Scalability Advantage of the Proposed CBNP Framework: Based on our tests, running 1000 iterations on the Adult dataset took between 3.5 to 4 hours. This is relatively efficient, especially compared to one of our state-of-the-art baselines, DECAF, which took around 8 days under similar conditions. Notably, DECAF is designed solely for enforcing fairness, whereas our framework simultaneously addresses three key policy requirements, faireess included. Attempting to adapt DECAF for additional guarantees- Suchh as applying DP-SGD to introduce gradient noise for differ- ential privacy-wonl would significantly complicate its implementation and further increase computational
39

---

Published as a conference paper at ICLR 2026
burden. This highlights the scalability advantage of our proposed framework despite incorporating Bayesian nonparametrie modeling and faimess--prvaccy components.
1.0
1.0
0.5
0.5
0.0
0.0
0.5
0.5
-1.0
11.0
EN 95
C. NPWNC
A
c. HTWNC
(a) Real Dataset
(b) Fake: Ap
1.0
0.5
0.5
0.0
0.5
NC WWW
-1.0
(c)
Fake: Xp 0.5
(d) Fake: A
Figure 13: Heatmap of the correlation matrix for the Adult Dataset (real) and synthetic samples
(fake), where sex is the sensitive attribute and income (>50K) is the desired outcome.
1.0
1.0
0.5
0.5
0.0
0.0
0.5
0.5
NCC FPW CL
-1.0
-1.0
2N
MS
CU CL dorenii
(a) Fake: X
(b)
Fake: Ap
Figure 14: Heatmap of the correlation matrix for the Adult Dataset (real) and synthetic samples
(fake), where relationship is the sensitive attribute and income (>50K) is the desired outcome.
40

---

Published as a conference paper at ICLR 2026
MS
0
EN
True MI
Ra
Re
1100
HPW
100
C90
S
064
0. 0.005
NC
114
0.041
F
025
0. 0.005
CG
W
36666
0.083
00115
CL
A
0.037
0.068
I
(a)
EN
MIS
Ra
True MI
EN
M1
Ra
True MI
Re
NC
NC
9.11
CG
620
500
2,006
2,040
(b) Sensitive Attribute: Sex
(c) Sensitive Attribute: Relationship
Estimated M
Estimated
R
HPW
HPW
NC
5
NC
CG
W
090
0 022
0.018
(d) Sensitive Attribute: Sex
(e) Sensitive Attribute: Relationship
Figure 15: (a) Direct dependency structure between 'I' and related attributes. (b) Dependency structure between 'S' and its related attributes. (c) Dependency structure between 'Re' and related attributes. (d), (e) Estimated dependency structure between 'I' and related attributes in the presence
of fairness, i.e. AF
1.4 PROPUBLICA COMPAS DATASET
1.4.1 DATE DISCRIPTION
We use the COMPAS data, downloaded from the ProPublica repository, which contains 60,843 records and 28 attributes. After removing personally identifiable information such as person identifiers. names, and date of birth, the remaining attributes can be shown into two groups:
Continuous Attributes:
Raw_Score: numeric COMPAS risk score before scaling.
Decile.Score risk score transformed into decile from (lowest) 10 (highest).
Rec supervision Level: recommended supervision level, recorded as a numeric value.
41

---

Published as a conference paper at ICLR 2026
10
10
MMD
10-1
Work-Class
Marital-Statur
Sample
Sample
Sample
(a) Utility
(b) Fairness
(c) Balance
Figure 16: Consistency of the proposed approach as the sample size increases with Xp aiming for balance with respect to W and MS. All metrics are measured between generated samples and their corresponding true underlying distributions.
Discrete (Categorical) Attributes:
Agency: agency conducting the assessment.
Sex: gender of the individual.
Race/Ethnii face/Ethnicity: recorded racial or ethnic group.
Assessment_Reason reason why the assessment was performed.
Language: preferred language for communication.
Legal_Status: the individual's current legal status.
Custody.Status: whether the individual in custody at the time of assessment.
Parital-Statur marital status of the individual.
Screening Date: date when the screening completed.
Rec.SuperriaaionneeeeL..e. textual description of supervision level.
Scal Sicale_Tnformaat: (ScaleSet.ID, ScaleSet, Scale.ID).
identifiers and names of the COMPAS scales used
Display Text: the type of risk being scored (e.g., recidivism, violence).
Score Text: qualitative risk category (Low, Medium, or High).
Assessment.Typer whether the assessment is initial, reassessment, or another type.
ags: indicators for whether the assessment completed (IsCompleted) or marked as deleted (IsDeleted).
1.4.2 DATA LIMITATION
Bao et al. 22021 argued that the COMPAS dataset contains substantial measurement error, jurisdiction specific artifacts, and systemic policing disparities that influence both the covariates and the outcome labels. Crucially, key outcomes such rearrest or miilure-to-apperd do not reflect ground-trui risk but are shaped by discretionary criminal-justice processes, leading to structurally biased labels. These limitations imply that fairness metrics computed on COMPAS should not be interpreted as evidence of not translate into fairer decision-making in practice. Consequently, although COMPAS may remain useful as a controlled benchmark for comparing algorithmic behavior, it cannot support claims about fairness in real criminal-justic settings.
fairness: a reduction demographic-parit disparity within the dataset does
1.4.3 EVALUATION RESULTS
In this dataset, the column Score.Text represents how the COMPAS system classifies individ- ual's s risk future crime. has three possible values: Low, Medium, and High, which correspond to whether the system believes person is unlikely, somewhat likely, highly likely to reoffend.
42

---

Published as a conference paper at ICLR 2026
These risk categories matter because they can influence important decisions such as bail amounts, sentencing, or parole recommendations. The column Ethnic identifies each person's 's racial or ethnic group, such protected attribute, it should not unfairly affect whether someone is labeled Low, Medium, or High risk.
Caucasian, Hispanie, Asian, and others. Because race is
For clarity, throughout our discussion we use Y to denote the outcome variable- here, Y Swore-Texxe which encodes the risk category assigned by COMPAS-and S to denote the sensitive attribute-here, S stnnic, which represents a person's 's race or ethnic group. When studying fairness with this dataset, we examine how the distribution of Y varies across the categories of one group has disproportionately high probability of being labeled "High" risk compared to others, even when there is no legitimate predictive justification for that difference, may indicate bias the risk assessment system.
The conditional distribution shown in Table and clearly illustrates disparities across racial and ethnic groups. For example, for the outcome Y "Low",
Pr( "Low" S "AfricaanAAmeriaan") # Pr(Y "Low" S "'other",, (35)
indicating that the probability of individuals from different ethnic groups being classified as "Low" risk is not the same, and similarly for the "Medium" " and "High" categories. This inequality highlights the presence of group-level disparities that our method seeks mitigate in the generated data.
Any generative model built to create synthetic data from this unfair dataset should preserve meaningful relationships (e.g., between past behavior and risk category) while reducing removing spurious dependencies between S (race) and Y (risk label). This ensures that the generated data promotes fairness and does amplify racial disparities present in the original data.
Table 13: Compas: Conditional probabilities Pr(Y "Score Text" S "Ethnic")) for each racial or ethnic group in the ProPublica COMPAS dataset, where Y is the COMPAS risk category and S is the race/ethnicity.
Ethnic Group
Score Text
Low
Medium
High
African-Anne Amerrcan
0.572
0.259
0.169
Arabic
0.773
0.160
0.067
Asian
0.858
0.099
0.043
Caucasian
0.752
0.173
0.076
Hispanic
0.797
0.149
0.054
Native American
0.639
0.215
0.146
Oriental
0.846
0.103
0.051
Other
0.837
0.128
0.034
In Section 5 was demonstrated that the model performs well on this dataset while simultaneously incorporating three key policies: privacy, fairness, and class balancing. As complementary material, bar plots for the attributes sex, marital, and language are presented in Fig. [7 comparing category percentages across the original data, the unbalanced generated data, and the balanced generated data.
In all three plots, the green bars represent the ideal balanced proportions, assigning equal weight to each category as a baseline. The orange bars display the category proportions in the generated dataset produced by the trained model without applying balance constraints the three attributes.
43

---

Published as a conference paper at ICLR 2026
Table 14: Compas: Distribution of racial/ethnic groups in the original COMPAS dataset.
Ethnic Group
Proportion
Percentage (Sise)
African-American
0.444899
44.49%
Caucasian
0.358020
35.80%
Hispanic
0. 0.143681
14.37%
Other
0.042601
4.26%
Asian
0.005325
0.53%
Native American
0.003599
0.36%
Arabic
0.001233
0.12%
Oriental
0.000641
0.06%
As expected, the charts show that the generated samples become balanced once the balancing constraints are activated. For example, the proportion of the minority Spanish language group increases toward the ideal balance level.
1.5
BANK MARKETING DATASET
I.5.1 DATA DESCRIPTION
We use the Bank Marketing dataset from the UCI Machine Learning Repository, which contains 45,21 records and 21 attributes collected from series of direct marketing campaigns conducted by a Portuguese banking institution between May 2008 and November 2010. No campaigns were during January and February of each year, and therefore these months do not appear in the dataset. Each record corresponds to single phone contact with a client, with attributes describing demographic information, previous interactions, economic indicators, and campaign details. Each row corresponds to single phone contact made to a client as part of a campaign aimed at promoting subscription to a term deposit product. The attributes can be grouped into two classes:
Continuous Attributes:
age: age of the client in years.
durati on: duration of the last call, in seconds.
onn
campaign: number contacts performed during this campaign for this client.
pdays: number of days since the client was last contacted from a previous campaign (-1 means never contacted).
previous: number of contacts performed before this campaign.
amp_var_rater employment variation rate (quarterly indicator).
cons pri ce idx: consumer price index.
cons conf idx: consumer confidence index.
euribor3mg 3m: Euribor 3-month rate.
rremmlloedd number of employees (economic activity indicator).
Discrete (Categorical) Attributes:
job: type job (e.g., admin., technician, blue-collar).
marital: marital status (married, single, divorced).
education: education level (basic, secondary, tertiary, unknown).
default t: whether the client has credit in default (yes, no).
44

---

Published as a conference paper at ICLR 2026
Original/Training
Generated/Fake (Unbalanced)
Ideal Balance ( % per Category)
Percentage
Generated/Fake (Balanced)
Single
Married
Divorced
Seperated
Signiffiantt Other
Widowed
(a)
Marital Status
Original/Training Generated/Fake (Unbalanced)
100
Percentage
Ideal Balance t3 % per Category) Generated/Fake (Balanced)
Male
Female
(b) Sex
Original/Training Generated/Fake (Unbalanced)
100
Ideal Balance (5 % per Category)
Percentage
Generated/Fake (Balanced)
English
Sparish
(c) Language
Figure 17: Compas: Bar plots of different categories for three categorical unprotectd attributes Marital Status,, Sex, and Language.
45

---

Published as a conference paper at ICLR 2026
housing: whether the client has housing loan (yes, no).
1 loan: whether the client has personal loan (yes, no).
contact: type of communication contact (cellular, telephone).
month: last contact month of the year.
day of week: last contact day of the week.
poutcome: outcome of the previous marketing campaign (success, failure, nonexistent).
subscribed: binary target variable indicating whether the client subscribed to a term deposit (yes or no).
I.5.2 EVALUATION RESULTS
In this dataset, the column subscribed records the outcome of the marketing campaign, specifying whether the client subscribed to a term deposit. The values are binary: yes (the client subscribed) or no (the client did not). This variable is the primary focus of predictive modeling as it directly reflects campaign success.
The feature month indicates the month in which the client was last contacted, with possible values from March to December. January and February are absent because the bank did not conduct campaigns in those months during the collection period. Although month is not demographic attribute, it captures strong seasonal and campaign-intensity effects: some months show markedly higher subscription rates, which are likely driven by marketing schedules rather than inherent differences in client willingness. Relying such feature may cause models-anc and synthetic data generators to overfit to seasonal trends rather than underlying behavior, potentially skewing analyses or downstream decisions.
Here Y Subser i bed and Month. Our fairness assessment examines how the distribution of Y varies across the different months represented by S. When certain months have disproportionately high probabilities of subscription, this reflects a form of campaign bias that may not be desirable to reproduce.
The conditional distribution given Table 15 which makes clear that subscription probabilities differ substantially by month. For instance, for the outcome Y = "Yes",
Pr Y "Yes" S "March") * Pr(Y = "Yes" S ==May""),
showing that clients contacted in different months do not have equal likelihoods of subscribing. These differences are likely driven by campaign timing and call strategies, and thus are not necessarily representative of persistent client characteristics.
We aim to generate data evaluate the model's performance on this dataset and to verify whether preserves useful patterns while mitigating the month-specific biases present in this additional dataset.
To evaluate approach, again employ the tools outlined in Section 5 using the same settings for the local and global privacy budgets and the equal proportions of sensitive attributes described in Section 5 Here, two active attributes, marit al and educat ion, are considered for balancing. This setup allows us to verify that the generated data achieve balance between utility and fairness while satisfying the specified privacy requirements.
Utility. Regarding Table I6 by comparing the non- con-faineess part (orange cells) with the proportions of the original dataset given in Table [5] it is evident that the model preserves perfect utility in the absence of fairness constraints. We compute the MMD to quantify the similarity between the generated and original data distributions. Lower values indicate closer alignment, which crucial for maintaining fidelity the original population. Additionally, predictive performance is assessed through Fl Fl-score and overall accuracy (Acc) using a classifier trained on synthetic data and validated on real hold-out data. High scores suggest that relevant relationships between variables are preserved.
Fairness. We use Mutual Information (MI) between and to measure how much information the month of contact carries about the likelihood of subscription. Negligible MI in Table 17 implies weaker dependence. signaling that the generative model has reduced the influence of month-specific campaign effects. Furthermore, the results in the blue region of Table 16 confirm that the model
46

---

Published as a conference paper at ICLR 2026
Table 15: Bank Marketing: Conditional probabilities Pr(Y 'Subscribed" S Month") for each contact month in the original dataset, where Y is the subscription outcome and $ is the contact month.
Month
Subscription Outcome
No
Yes
March
0.495
0.505
April
0.795
0.205
May
0.936
0.064
June
0.895
0,105
July
0.910
0.090
August
0.894
0.106
September
0,551
0.449
October
0.561
0.439
November 0.899
0.101
December
0.511
0.489
Table 16: Bank Marketing: Conditional probabilities Pr(Y "Subscribed" | "Month") for each contact month in the generated samples, where Y is the subscription outcome and S is the contact month.
Month
Subscription Outcome (Ap 0)
Subscription Outcome (Ap = 1)
No
Yes
No
Yes
March
0.501
0.499
0.863
0.137
April
0.763
0.237
0.812
0.188
May
0.911
0.089
0.900
0.100
June
0.904
0.096
0.854
0.146
July
0.896
0.104
0.917
0.083
August
0.913
0.087
0.867
0.133
September
0.498
0.502
0.827
0.173
October
0.592
0.408
0.888
0.112
November 0.926
0.074
0.902
0.098
December
0.497
0.503
0.886
0.114
successfully yields nearly equal probabilities across the two categories of the sensitive attribute month.
Together, MMD, Fl-score, and Ace measure how well utility is retained, while MI reflects fairness improvement. Finally, because S (month) is multi-class, most binary faimess baselines are not applicable, underscoring the importance of approaches- such as ours thatt handle multi-category sensitive attributes.
47

---

Published as a conference paper at ICLR 2026
Table 17: Bank Marketing: Ml C (Y,S)
0.0264, F1 True 0.540, and AcCTrue 0.884 over
10 runs.
AB =
AP
MI
MMD
FI (DTC)
Acc (DTC)
None
0
0.0281
0.0014 0.0020
0.855
0.531
le 5
0.851
0.523
Sex
Education
1
le - 5 0.0026
0.858
0.525
J GRAPHICAL OVERVIEW AND CLARIFICATION
Tabular Data Representation: A general diagram in Fig. 18 illustrates the tabular data representa- tion, making the concepts Section 4 casier to follow.
Contribution of Perturbed Prior HPert to Localized Privacy: A review of the construction of Pert is provided in Fig. 19 which depicts how the base measure in the Dirichlet Process is designed to inject localized differential privacy.
Target Attack Classes: An overall overview of the fair-private mechanisms in the proposed
approach for addressing several types of attackers is presented in Fig. 20, providing clearer
understanding of all mechanisms. For further details on such attackers, see Luo et al. (2024).
K COMPUTATIONAL ALGORITHM
Algorithm 1 outlines the computational steps for generating samples from Pert using latent Gaussian copula approach. Figure 21 presents an overview of the proposed generative models for synthesizing private, fair, and balanced tabular data.
We clarify that the perturbed base measure Pert is constructed only once, prior training. is not recomputed at each iteration. Once built, Perttt is used to define the posterior base measure H' from which we repeatedly sample during training. This sampling step computationally comparable to drawing from any standard distribution and does not incur significant overhead. Because this construction is a one-time operation, the method remains scalable even for datasets with dozens of variables. The perturbation of marginal distributions for the base measure is done upfront, and the rest of the training proceeds with standard sampling-basee techniques.
Algorithm - displays the implementation steps of the proposed BNP framework for generating fair, private, and balanced samples, implemented using VAECGAN, member of the genera- torddecoder-based family of generative models. We note that the triple-policy proposed in this paper can also be implemented with other members of this family by applying their respective algorithms.
L LIMITATIONS
In our model, fairness is enforced by explicitly conditioning the generator protected attributes and minimizing the MI between outputs and those attributes. This is feasible in CGANs with structured inputs, but LLMs like ChatGPT, conditioning occurs implicitly via prompts, and protected attributes are not explicitly disentangled, making it difficult to define or minimize MI in a well-posed manner. On the privacy side, our BNP mechanism perturbs structured data distributions using a DirP with copula-based base measure, which is tractable for tabular data but not directly applicable to token-level LLM training in its current form, especially since LLMs lack access to explicit marginal structures. However, hope address this by leveraging DirP mixture models to construct sequence-level prior over token representations, enabling structured uncertainty at the sequence or span level while preserving coherence and privacy in autoregressive generation. Extending our framework would thus require a fundamental reformulation of how faimesss and privacy constraints are represented and enforced in autoregressive architectures.
48

---

Published as a conference paper at ICLR 2026
D
(C)
D (D) LNN 11
(
D(D) (D)
D (D)
111
LND -2 LND-1
I.ND
D
:
(
DD)
(D
n,1
R.NC n,
HND 2 RNND-1
n,ND
Compressed Representation
D
R MoxAe
( Luk BRxiN
D 1:n
(D)
D
(D)
(D)
1:8,1
1:1,1
1a
1:n.No.
Xim
Yin
51.0
(C)
the 1-th continucus feature of the 1-th observation
N: # of continuous feature
D ((0))
No+ No.
the 1-th categorical festure of the z-th observation
N categoricall Teature
Unprotacted
Protected
(a) General View
Numerical
(-00 240)
Numerical Bargeshveum
(-29,, +-0
Categorical
Categorica Ronge Prngeo/ratuo prouoo
Categorical
Categorical ge Shattons
D.... Sag.e
Q... Kige
D11
(C)
D
(C)
D
(D)
D (D)
D (D)
D
(D)
INN 1,1
1,ND-2 1,Np-1
1.ND
D1n
D (C)
D (C)
(D)
D (D)
(D)
D (D)
n,Nc
n,
n n-ND-1
n,N
K-1
Mag-t
Mag---
Mang -1
Mipu is the number of samples that fall category j of fo categorical column, in = 1, ND
...
...
P5.
Pl -M
Mag-ta
1, Kip
...
M1.0
...
Kig
Range of Values
D (D) %
0
KND - 1
n n
D.N
NND.NND
PND
KND NN j=1
KND Σj nNDJ
KND nND.
Σ
Σ Σ
(b) Detailed view
Figure 18: Clarified view of the triple tabular data representation.
49

---

Published as a conference paper at ICLR 2026
Pos 1:N
(D Pos(C)
Pos(C) 1:N.Nc
D PDs(D)
D Pos(D) D Pos(D)
Pos(D)
Prs Trs-Procsssing
1:N,1
1:N,1
1:M.ND NM -2 I.N.N 1
1:N.ND
Quamila
Quantite
Transform
One-Hot
Voctor
One-Hot
Vector
One-Hot
Vector
Pos 1:W
(D
Pos(C)
D Pos(C) Pos(D) 1:N.Nc 1:N.1
D
D
Pos(D)
D
Pos(D)
Pos(D)
I:N,1
1-
D:N.ND-1
1:N.ND
Using Prior to Infer
Posterior Sampling
H Pert
H
P))t(()
H
P)rt(C)
H Pert(D)
Pert(D)
Nc
ND
Attach perturbed margins using the
Copula Theorem
Perturbing
H
C)
Pe
H
PD)
H ND
Pert(D)
Nc
NOW
co
RORM
(6:00
RRM
H (c)
(C)
(D)
H (D)
H
Nc
Np
Regularizing
NGAS (3
10
Cate KatKKnrruuu Plgg)
Cat PL)
H (C)
(C) Nc
H (D)
H
N
treakto N No marginal
distributions.
H
Dirichlet Process Properties:
[ H Port
"H Pert" incorporates perturbation the prior knowledge on data distribution
i. Var[F] (-
"a" controls how how strongly F concentrates around Pert
a+1
Figure 19: Perturbation scheme for designing Pert to inject localized privacy.
50

---

Published as a conference paper at ICLR 2026
No Differential Privacy!
With Differential Privacy!
Emily Training Dataset
Without Emily
With Emily In Training Dataset
Without Emily Training Dataset
Training Dataset
Mire
latn) (Foim i gooo (gtob) M (F (Peinn Sglob oglob)gglbb)
(F
DirP
M (a+i
Diry
(FD, Eglobo Oglob)
M (a+m) Dirr
(P Di.. Cglobo
MAGM(D Jomie e, 6) MaCM (D) (cy Mussa (D) (D)
todel E, 6)
MaGM (G) (G)
Linde E, 6) MaGM (Cy
Lidde
E, 6)
kmmg' c) = MRRM (D)
(D))
MRRnd (D (0)
tomde c) = MRRM tom
(0))
Generator
Generator
Inforence attacks.
Emily's Information is in the dataset!
Mambership inference attacks.
cann't Icant tall Emily a Information is
Membership
Riconnetructon attacks.
Reconstrudion attackss.
dataset!
(a) Privacy Mechanism
No Fairness Awareness
With Fairness Awareness!
D 1m
D 1:7
MI = 0 ( Uip
MI(Y,S) 0
DKL
DKL. ( Up >
Generator
Generator
Sensitive Data information and its Distribution Property cannot
Sensttive Data Information can be found as well Distribultin Properties pertiee
Poisoning Allacks
Property Inforence Attacks.
Faimess Poisoning Attacks
Property Inference Attacks.
(b) Fairness Mechanism
Figure 20: Overview the triple policies Prirr-Pivvat-Bal enforced by our proposed approach to address potential attacks.
51

---

// Subsequent thresholds from Published as a conference paper at ICLR 2026
Algorithm 1: Generate a random vector from P using latent Gaussian copula approach
Input:
R: Estimated correlation matrix of dimension (d x d)
FAGM Inverse of the continuous CDF FAGM,c
(Pro ..... ........... .Pii ): Probability vector for the categorical variable MRRM (D) Pri(D) (D) taking values from set {
Jocin
Kip Number of categories for each categorical variable
Output: Pert (DPert(C) ,..., Ne PCrt(C)
....,DPert(D) Pert(D)
Nn
Sample from Multivariate Normal Distribution (Latent Copula Variable):
Number of Lerms: No
Number of terms: No
Zi..... ZNo ZNo+1s.... ZNo+ND
~ ~N(0, R); // is of size x1x
Continuous Variables
for ic to .C do Ule f- (Z D. Pc) FAMMic (Utc); 10
/1 Apply standard normal CDF
1/ Apply inverse marginal
Discrete Variables
for iD = to ND do
Creating Thresholds
Tip f- (p- // First threshold from first probability
for k = 2 to Kip
1 do
Tink + $ --1 E port)
12
cumulative sum
13
Ne < Tip1 then (D) +-1
else if Till ZNotti Tip then
Pe) + 2 in else if then
16
17
// Continue for other thresholds
else if ZNetip > TinKip then
D. D
PD))(D) t Kip
29
Separately, our fairness term is based on regular mutual information (MI), which directly measures dependence between the sensitive attribute and the generated outcomes. This formulation does not capture fairness notions that are inherently classifier-oriented opportunity, and predictive equality, because these notions require conditioning the true outcome and therefore do not directly translate to generative models.
as equalized odds, equality of
An additional extension of interest is conditional statistical parity, which is not classifier- but extends statistical parity by conditioning on observable non-sensitive features. Supporting conditional SP within framework would require estimating conditional MI under Dirichlet- process formulation, specifically through a conditional DirPMINE estimator. Developing such an estimator substantial methodological undertaking and remains outside the scope of the present work.
52

---

Published as a conference paper at ICLR 2026
Algorithm 2: Training CBNP- VAECGAN
Input: Training data 1imi Riter iterations; Reiht discriminator iterations: RDB number of Discrete variables to be Balanced (DB) with respect to their categories; - iterations; Tibalance balance iterations; Pumb-mini-bate size, 10; learning rates:
OcG = -DD QG_ = E, OT, = o'Tii = 2c-,; penalty Xp = 10; Adam
optimizer parameters: (s le f1 0.9, 82 0.999).
Output: Optimized parameter of the generator
for t to Vljter do
Sample (Xi:nma'> Vl:mma $1:1mbb) F; Pos
Sample (xPox Y I:N+ IINN) ~ Foss
11 F ~ DP(a Rabby [
(xelor V1:N+ $1:N x- (xfor I 1 s
Por
1/ Apply QT for continuous
one one-hot for discrete columns
for 1 Merit do
f1:N ~ [Opp, (p): ELN (0)
C1.N t- CGw (S1,N), C1:N & E (NN (x vPo)). LiNN <- (EI.N, CI.N, CI:N);
(Xi:N y t Gu spor); sINN)
Update discriminator parameters:
10
0 < Adam(Vo
VoC(Do sPas') ) 0, e1, 82);
11
Update generator & encoder parameters for utility: Repeat lines 6-8;
(w, 77) + Adam Vw E, s (w, m), e, B1, 82); Update MI-DirPDV parameters to estimate MI:
13
15
Repeat lines 6-8;
< Adam(- Mlamm-VyLFann VaLFair
Pos' ), 1, B2);
17
Update KI.-DirPDV parameters to estimate KI.:: Repeat lines 6-8;
19
for ip = 1 to NDB do
Vin < Adam(-Vo -V Baaaacce Te., Ps E, (2)
20
21
Update code-generator:
22
CI.N & CGw (Ei.N);
w' +
L(CG P w', 32)
23
24
Update generator for Fairness:
25
if't Plair > 0 then
26
Repeat lines 6-8;
w + Adam (C En s + XpFFoor (Gw: ,,, B1.
27
28
Update generator for class balance:
29
if Tbalance > 0 then
30
Repeat lines 6-8; for in 1 to NDR do
w - Adam (L En s s
32
AB. L Batance (Gw Ter, ( w. $1, 01.33))
33
53

---

Published as a conference paper at ICLR 2026
Pos
VAE
MI DPDV
MI
Generator
Code Generator
Decoder
Encoder
D KL
KL
(X Pos POS)
Pos
KL
Pos
MI
YPos
SPos
Pos D
1:N
D Ray(C)
D Ra(())
D
Po(D)
I:N.1
D
Pos(D)
2: D
Poa(DD PIN.ND 11
Pos(D)
D
DIN,ND
Pos
Pos
SPos
Figure 21: General diagram of the proposed Sannte-Fai-BBllance framework within BNPL, imple- mented using VAECGAN, member of the family of generative models with generator/decoder structure.
54


Reviews:
[{"review_id": "nYHf8a5DZ4", "reviewer": "Reviewer_6kYv", "summary": "This paper proposes an approach for generating fair and private tabular data by integrating a conditional generator within the framework of Bayesian non-parametric learning. The main objective of the approach is to be able to generate synthetic data that is private, fair and balanced by leveraging a mutual information regularization term which is conditioned on the protected attributes.", "questions": "Please see the main points raised in the weaknesses section.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "-The paper is well-written and the authors have clearly reviewed the challenges associated with the generation of synthetic tabular data. The main contributions are clearly summarized and the outline of the paper is clearly described.\n\n-The proposed approach combines in an innovative way the Dirichlet process together with differentially-private mechanisms such the analytic Gaussian mechanism and randomized response to produce a new differentially-private generative model. One of the strength of the approach is can be combined with a wide range of generator-decoder models. For instance, in the current version of the paper it is implemented through a generative adversarial network combined with variational auto encoder.\n\n-Detailed investigations have been conducted on the Adult and Compas dataset demonstrating that the model is able to generate high quality data that is both privacy-preserving and fair.", "weaknesses": "-While the literature review surveys a wide range of approaches for privacy-preserving, fair or balanced tabular data generation there are no mention of existing works at the intersection of these domains such as for instance :\n-David Pujol, Amir Gilad, and Ashwin Machanavajjhala. 2024. PreFair: Privately Generating Justifiably Fair Synthetic Data. In Proceedings of the VLDB Endowment (PVLDB), Vol. 16. https://www.vldb.org/pvldb/vol16/p1573-pujol.pdf\n-Sarmin, F. J., Rahman, A. R., Henry, C. J., & Mohammed, N. (2025). Privacy-Preserving Fair Synthetic Tabular Data. arXiv preprint arXiv:2503.02968.\nAdditionally, how the proposed approach builds on the Dirichlet mechanism from Gohari el al. 2021 should be further clarified. \n\n-The privacy budget considered are quite high and should be better justified. The privacy analysis of the approach should also integrate an analysis of the success of the privacy attacks such as membership inference to be able to assess empirically the strength of the approach provided. \n\n-One of the limit of the approach is that for now it seems limited to the statistical parity fairness metric and there is no discussion if it could easily be extended to integrates other group fairness metrics.\n\n-The figures 2 and 3 are not really visible and should be improved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iCeMUOicKZ", "reviewer": "Reviewer_194m", "summary": "The paper proposes a conditional Bayesian nonparametric (BNP) framework for tabular data synthesis that aims to jointly enforce privacy, fairness, and class balance. Privacy is introduced by resampling from a Dirichlet-process posterior (a finite approximation of DP(a, H)), yielding a distributional randomization that the authors formalize as a Dirichlet mechanism with a DP guarantee on the posterior weights. Fairness is enforced by minimizing dependence between outcomes and protected attributes via a mutual-information regularizer based on a BNP variant of the Donsker–Varadhan lower bound (“DirPMINE”). Class balance is handled by conditioning the generator on protected attributes and adding a KL-to-uniform balancing term for other discrete columns. Experiments on Adult and COMPAS suggest improved fairness/utility trade-offs relative to FairGAN/DECAF while supporting non-binary sensitive attributes.", "questions": "1. Proposition 1 defines adjacency on the probability simplex for posterior weights. How does this mapping correspond to standard record-level neighboring datasets in DP, and what guarantee applies to the atoms/locations vs. just the weights?\n2. When combining the global Dirichlet mechanism with local per-attribute privatization, what is the resulting epsilon delta after composition?\n3. Is it possible to have a plot to draw the privacy–fairness–utility curves for the trade-off?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "This paper presents a unified objective that couples utility with MI-based fairness and KL-based balancing, all inside a single conditional generator (VAECGAN), which is practical for tabular data.", "weaknesses": "1. I think there a few prior works that mention that DP-SGD can disproportionately harm minority groups via clipping + noise. But still, I don't fully understand what is the extra challenge when we consider both privacy and fairness at the same time. It would be better if the authors could address the interaction between privacy and fairness in the up front. In practice, enforcing fairness requires accurate group-conditioned statistics which DP then perturbs and budgets, especially hurting small groups; meanwhile fairness constraints can further tighten an already noisy optimization. So it’s unclear whether the proposed coupling makes the joint problem harder or sometimes acts as a helpful regularizer. To me, this paper feels like a combination of two constraints, but it is not clear how these two constraints play together. Is there any lowerbound for canonical private and fair problems like generative modeling of gaussian distributions?\n\n2. (Minor) the fonts in the plots are hard to read.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "07Ez0Eu6WD", "reviewer": "Reviewer_PWAy", "summary": "Proposes a Bayesian–nonparametric pipeline for tabular synthetic data that jointly enforces (i) privacy via Dirichlet-process (global) and copula-based (localized) mechanisms, (ii) fairness via minimizing mutual information between outcomes and protected attributes, and (iii) class balancing by conditioning the generator on group labels; instantiated with a conditional VAE+GAN and evaluated on Adult and COMPAS.", "questions": "Please see weakness.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The authors unified objectives across privacy–fairness–balance. BNPL resampling + MI regularizer + conditional generation is a complete solution to DP + fairness.\n2. DP analysis for a Dirichlet mechanism and localized privacy via copula-based marginals are novel in this combination.", "weaknesses": "1. Fairness target & baselines. The method mainly targets statistical parity via MI; could you please justify this choice against equalized odds/opportunity, and have some head-to-head comparisons (or at least rank-correlations) to fairness baselines beyond DECAF/TabFairGAN/FairGAN? \n2. Scalability/“high-dim” claims. DirPMINE is proposed for scalable MI, yet there’s no analysis of variance, sample complexity, or failure modes versus standard MINE. It would be better if authors could discuss more about runtime/memory, or stability across hyperparameters, and add confidence intervals for MI/MMD/utility.\n3. Some simple examples/simulation for intuitive insights. It would be clearer if there were any toy experiment showing: (i) how DP resampling alters the empirical distribution, (ii) how localized privacy trades utility vs. protection per column, (iii) how class balancing alone affects SP/MI vs. fairness-regularization alone. The current figures hint at effects but don’t isolate mechanisms.\n4. Datasets. The authors provide two datasets. However, I feel these two choices are rather too limited. In recent years, there've been multiple studies discussing the limitations of these two datasets (https://arxiv.org/abs/2108.04884 and https://arxiv.org/abs/2106.05498). It's more convincing if there are more datasets considered or include more discussion about the limitations of these datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)