Beyond the convexity assumption: Realistic tabular data generation under quantifier-free real linear constraints
Authors: Mihaela C. Stoian
Eleonora Giunchiglia
Venue: ICLR 2025 Poster (2025)
Keywords: tabular data generation
neuro-symbolic AI
informed machine learning
safe AI
Primary area: neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)

Abstract: Synthetic tabular data generation has traditionally been a challenging problem due to the high complexity of the underlying distributions that characterise this type of data. Despite recent advances in deep generative models (DGMs), existing methods often fail to produce realistic datapoints that are well-aligned with available background knowledge.
In this paper, we address this limitation by introducing Disjunctive Refinement Layer (DRL), a novel layer designed
to enforce the alignment of generated data with the background knowledge specified in user-defined constraints.
DRL is the first method able to automatically make deep learning models inherently compliant with constraints as expressive as quantifier-free linear formulas, which can define non-convex and even disconnected spaces. 
Our experimental analysis shows that DRL not only guarantees constraint satisfaction but also improves efficacy in downstream tasks. Notably, when applied to DGMs that frequently violate constraints, DRL eliminates violations entirely. Further, it improves performance metrics by up to 21.4\% in F1-score and 20.9\% in Area Under the ROC Curve, thus demonstrating its practical impact on data generation.

Published as a conference paper at ICLR 2025
BEYOND THE CONVEXITY ASSUMPTION: REALISTIC TABULAR DATA GENERATION UNDER QUANTIFIER-FREE REAL LINEAR CONSTRAINTS
Mihaela Cätälina Stoian
Eleonora Giunchiglia
University of Oxford
Imperial College London g wiggchhigggii imperial ac.
mihaela. stoianlcs. ox. ac.
ABSTRACT
Synthetic tabular data generation has traditionally been challenging problem due to the high complexity of the underlying distributions that characterise this type of data. Despite recent advances in deep generative models (DGMs), existing meth- ods often fail to produce realistic datapoints that are well-aligned with available background knowledge. In this paper, we address this limitation by introducing Disjunctive Refinement Layer (DRL), a novel layer designed to enforce the align- ment of generated data with the background knowledge specified in user-defined constraints. DRL is the first method able to automatically make deep learning models inherently compliant with constraints as expressive quantifier-free lin- ear formulas, which can define non-convex and even disconnected spaces. Our experimental analysis shows that DRL not only guarantees constraint satisfaction but also improves efficacy in downstream tasks. Notably, when applied to DGMs that frequently violate constraints, DRL eliminates violations entirely. Further, it improves performance metrics by up to 21.4% in F1-score and 20.9% in Area Under the ROC Curve, thus demonstrating its practical impact on data generation.
INTRODUCTION
The problem of tabular data generation is critical area of research, driven by its numerous practical applications across various domains. High-quality synthetic data offers solutions to pressing chal- lenges such as data scarcity (Choi et (017)) bias in unbalanced datasets (van Breugel al. 2021). and the general need for privacy protection (Lee et al. [022]). However, due to the varied nature of the data distributions in the tabular domain wwhich are often multi-modal, and present complex de- pendencies among features-it is difficult to create models able to generate realistic data. Indeed, no matter the Deep Generative Model (DGM) used, when synthetic datapoints are tested for alignment with the available background knowledge, they frequently fail such test. Even when considering simple knowledge like "the feature representing the maximum recorded level of hemoglobin should be greater than equal to the one representing its minimum". DGMs often generate datapoints vio- lating it. So far, this problem has only been solved by either rejecting the non-aligned samples, or by adding a layer to the DGM that restricts its output space coincide with the one defined by linear inequalities (Stoian et al. 20244.. However, while the first solution is not feasible in the presence of high violation rate, the second is only available when the knowledge can be captured by linear inequalities, which have very limited expressivity.
In this paper, we propose a novel layer- called Disjunctive Refine- ment Layer able to constrain any DGM output space ac- cording to background knowledge expressed as Quantifier-Free Lin- ear Real Arithmetic (QFLRA) formulas. QFLRA formulas can cap- ture any relationship over the features that can be represented as a combination of conjunctions, disjunctions and negations of linear in- equalities. Thanks to their expressivity, QFLRA formulas can define spaces that are not only non-convex but also be disconnected. On the contrary, linear incqualities can only capture convex output spaces. See Figure for an example of spaces defined by linear
(b)
Figure 1: Example of spaces defined by (a) set of linear inequalities and (b) a set of QFLRA formulas.

---

Published as a conference paper at ICLR 2025
inequalities and QFLRA formulas. While linear inequalities establish single lower and upper bound (if existent) for each feature, QFLRA formulas define multiple intervals where the back- ground knowledge holds, each with its own boundaries. This significantly increases the complexity of the problem, as compiling knowledge into DRL not only requires keeping track of these intervals but also deriving the intricate hidden interactions among variables.
Example 1. The knowledge: "The value of wz, should be always at least x1, and if greater than 22 then it should also be at leust equal to x3. In any case, I5 should never be greater than v4", which cannot be expressed by a set of linear inequalities, corresponds to the QFLRA formula:
(xs 2 x1) A ((#3 > x22) + (as 2 xa)) A (Ig S xA).
(1)
Moreover, this formula entails other hidden relations among the variables such as, e.g., -(11 > xA).
To derive such additional hidden relations, developed novel variable elimination method which generalises the analogous procedure for systems of linear inequalities based on the Motter-nMttzinn result (see, e.g., (Dechter 1999)). Once compiled, by definition. DRL (i) guarantees the satisfaction of the constraints. (ii) can be seamlessly added to the topology of any neural model, (iii) allows the backpropagation of the gradients at training time, (iv) performs all the computations in a single forward pass (i.e., no cycles), and (v) given a sample generated by DGM, it returns new one that is optimal with respect to the original (intuitively, which minimally differs from the original sample while taking into account the user preferences which features should be changed first). Our experimental analysis also shows that adding DRL to DGMs improves their machine learning efficacy (Xu et al.] (2019) on range of different scenarios. This is the most widely used performance measure for evaluating the quality of synthetic data, as it assesses how useful the generated data is for downstream tasks. In partieular, we considered five DGMs, added DRL into their topology and got improvements for all datasets of up to 21. 21.4%, 20.5%, and 20.9% in terms of F1-score, weighted F1- score, and Area Under the ROC Curve, respectively. Finally, our experiments demonstrate a strong need for a method like ours. Indeed, DGMs generate synthetic datapoints violating the background knowledge more often than expected. In 13 out of 25 scenarios, the DGMs produced datasets with over 50% datapoints violating the constraints, and in five cases this reached 100%.
Main contributions: (i) We propose the first-ever layer that can be integrated into any DGM to enforce background knowledge expressed as QFLRA formulas. This required generalising the Mourier-Motzkin variable climination procedure in order to handle disjunctions of linear inequalities. (ii) We show experimentally how integrating our layer in DGMs improves their machine learning efficacy, even when the constraints define a possibly non-convex and disconnected space.
2 PROBLEM DEFINITION AND NOTATION
Constrained generative modelling is defined as the problem of learning the parameters @ of gener- ative model, given an unknown distribution px over X R.D, training dataset DD consisting of N i..d. samples drawn from Px, and formally expressed background knowledge about the problem- stating which samples are admissible and which are not-such that (i) the model distribution pa approximates Px. and (ii) the sample space of pe is aligned with what is stated in the background knowledge. As described in the introduction, so far this problem has only been solved by either re- jecting the non-aligned samples by including layer into the DGM that restricts its output space to coincide with the one defined by the linear inequalities (Stoian et al. 2024
In this paper, we allow for background knowledge expressed as a set of formulas, each being a dis- junction of linear inequalities. This enables us to capture any relationship among features which can be represented as Quantifier-Free Linear Real Arithmetic (QFLRA) formulas. Indeed, through syn- tactic manipulation using De Morgan's 's laws and the mathematical properties of linear inequalities, any knowledge formulated as combination of conjunctions, disjunctions, and negations of linear inequalities can be rewritten as a set of disjunctions over linear inequalities. Formally, we consider set II of constraints, where a constraint is disjunction of n N linear inequalities of the form:
V= = Φ V Φ2 V...
... V In
(2)
where each T; is a linear inequality over the set of variables X' = (IL 1: = 1,..., D), each variable uniquely corresponding to a feature in the dataset. We assume each linear inequality has form:
Wk + b 2 0, Σ
(3)
2

---

Published as a conference paper at ICLR 2025
with wa E R. b E R, and zy ranging over R. When w, * 0, say that 2; occurs in (3), and that it occurs positively if w/ > O and negatively otherwise.
For an easy formulation of the problem and its solution, given two linear expressions
Ex Wyat - b and sp' = Ex Wg,Tg + b', we write, c.g., (f + x²) for the linear expression Lu(wx + )a + (b+ 8), and similarly for (9- and up/w: if w E IR (0). We will also express the linear inequality (3) as w;z; + p, by this implicitly assuming w; F 0 and that z; does not occur in se, i.e. that @ Lxzs wi-k + b. Finally, we also write ip & (resp. & 4/) as abbreviations for 4 4 2 0 (resp. 9 p 0).
Given DGM with distribution po, sample & ~ is an assignment to the variables in X, and ix indicates the value assigned by e to the variable zy.- We say that sample ă satisfies
the linear incquality 3 if >R waix + +6200 O,
the constraint with form 2 if Φ; is satisfied by & for some = 1,..., n. and
set II of constraints if ã satisfies all the constraints in II.
Further, we associate each linear inequality t, (resp. constraint V, resp. set of constraints II) the set (()) (resp. (2VV), resp. ?(II) of the points in IRD that satisfy & (resp. V, resp. II). Clearly, ( (VV) and (ITI) define a subspace R.D, and have the following properties:
1. ()()) is non-empty and convex.
2. ()) is non-empty but may be non-convex and also disconnected, and
3. ((TT) may be empty, non-convex and also disconnected,
all the above assuming some variable occurs in 6, V and II. linear inequality (resp. a constraint V, resp. set of constraints II) is violated by sample v if H does not belong to the corresponding set ((2) (resp. ()) resp. ((1)).. A linear incquality & (resp. a constraint V resp. set constraints II) is satisfiable if the corresponding for the sake of simplicity, we do not consider strict inequalities (i.e., inequalities with >). From a theoretical perspective, the entire theory can be easily generalised to consider them. From a practical perspective, for any computing system of choice, we can simply rewrite each strict inequality of the e O, where > denotes the desired
()) (resp. (2( resp. (((TI) is empty. Notice that,
following form: Σ. Wx.FK precision of the representation. taking into account the limitations of floating- we represent each constraint v E II of form (2) also as the set { .2..... 2ng }. With this notation, IT is set of sets of linear inequalities. Hence, the linear inequalities in set should be interpreted as disjunctively defining constraint in II, while the constraints are to interpreted as conjunctively defining II.
0 as Σn Wk-Fk
accuracy. Finally,
3 DISJUNCTIVE REFINEMENT LAYER
Given a finite set of constraints II and a DGM, we show how to build a layer with all the desired properties stated in the introduction. In Appendix[A]we visualize how add our DRL each of the DGMs considered in the experimental analysis. Before illustrating the general case, in the following subsection we assume II is finite set of constraints in single variable T;.
3.1
SINGLE VARIABLE CASE
Each constraint V of form the variable F;
2 defines a single left boundary 1 and a single right boundary " for
max (1022 + (200)2000ss
40
v (w.a,+
min
(4)
w,
W.
Assuming V contains a linear inequality in which w; * 0, a sample ă satisfies if and only
either ã, < 18 or 2; 2 T. represented in Figure 2 As the Figure clearly shows, S2(VV) is already non-convex whenever 1, -x. r + +oo and 1.0 < <. When considering a set II with multiple
'We use the "left" and "right" terminology because in non-vacuous constraint we have 1 < r. We assume the function min(S) over finite set S of values in IR to be defined as min(0) +00, and min({u) 8') if v S min(S') and min(S') otherwise. Analogously for the function max
3

---

Published as a conference paper at ICLR 2025
constraints in 2i, we may arrive at set (I(II) that is the union of up
to [II] + disjoint intervals.
(7) (((0)
In general, computing the intervals requires finding the satisfying boundaries and then ordering them. Luckily, given a sample ă violat- only interested in setting DRL(
Figure Visualisation of left and right boundaries de- fined by constraint V The green regions correspond to the values of F; E (VV)).
ing some constraint in II, equal to the bound that satisfies and that is at minimal Euclidean distance from ã;. To this end, we first define the closest satisfying left and right boundary for T; as:
(2)
max wel
(1*" b. 12,47 E 2(II))).
wel (fr
r (I(I))), (5)
respectively. Then, for k + i, (RL(F)+
3 and if I(II,, if E Q(TT)) and L - (.) < 1 - -.)
5;
DRL(#); =
2T (x)
(6)
(T)
otherwise.
By construction, (RI(() satisfies the constraints in II and is optimal w.r.t i: there does not exist sample satisfying II with smaller Euclidean distance from E.
Lemma 3.1. Let II be a finite and satisfiable set of constraints a single variable I₁. For every sample ã. DRL(E) satisfies II and is optimal w.r.t. ã.
The proof of the Lemma can be found in Appendix B
Example 2. Let 11 be the set of constraints { 1, 22 V33 over the unique variable w5, with 11 2 and V as shown in Fig- ure Then, 15 = -oo,15 1 = b, 5
W:-
d, 5 = a, a,r5 U2
c, rs +o0. Depending on the value of S.s we get correspond- ingly different values for DRL( In particular,
Figure 3: Constraints for Ex.
1. if 25 < then DRL( 0,
4. if (b c0/2 ã5 < c then DRL(3),
2. ifa S āo S b then DRL(T)s = Σz.
5. if S ão S d then DRL( = āz.
3. b < ï5 < (b + c)/2 then DRL(T)s b,
6. ãs > d then DRL( = d.
Independently from the value of T5, DRL( satisfies the constraints and is optimal w.r.t. DRL(ā)s.
3.2 GENERAL CASE
Among the desiderata for our DRL, we have that all the necessary computations need to be done in a single forward pass. To this end, we consider a variable ordering æ1; -.; Jp corresponding to the order of computation of the features. The ordering can be arbitrarily selected or, more appropri- ately, may reflect the user preferences on which features should be changed first when the sample violates the constraints. Indeed, the value of each feature I; will be computed taking into account the values of the features 21,.... -; the latter considered immutable. To make this possible, when building the layer, we need ensure that the chosen value for the variables I,, with i, guaran- tees the existence of value for xæ satisfying the constraints. Starting from II II and = D, this amounts to deriving finite set I1 of constraints in the variables æ #2, Zi-1 whose conjunc- tion is logically equivalent to 3r, Awen,
V. This entails that for every value of T2, -4 satisfying 11,-1 there must exist a value for 2; satisfying II, or alternatively, that each assignment to I1, T22 ... - and satisfying 11,-1 can be extended to satisfy also 11,.
In order to define such set II;-1, given two constraints
(Vi= (wwa; + 9x Z 0) V +) and V = (V (wfa + 4// 2 0) V &¹), with w(...., w < 0 < wi, wi,..., wn and m, n 2 1, we define
the cutting planes (CP) resolution rule between & and y on H4 to be:
ソ (w,,; + b 2 0'
Vh (wbr; + Pk 4
T k (p4/wn -///// VV
In the above rule, v and V' are the premises, and the formula below the line is the conclusion denoted with CPres (V, V'). This rule, which can be derived from the standard propositional and
4

---

Published as a conference paper at ICLR 2025
CP rules defined, e.g., in (Krajicek) 1998), is sound for any possible + and Φ'. Despite this, assume that &' (resp. ②) does not contain negative (resp. positive) occurrences of 24. As we will sec, it is possible to impose much stronger conditions (defined later) the applicability of the rule, still enabling the derivation of a set of constraints IT,-1 with the desired properties.
Lemma 3.2. The CP resolution rule sound: the premises entail the conclusion of the rule. Example 3 (Example cont'd). The QFLRA QFLRA,ormull in the introduction translates into the of constraints: II = { 22 Vz} with 1 = (æs 2 I1), V2 = ((x5 I x2) V (Is 2 zz)) and 33 (ag I4). By applying the CP resolution rule, we can obtain new set of constraints entailed by II and logically equivalent to Ars Awen" V.
CPress(V ( V2) = 12 V vs 2 23
CPress(V) V3) = S I44
CPress 33 CPress 1, v22)
= x1 S 22 V I3 S xs-
As derived from the multiple application of the CP resolution rule, the above set of construints admits solution for Iı if and only if (x1 S 2x) A (a1 I D2 V S aA)-
The proof of Lemma 3.2 is in Appendix C In the example, there is only one constraint with both positive and negative occurrences of x; and the CP resolution of any distinct constraints always leads to a conclusion with either only positive or negative occurrences of z;. However, in general, the CP resolution of two constraints and V will lead new constraint CPres; (V, V which might contain both positive and negative occurrences of I;, This new constraint can be the premise of other CP resolutions which can produce new constraints and the process can iterate. Nevertheless, our goal is to derive the constraints in the variables æ1,- ......., Ii-1 whose satisfying assignments can be extended to satisfy also the constraints with z;. The standard solution to make all the possible CP resolutions I; while considering also the CP resolvent of the already done resolution may turn out to be too computationally expensive. Luckily, we can further restrict to CP resolutions between two constraints V and V in which T does not contain negative occurrences of 24. To this end, let
1. 11 (resp. 11;7) to be the set of constraints in II; with (resp. without) positive occurrences of 2; and without (resp. with) negative occurrences of 2;
2. IIF to be the set of constraints in πI; with both positive and negative occurrences of 2;;
3. II to be the set of constraints obtained by the recursive application of the CP-resolution between one constraint without negative occurrences of z, and one constraint in II
m
T
with m++ (CPres: (V, V/ v E M V e m
k=
and I1 II." Every constraint in II has only positive occurrences of xy.
Then, I1,1 is the set of constraints in II, in which x; does not occur plus the set of constraints obtained by the CP resolution of the constraints in IT UI, UNE. More formally,
II = (II, \ (ITF UIT (TT))) U ( (CPres; (V, V') v E IT,, V E II
(8)
Clearly, each set I1;-1 does not contain any occurrence of x; and can contain a non non-polynomial number of constraints, the latter fact echoing similar results for variable elimination methods in propositional logic and sets of linear inequalities (Dechter) 1999). The above definition, generalises to disjunctions of linear inequalities the standard variable elimination procedure proposes for sys- tems of linear inequalities based on the Sourier-Motzkin result.
Example (Example 3. cont'd) '). Consider the variable ordering æ 222 03t I4 Is. Then, II; =
I1, 115 = (V3). = (V2) 11g = ng (V, 113 {x1 < x2 Vas 2 x3). and n ng UI
As a consequence, II, = {01 < F1, I1 x2 v x3 I x1), and IIg = Il, = 1I,
For each set of constraints II, the set I1,-1 has the desired property. stated in the lemma below.
Lemma 3.3. Let II be set of constraints in the variables Z1,..., Zt. II, II and II, are equisatisfiable and each assignment to the variables Z1, ; satisfying II,-1 1 can be extended in order to satisfy II
5

---

Published as a conference paper at ICLR 2025
Algorithm 1 Compile & Apply DRL
function COMPILEL,,, 21;...;;00
function DRL_APPLY( 11 1I
1 < II for i + D downto 1 do
for - to D do compute I1, _ii 1 (2), r (. (i) if c; S2(TT,) then DRL( 8;
compute IT II, DIF I 1T,11 (TI,' (IT, U )
else if [4 - 1 (a) < - (i) then
{CPres, P' E v e
DRL( + !! (i)
if To is unsatisfiable then return WUSAT_FLAS else return II :l;
else DRL( + r. (7)
return DRL 12.. (R)
The proof of the Lemma is Appendix D As a corollary of the above lemma we have that the CP resolution is refutationally complete: if is unsatisfiable then it is possible to derive disjunction of linear inequalities in Il in which each inequality (3) has w, = for = I...., D and b < (otherwise, it is possible to incrementally define assignments satisfying IT, I12,.... ...... T IT). Thus, at the end of the layer construction, we are able to automatically detect unsatisfiability, returning a corresponding value in such case.
Corollary 3.4. For any finite set of constraints, the CP resolution rule is refutationally complete.
Starting from 1, the value of DRL( ; computed considering the constraints in II, where II, is the set of constraints in the variable z; obtained by substituting the variables I Z2.2.... 2i-1 with DRL( ( (ō) ...., ( in II, As in the single variable case, assuming H violates some constraint in II, we define the closest satisfying left and right boundaries for H, as:
t (ō)
max (14 2 19 C ((IL)))
( (ū) min (tr ã, ,r. ((IL))).
well,
Well.
Then, for j > i, ( = Zz. for < (R)( (RL( and i (II... ā, & s(LI.) and |ز+ (+)] u. (+)).
(R)
(G) (G)
otherwise.
The compilation step happens only once before training, while the application step is performed for each sample.
A simple, non-optimised version of the algorithm is given in Algorithm [
Example (Examples 2 cont'd), d). Consider a sample v where i = a, .2 = b, Z.. = c, and 34 d (i.e., arranged in Figure 3). Then, since IIg II, II, 0. DRL leaves the values unchanged for the features 21, 32, F3 and DRL(T) = a, DRL(#)2 = b, DRI.(T) = c. Regarding b x4 2 cf which reduces to {arA 2 a). and, since it is
Ls know that 1I4 {24 satisfied, DRI(Ø) d. Fimally, πs {xs 2 a, Mz < bv 2 c, d) and the value of (RI((5ss can be computed on the ground of ãs. as detailed in Example
Theorem 3.5. Let 1I be finite and satisfiable set of constraints. For any sample it and variable ordering, the corresponding sample DRL( satisfies 11.
Further, considering the variable ordering I1; ..: the variable ordering: for each 12,-73 F/ < | (RL()) and for all j < i, K3J DRL(
: ;DD, DRI(()) is optimal w.r.t. and and 1, ..... D there does not exist sample H' E ?(II) such that
Theorem 3.6. Let II be a finite and satisfiable set of constraints. For any sample i and variable ordering, the corresponding sample (RL(2) optimal w.r.t ž, II and the variable ordering.
The proofs of Theorems 3.5 and 3.6 are Appendix and respectively.
EXPERIMENTAL ANALYSIS
To assess how DRL performs in practice, conduct the following studies. First, in Section 4.1 we investigate whether our layer improves the quality of the synthetic data generated by standard
The code is available at /ttpp::://gtcchu..co///titti
stoi /

---

Published as a conference paper at ICLR 2025
6
Real
TTVAE
++LL
++DRL
Price (1e5)
5
3-
98004
98005
98006
98007
98008
98004
98005
98006
98007
98008
98004
98005
98006
98007
98008
98004
98005
98006
98007
98008
Zipcode
Zipcode
Zipcode
Zipcode
Figure 4: Sample distributions for real and synthetic data from TVAE, TVAE+LL and TVAE+DRL.
The regions where samples violate the constraints are in red.
we then compare our constrained models (which we refer to as DGMs+DRL) with the models obtained by considering only the linear constraints in each dataset and adding the layer proposed by (Stoian et al. (2024). We refer to the linearly constrained DGMs as DGMs + Linear Layer (DGMs+LL). Then, in Section 4.3 we conduct experiments to determine how the background knowledge injection affects the sample generation time. Before delving into these studies, we de- scribe the metrics we use to compute the sample quality, along with the models and datasets.
DGMs. In Section 4.2
Sample Quality Evaluation. To judge the quality of our samples we measure (i) how well they align with the background knowledge and (ii) how well they can replace the real data in downstream tasks. To measure background knowledge alignment, we consider the metrics proposed in (Stoian et al.) 2024]: i.e, constraint violation rate (CVR), constraint violation coverage (CVC), and samplewise constraints violation coverage (sCVC), To determine their usability in downstream tasks we consider the metric machine learning efficacy (Kim et al. 2023), also known as utility (e.g.. in (Liu et al,, To compute it, follow the "Train Synthetic, Test on Real" protocol (Esteban al. 2017 Specifically, to compute the efficacy for classification (resp., regression) datasets, we train six classifiers (resp., four regressors) on synthetic data and test them on real data. A detailed description of the evaluation protocol and the hyperparameter tuning description for the classifiers and regressors can be found in Appendix[] For classification datasets, report: F1-score (F1), weighted F1-score (wFI), and Area Under the ROC Curve (AUC), while for the regression dataset, we compute the mean absolute error (MAE) and the root mean square error (RMSE). For reference, we report the same metrics when training on the real data in Table 25 of Appendix O
2022
Models. We consider five DGMs: WGAN (Arjovsky al. (017). TableGAN (Park et al. 2018), CTGAN (Xu et al. 2019), TVAE (Xu et al. 2019), and GOGGLE (Liu et al. (2022). and build our DRL, on top of each to create DGM+DRL. models. description of these models Appendix H
Datasets. We consider five real-world datasets and associated constraints. Four datasets (i.e., URL, CCS, LCLD, and Heloc) are used for classification tasks, while one dataset (i.e., House) is used for regression. A detailed description of the datasets and their respective constraints are in Appendix G
4.1 SYNTHETIC DATA QUALITY
Background knowledge align- ment. To assess how often the samples violate the constraints, calculate the CVR, which is defined as the percentage of samples that violate least one constraint. Table [ shows the CVR for each unconstrained model (first five rows) and our models equipped with the DRL (last row). More detailed findings are reported in Appendix K (Table K). where the results for SCVC and CVC can also be found (Tables 9 10 As expected, the models with our DRL always satisfy the constraints, while the samples obtained with standard DGMs very often violate them. Additionally, in many cases, the CVR is extremely high: of 25 cases, 13 cases have CVR greater than 50% and 5 cases have CVR equal 100%, thus
Table 1: CVR for each model and dataset, Cases with CVR250% are underlined. Best results are in bold.
URL
CCS
LCLD
Heloc
House
WGAN TableGAN CTGAN
22.8=4.9
44.747.1
47.5+14.5
80.619.3
100.0#
8.5
61.2413.3
32.0=4.7
59.94 16.7
100.0+0.0
9.7=2.0
78.545.7
7.1-1.3
56.649.8
100.0+0.0
TVAE
10.3#1.1
16.9+1.6
10.3=0.6
1..9
100.0+0.0
GOGGLE
7.328.1
60.316.8
70.4
52.716.3
100.0:1.00
All + DRL
0.0=0.0
0.0+0.0
0.0=0.0
0.0+0.0
0.0+0.0

---

Published as a conference paper at ICLR 2025
Table Efficacy comparison between the unconstrained DGMs, and their +DRL and +RS counter- parts. The performance is measured using FI, wF1, and AUC, for each classification dataset.
F1
wF1
AUC
URL
CCS
LCLD
Heloc
URL.
CCS
LCLD
Heloc
URL.
CCS
L.CLD
Heloc
WGAN
0.794
0.303
0.139
0.665
0.796
0.330
0.296
0.648
0.870
0.814
0,605
0.717
+ RS
0.792
0.051
0.156
0.628
0.794
0.088
0.312
0.617
0.862
0.570
0.611
0.685
+ DRL
0.800
0.313
0.197
0.721
0.801
0.340
0.339
0.652
0.875
0.885
0.623
0.717
TableGAN
0.562
0.196
0.259
0.593
0.659
0.228
0.393
0.615
0.843
0.802
0.655
0,707
+ RS
0.544
0.138
0.251
0.568
0.648
0.172
0.389
0.599
0,854
0.682
0.653
0.685
+ DRL
0.619
0.163
0.269
0.628
0.693
0.196
0.401
0.628
0,865
0.742
0.657
0.709
CTGAN
0.822
0.145
0.247
0.736
0.799
0.159
0.379
0.675
0.859
0.914
0.651
0.744
+ RS
0.817
0.086
0.201
0.706
0.795
0.095
0.342
0.650
0.856
0.515
0.615
0.706
+ DRL
0.836
0.288
0.288
0.744
0.815
0.308
0.409
0.680
0.883
0.955
0.643
0.745
TVAE
0.810
0.325
0.185
0.717
0.802
0.351
0,330
0.686
0.863
0.858
0.631
0.750
+ RS
0.788
0.024
0.237
0.420
0.778
0.061
0.283
0.465
0.846
0.522
0,480
0.497
+ DRL
0.835
0.467
0.189
0.731
0.832
0.487
0.330
0.694
0.893
0.926
0.635
0.752
GOGGLE
0,622
0.039
0.248
0.596
0.648
0.076
0.296
0.566
0.742
0.549
0.551
0.600
+ RS
0.608
0.047
0.235
0.577
0.639
0.084
0.322
0.549
0.727
0.571
0.532
0.592
+ +DRL
0.720
0.253
0.298
0.698
0.673
0.281
0.310
0.636
0.747
0.758
0.563
0.691
making the standard procedure of rejecting non-aligned samples unfeasible. Further, to visualise the impact of DRL, we consider the features Price and Zipcode from the House dataset and create the scatter plots of (i) the real data, and the synthetic data from (ii) the unconstrained DGMs, (iii) the DGMs+LL, and (iv) the DGMs+DRL. We also highlight in red the regions that violate the constraints: (i) if the Zipcode is 98004 or 98005 then the Price is greater than 400K USD and (ii) if the Zipcode is between 98006 and 98008 then the Price exceeds 225K USD. The scatter plots obtained for the real datapoints and TVAE (with and without I.I. and DRL. are shown Figure 4 while the ones obtained from the other models are given in Figure 6 Appendix M. The Figures clearly show that standard DGMs and DGMs+LL fail to comply with the constraints, and indeed, many of the samples fall in the red-shaded regions. On the contrary, the samples obtained using DRL not only never violate the constraints, but also better match the real data distribution.
Machine Learning Efficacy. Table 2 shows that: (i) making the samples compliant with the constraints via rejection sampling (RS) often reduces their machine learning efficacy (indicated as DGMs+RS), and that (ii) adding DRL improves the performance of the unconstrained models according to at least one metric in all cases but one (TableGAN over the CCS dataset). Regarding the performance obtained with rejection sampling we can see that it decreases with respect to the standard DGMs in 17, 17 and 17 out of 20 cases for FI, wF1 and AUC, respectively. Regarding the performance of DGMs+DRL, the layer improves the performance w.r.t. the unconstrained models in 19, 18 and 17 out of 20 cases for F1, wF1 and AUC, respectively. Additionally, the improvements are often non-negligibl with the largest one recorded on GOGGLE for CCS of 21.4%. For wF1, in more than half of the cases, the improvement is of at least 1.0%, with the largest improvement, of 20.5%, again recorded on GOGGLE for CCS. And, for AUC, the improvement is at least 1.2% in half of the cases, with the largest improvement, of 20.9%, recorded on GOGGLE for CCS. On the regression dataset, House, we find similar trend in terms of improvements brought by DRL (Appendix Table [4), where the DGM+DRL models improve the performance w.r.t. the unconstrained models in all cases. We also verify the statistical significance of the results following the recommendation of (Demsar (2006). We perform the Wilcoxon signed-rank test on the efficacy results for the classification datasets and we obtain p-value < 0.01 w.r.t. the F1 and wF1 results and < 0.05 w.r.t. AUC, thus confirming that DRI. significantly improves the performances of DGMs.
For F1, in more than half of the cases, the improvement is of at least 3,5%,
4.2 LINEAR VS. QFLRA CONSTRAINTS
Background knowledge aligument. Table shows the CVR for each DGM+LL model (first five rows) and for the DGM+DRL models (last row). As expected, DGMs+LL cannot guarantee the
8

---

Published as a conference paper at ICLR 2025
Table Efficacy comparison between the DGM+LL models and the models with DRL. The perfor- mance is measured using FI, wF1, and AUC, for each classification dataset.
F1
wF1
AUC
URL
CCS
LCLD
Heloe
URL
CCS
LCLDD
Heloc
URL.
CCS
LCLD
Heloc
WGAN+LL
0,803
0.359
0.183
0,694
0.799
0.383
0.330
0,662
0.869
0.857
0.608
0.732
WGAN+DRL
0.800 0.313
0.197
0.721
0.801
0.340
0.339
0.652
0.875
0.885
0.623
0.717
TableGAN+LL
0.612
0.169
0.232
0,638
0.695
0.203
0.373
0,633
0.868
0.794
0.640
0.704
TableGAN+DRL
0.619
0.163
0.269
0.628
0.693
0.196
0.401
0.628
0.865
0.742
0.657
0.709
CTGAN+LL
0,836
0.250
0.265
0.729
0.820
0.271
0,392
0,688
0.880
0.959
0.641
0.755
CTGAN+DRL
0.836 0.288
0.288
0.744
0.815
0.308
0.409
0.680
0.955 0.883
0.643
0.745
TVAE+LL
0.824
0,413
0.158
0.730
0,816
0.436
0.310
0.691
0.878
0.933
0.633
0.747
TVAE+DRI.
0.835 0.467
0.189
0.731
0.832
0.487
0.330
0.694
0.893
0.926
0.635
0.752
GOGGLEALI
0.787
0.233
0.284
0.723
0.749
0.262
0.310
0.663
0.802
0.765
0.554
0.719
GOGGLE+DRI.
0.720
0.253
0.298
0.698
0.673
0.281
0.310
0.636
0.747
0.758
0.563
0.691
compliance with QFLRA constraints and in 5 out of 25 cases we see a CVR greater than 50%. Moreover, we have one case where CVR 100%, thus demonstrating the need for models that support more expressive constraints. In Appendix L Tables 15 16 metries: CVR, sCVC, and CVC.
we report the results for all
Machine Learning Efficacy. For the sake of completeness, in Table we include the comparison between DGMs+DRL and DGMs+LL on the classification datasets. Since Stoian et al. (2024) already re- ported improvements over their un- constrained counterpart by adding the linear layer, as expected in this scenario, we get more modest im- provements than the ones w.r.t. the unconstrained models. As we can see from the Table, the DGM+DRL models improve the efficacy w.r.t. the DGMs+LL for at least one metric in 17 out of 20 cases. Similarly, the number of times DGM+DRL outperforms the respective DGM+LL is lower than the number of times it outperforms its unconstrained counterpart. Indeed, out of 20 comparisons, the models with DRL outperfor their linearly constrained counterparts 13, 10 and 11 times for F1, wF1, and AUC, respectively. Regard- ing the regression dataset, House, Table 21 in Appendix [L shows that the DGM+DRL models have a comparable performance to the DGM+LL models, with 6 out of 10 the cases showing an improve- ment in performance when using our layer. As in the previous experiment, we use the Wilcoxon signed-rank test to assess whether adding DRL significantly improves over the linear layer. In this case, we obtain p-value 0.05 for F1, while as expected, the test confirms that the performances of DGMs+DRL and DGMs+LL are not statistically different w.r.t. wF1 and AUC.
Table 4: CVR for each DGM+LL model and dataset. Cases with 5VR~50% are underlined. Best results are in bold.
URL
CCS
LCLD
Heloc
House
WGAN+LL
8,9+3.2
51,5+11.2
27.0+3.6
20.646.3
100.0=0.0
TableGAN+LL
3.6+0.8
54.0=17.8
11.3+0.9
26.627.7
23.9=2.7
CTGAN+LL
7.0+2.6
55.7+16.3
2.6+1.1
2.6+24
10.8=7.8
TVAE+LL
6.810.6
8.412.0
5.810.8
0.0.0.00
13.0
GOGGLE+LL
6.517.0
23.0
81.916.5
11.517.1
2.6.2.6
All + DRL
0.0+0.0
0.0
0.0
0.0
0.0=0.0
4.3 SAMPLE GENERATION TIME
To assess the impact of constraints on sample gen- eration time, we compare the runtimes of uncon- strained DGMs, DGMs+DRL and DGMs+RS. We generate 1,000 samples for each model and dataset using five different seeds and report the average run- time in Table 5 (for detailed breakdown, see Ap- pendix N Table 22 As expected, DGMs+DRL are slower average than their unconstrained counterparts. However, they are faster than DGMs+RS. Indeed, excluding extreme cases with 100% CVR (where we were unable to generate samples even in 24h), in all other cases, DGMs+RS take more than twice long the unconstrained DGMs.
Table 5: Sample generation time in seconds.
URL.
CCS
LCLD
Heloc
House
DGM DGM+RS
0.15
0.08
0.07
0.06
0.05
0.37
0.83
1.54
0.66
DGM+DRL
0.22
0.13
0.14
0.10
0.13

---

Published as a conference paper at ICLR 2025
5 RELATED WORK
Our work lies at the intersection of two fields: Neuro-symbolic Al and tabular data generation. Thus, related work section will mirror this duality.
Neuro-symbolic AL. Neuro-symbolic AI (Raedt et al. 2020 d' Avila Garcez & Lamb 2023) refers to the broad area of AI that combines the strengths of symbolic reasoning with neural networks. As our work falls into the more specific field of injection of background knowledge into neural models,
(see, e.g., (Stewart & Ermon 2017, Hoernle et al.] 2022; Ghunchiglia et al., 2024a] Daniele et al.] 2023 Calanzone et al. (2044) we will focus the discussion on this topic. Many methods for this task are based on the intuition that logical constraints can be transformed into differentiable loss function terms that penalise the networks for violating them (see, e.g., (Xu et al. 2018 [adreddine et al.)
2022 Diligenti et al. [0122] Fischer et al. 2.19). As expected, since these methods operate at a loss level, they give guarantee that the constraints will be satisfied. Other works manage to integrate neural networks and probabilistic reasoning through the mapping of predicates appearing in logical formulae neural networks (Manhaeve et al., 2018 Yang et al., (020) [Sachan et al.) 2018: Pryor itt al. 2023 van Krieken et al.) (022). This allows these methods to both perform reasoning on the networks' predictions as well constrain the output according to the background knowledge. The most similar line of work to ours is the one where the constraints in input are automatically compiled into neural layers (Guunchiglaa & Lukasiewicz 2021 Ahmed al. 2022 Giunchiglia et al. 2024b). However, these methods can compile and incorporate constraints that are at best as expressive as propositional logic formulae. Focusing specifically on the incorporation of constraints on generic generative models, we can find the work by [Stoian et al. (2024), where the tabular generation process was simply constrained by linear inequalities. If we consider different application domains, we can find the work proposed by Misino et al. (2022). where ProbLog (Raedt et al. (007)) works in tandem with aatiiionallauttenoderss and the one by [iello et al.] (2020), where the authors incorporate propositional logic constraints on GANs for structured objects generation.
Tabular Data Generation. In recent years, various DGMs have been proposed to tackle the prob- lem of tabular data synthesis. Many of these approaches are based on Generative Adversarial Net- works (GANs), like TableGAN (Park et al. 2018). CTGAN (Xu et al. 2019), IT-GAN (Lee et al.) 2021), OCT-GAN (Kim et al. 2021), and PacGAN et 2018).
Other methods try to reduce the problems that often characterise GANs, such as mode collapse and unstable training. by introducing Variational AutoEncoders (VAEs) based models, see, e.g., (Xu et al., 2018 Srivastava et al., 2017 Wan et al.) 2017). An alternative solution to such problems is given by the usage of denoising diffu- sion probabilistic models as done in (Kotelnikov et al. 2023) or (Kim et al.] (023), where the authors designed self- learning method and fine-tuning approach to adapt the standard score- coore-based generative modeling to the challenges of tabular data generation. Finally, GOGGLE (Liu et al. 2022) uses graph learning to infer relational structure from the data and use it to their advantage especially in data-scarce setting. Since synthetic tabular data are often used to replace the original dataset to preserve privacy in sensitive settings, parallel line of research revolves around the devel- opment of DGMs with privacy guarantees. Examples of models that have such privacy guarantees are PATEGAN (Yoon et al. 2020) and DP-CGAN (Torkzadehmmhamí et al. 2020).
CONCLUSIONS
In this paper, we have proposed Disjunctive Refinement Layer (DRL) the first-ever Neuro-symbolic AI layer able to automatically compile constraints expressed as QFLRA formulas into a neural layer and thus guarantee their satisfaction. This sort of work is really needed in the tabular data synthe- sis field, as our experimental analysis shows that Deep Generative Models (DGMs) very frequently generate datapoints that are not aligned with the background knowledge, with some extreme cases where all the datapoints are violating the constraints. DRL presents many desirable properties: (i) it can be seamlessly integrated into the topology of any neural network, (ii) it allows the backpropa- gation of the gradients, (iii) it performs all the computations in a single forward pass (i.e., there are no cycles). (iv) it optimally refines the original predictions and, last but not least, (v) it improves the performance of all the tested DGMs in terms of machine learning efficacy. Indeed, in our experi- mental analysis we got improvements for all datasets of up to 21.4%, 20.5%, and 20.9% in terms of F1 F1-score, weighted Fc-rc,,,, and Area Under the ROC Curve, respectively.
10

---

Published as a conference paper at ICLR 2025
7 ETHICS STATEMENT
The development and application of synthetic data generation techniques, particularly in tabular data, have the potential to significantly impact a wide range sectors, including healthcare, fi- nance, and social sciences. While our method, Disjunctive Refinement Layer (DRL), improves the quality and fidelity of generated data by ensuring alignment with user-specified constraints, there ethical implications of synthetic data use. Firstly, there is the potential for misuse. Synthetic data may be seen as a substitute for real-world data, but it should not viewed as a perfect replace- ment. Secondly, the use of synthetic data in automated decision-making systems poses risks for fairness and bias. While DRL allows for the specification of constraints that align with real-world domain knowledge, it is important that the user-specified constraints do encode existing biases or discrimination.
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our results, included all the necessary details in the Appendix of the paper. The proofs of the Lemmas and Theorems can be found Appendices B C F] the detailed description of the datasets, the baseline models used (together with their links), evaluation protocol for the machine learning efficacy metric, and the chosen hyperparameters can be found in and
ACKNOWLEDGMENTS
Mihaela Cätälina Stoian is supported by the EPSRC under the grant EP/5517811/L. She has also re- ceived support for this work through the G-Research Women in Quant Finance Grant and St Hilda's ' College Travel for Research and Study Grant. We also acknowledge the use of the Advanced Re- search Computing (ARC) facilities of University of Oxford.
REFERENCES
Kareem Ahmed, Stefano Teso, Kai-Wei Chang, Guy Van den Broeck, and Antonio Vergari. Se- mantic probabilistic layers for neuro-symbolic learning. In Proceedings of Neural Information Processing Systems, 2022.
Martín Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein GAN. CoRR, abs/ 1701077777 2017.
Samy Badreddine, Artur d' Avila Garcez, Luciano Serafini, and Michael Spranger. Logic tensor networks. Artificial Intelligence, 303, 2022.
Vadim Borisov, Kathrin Sessler, Tobias Leemann, Martin Pawelezyk, and Gjergji Kasneci. Lan- guage models are realistic tabular data generators. In Proceedings of International Conference on Learning Representations, 2023.
Diego Calanzone, Stefano Teso, and Antonio Vergari. Logically consistent language models via seuro-symbolic integration. arXiv preprint 2xXiv:2409.13724. 2024.
Tianqi Chen and Carlos Guestrin. XGBoost: A scalable tree boosting system. In Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016.
Edward Choi, Siddharth Biswal, Bradley A. Malin, Jon Duke, Walter F. Stewart, and Jimeng Sun.
Generating multi-label discrete patient records using generative adversarial networks. Proceed- ings of Machine Learning for Health Care Conference, 2017.
David R. Cox. The regression analysis of binary sequences. Journal of the Royal Statistical Society: Series B (Methodological), 20, 1958.
Alessandro Danicle, Emile van Krieken, Luciano Serafini, and Frank van Harmelen. Refining neural network predictions using background knowledge. Machine Learning Journal, 1 112, 2023.
11

---

Published as a conference paper at ICLR 2025
Rina Dechter. Bucket elimination: A unifying framework for reasoning, Artificial Intelligence, 113, 1999.
Janez Demsar. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7, 2006.
Michelangelo Diligenti, Marco Gori, Marco Maggini, and Leonardo Rigutini. Bridging logic and kernel machines. Machine Learning, 2012.
Artur d' Avila Garcez and Luis C. Lamb, Neurosymbolic AI: The 3rd wave. Artificial Intelligence Review, 2023.
Cristóbal Esteban, Stephanie L. Hyland, and Gunnar Rätsch, Real-valued (medical) time series generation with recurrent conditional gans. CoRR, abs/1706.02633,, 2017.
Marc Fischer, Mislav Balunovic, Dana Drachsler-Cohen, Timon Gehr, Ce Zhang, and Martin Vechev. DL2: Training and querying neural networks with logic. In Proceedings of Interna- tional Conference on Machine Learning, 2019.
Eleonora Giunchiglia and Thomas Lukasiewiez. Multi-label classification neural networks with hard logical constraints. Journal of Artificial Intelligecce Research, 72, 2021.
Eleonora Giunchiglia. Fergus Imrie, Mihaela van der Schaar, and Thomas Lukasiewicz. Machine Learning with Requirements: a Manifesto. Neurosymbolic Al Journal, 1, 2024a.
Eleonora Giunchiglia, Alex Tatomir, Mihaela Catalina Stoian, and Thomas Lukasiewicz. CCN+: A neuro-symbolic framework for deep learning with requirements. International Journal of Ap- proximate Reasoning, 171, 2024b.
Abdelhakim Hannousse and Salima Yahiouche. Towards benchmark datasets for machine learning based website phishing detection: An experimental study. Engineering Applications of Artificial Intelligence, 104, 2021.
Simon Haykin. Neural networks: comprehensive foundation. Prentice Hall PTR, 1994.
Geoffrey Hinton. Lecture notes in neural networks for machine learning. 2014.
Tin Kam Ho. Random decision forests. In Proceedings of International Conference on Document Analysis and Recognition, volume 1, 1995.
Nicholas Hoernle, RachallMichael Karampatsis, Vaishak Belle, and Kobi Gal. MultiplexNet: To- wards fully satisfied logical constraints in neural networks. In Proceedings of Association for the Advancement of Artificial Intelligence, 2022.
Jayoung Kim, Jinsung Jeon, Jaehoon Lee, Jihyeon Hyeong, and Noseong Park. OCT-GAN: Neural ODE-based Conditional Tabular GANs. Proceedings of the Web Conference, 2021.
Jayoung Kim, Chaejeong Lee, and Noseong Park. STaSy: Score-based Tabular data Synthesis. In Proceedings of International Conference on Learning Representations, 2023.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proceedings of International Conference on Learning Representattons, 2015.
Akim Kotelnikov, Dmitry Baranchuk, Ivan Rubachev, and Artem Babenko. TabDDPM: Modelling Tabular Data with Diffusion Models. In Proceedings of International Conference Machine Learning. 2023.
Jan Krajicek. Discretely ordered modules a first-order extension of the cutting planes proof system. Journal of Symbolic Logic, 63(4), 1998.
Jachoon Lee, Jihyeon Hyeong, Jinsung Jeon, Noseong Park, and Jihoon Cho. Invertible tabular GANs: Killing two birds with one stone for tabular data synthesis. In Proceedings of Neural Information Processing Systems, 2021.
12

---

Published as a conference paper at ICLR 2025
Luca Di Liello, Pierfrancesco Ardino, Jacopo Gobbi, Paolo Morettin, Stefano Teso, and Andrea Passerini. Efficient generation of structured objects with constrained adversarial networks. In Proceedings of Neural Information Processing Systems, 2020.
Zinan Lin, Ashish Khetan, Giulia Fanti, and Sewoong Oh. PacGAN: The power of two samples in generative adversarial networks. In Proceedings Neural Information Processing Systems, 2018.
Tennison Liu, Zhaozhi Qian, Jeroen Berrevoets, and Mihaela der Schaar. GOGGLE: Genera- tive modelling for tabular data by learning relational structure. In Proceedings of International Conference Learning Representations, 2022.
Robin Manhaeve, Sehastijan Dumancic, Angelika Kimmig, Thomas Demeester, and Luc De Raedt. DeepProbl Neural probabilistic logic programming. In Proceedings of Neural Information Processing Systems, 2018.
Eleonora Misino, Giuseppe Marra, and Emanuele Sansone. VAEL:
Bridging Variational Autoen- coders and Probabilistic Logic Programming. In Proceedings of Neural Information Processing Systems, 2022.
Noseong Park, Mahmoud Mohammadi, Kshitij Gorde, Sushil Jajodia, Hongkyu Park, and Young- min Kim. Data synthesis based on generative adversarial networks. Proceedings of the VLDB Endowment, 11, 2018.
Connor Pryor, Charles Dickens, Eriq Augustine, Alon Albalak, William Yang Wang, and Lise Getoor. Neupsl: Neural probabilistic soft logic. In Proceedings of International Joint Confer- ence on Artificial Intelligence, 2023.
Luc De Raedt, Angelika Kimmig, and Hannu Toivonen. ProbLog: A Probabilistic Prolog and Its Application in Link Discovery. In Proceedings of International Joint Conference on Artificial Intelligence, 2007.
Lue De Raedt, Sebastijan Dumancic, Robin Manhaeve, and Giuseppe Marra. From statistical rela- tional to neuro-symbolic artificial intelligence. In Proceedings of International Joint Conference on Artificial Intelligence, 2020.
Sebastian Ruder. An overview of gradient descent optimization algorithms. arXiv preprint 1609000009 2016.
Mrinmaya Sachan, Kumar Avinava Dubey, Tom M. Mitchell, Dan Roth, and Eric P. Xing. Learning pipelines with limited data and domain knowledge: A study parsing physics problems. In Proceedings of Neural Information Processing Systems, 2018.
Robert E. Schapire. Explaining AdaBoost. In Empirical inference. Springer, 2013.
Thibault Simonetto, Salijona Dyrmishi, Salah Ghamizi, Maxime Cordy, and Yves Le Traon. A unified framework for adversarial attack and defense in constrained feature space. Proceedings of International Joint Conference on Artificial Intelligence, 2022.
Akash Srivastava, Lazar Valkov, Chris Russell, Michael U. Gutmann, and Charles Sutton. VEE- GAN; reducing mode collapse in gans using implicit variational learning. In Proceedings of Neural Information Processing Systems, 2017.
Russell Stewart and Stefano Ermon. Label-free supervision of neural networks with physics and domain knowledge. Proceedings of the Conference on Artificial Intelligence, 2017.
Mihaela C. Stoian, Salijona Dyrmishi, Maxime Cordy, Thomas Lukasiewicz, and Eleonora Giunchiglia. How Realistic Is Your Synthetic Data? Constraining Deep Generative Models for Tabular Data. In Proceedings of International Conference Learning Representations, 2024.
Reihaneh Torkzadehmahani, Peter Kairouz, and Benedict Paten. DP-CGAN: differentially private synthetic data and label generation. CoRR, abs/2001.09 099000, 2020.
13

---

Published as a conference paper at ICLR 2025
Boris van Breugel, Trent Kyono, Jeroen Berrevoets, and Mihaela van der Schaar. DECAF: gen- erating fair synthetic data using causally-aware generative networks. In Proceedings of Neural Information Processing Systems, 2021.
Laurens van der Maaten and Geoffrey Hinton. Visualizing Data using t-SNE, Journal of Machine Learning Research, 9(86), 2008.
Emile van Krieken, Thiviyan Thanapalasingam, Jakub M. Tomezak, Frank Van Harmelen, and An- nette Ten Teije. A-neSI: A scalable approximate method for probabilistic neurosymbolic infer- ence. In Proceedings of Neural Information Processing Systems, 2023.
Zhiqiang Wan, Yazhou Zhang, and Haibo He. Variational autoencoder based synthetic data gen- eration for imbalanced learning. In Proceedings of IEEE Symposium Series on Computational Intelligence, 2017.
Xindong Wu, Vipin Kumar, Ross Quinlan, Joydeep Ghosh, Qiang Yang, Hiroshi Motoda, Geof- frey J. McLachlan, Angus Ng. Bing Liu, S Yu Philip, et al. Top 10 algorithms in data mining. Knowledge and information systems, 14, 2008.
Jingyi Xu, Zilu Zhang, Tal Friedman, Yitao Liang, and Guy Van den Broeck. semantic loss function for deep learning with symbolic knowledge. In Proceedings of International Conference on Machine Learning. 2018.
Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante and Kalyan Veeramachaneni. Modeling tabular data using conditional GAN. In Proceedings Neural Information Processing Systems, 2019.
Zhun Yang, Adam Ishay, and Joohyung Lee. Neurasp: Embracing neural networks into answer set programming, In Proceedings of International Joint Conference on Artificial Intelligence, 2020.
Jinsung Yoon, Lydia N. Drumright, and Mihaela van der Schaar. Anonymization through data synthesis using generative adversarial networks (ADS-GAN). IEEE Journal of Biomedical and Health Informatics, 24, 2020.
14

---

Published as a conference paper at ICLR 2025
Diseiimaa
Encoder
Learabl Retatonal
Gauranr
DRL
DRL
DKL
(a) GAN-based models
(b) VAE-based models
(c) GOGGLE-like models
Figure 5: Visualisation of the considered types of DGMs and how add DRL in their topology.
A DISJUNCTIVE REFINEMENT LAYER VISUALIZATIONS
In Figure 5
we give an overview on how to add DRL in the topology of the three types models we considered. In all figures we indicate with noise vector, with real datapoint from the original dataset, with ž a sample generated with the DGM, and with (RL the final sample obtained from DRL. Considering each of the Figures,
that:
Figure 5a shows that DRL needs to be added top of the generator module in GAN-based models,
Figure 5b els, and
shows that DRL needs to be added after the decoder module in mod-
Figure
5c shows that DRL needs to be added after the generator module in GOGGLE-like models.
In general, we can that DRL can be added in many different DGMs, and it simply needs to be added right after the sample à is generated.
B PROOF OF LEMMA 3.1
Lemma. Let II be a finite and satisfiable set of constraints in a single variable rı. For every sample I. DRL(T) satisfies II and is optimal wrt ă.
Proof. We first prove that for every sample ã, DRL(2), always satisfies II, and then that for every
sample I, (T), the solution of II with minimal Euclidean distance from
Suppose there exists sample E such that DRI( & (I(TT). This entails (i) that IT + 0 and (ii) that & ((TT)) Since + 6 and IT is satisfiable, 2 (5) * -ao or ra) * +00, and DRI((E), = (*)
or (R)( = !!)) Since by definition 4 (7) and r (i) satisfy 11 we reached a contradiction.
Assume ā & ((III) (otherwise we would have again DRL( 7 and the thesis would trivially hold). Let be the minimum Euclidean distance between any point in SI(II) and Let r and / be the two samples with Fą 1g Es DRL((); when k * i and k E {1..... DJ. ra = ã; + d and 1, = 3. definition, E I(TT) and is optimal wrt G. Assume
Fither or or both belong to ((IT). Let be r if E 2(II), and I otherwise. By 1. Then, from the optimality of v, we have that for every o' with v/ E (1, Te + d), o' & Q(1I). Hence, there must exist constraint v such that 2 1." and thus U; DRL( (w), x. Analogously for the case
C PROOF OF LEMMA 3.2
Lemma. The CP resolution rule is sound: the premises entail the conclusion of the rule.
Proof. Consider the CP resolution nule
reported below for simplicity:
j=1 (
+ 0) s
Vi (Wba, + Sp 2 v
5 V (4k/
2)
15

---

Published as a conference paper at ICLR 2025
with w(,.... wn 0 Wi:..... W/ and m, n 2 1. We have to show that any model I of the premises is also a model the conclusion. Assuming satisfies the premises and not ( V Φ*) (otherwise the thesis trivially holds), it must be the case that:
min ((kk/((//))))) wk) and
S max j=1
where, given a linear expression sp, T(p) is the application of to s, i.e. the value obtained by replacing each variable xy with 3. in s. The above is possible if and only if
min ( wx) S mäx
j=1
i.e., there exist a pair (i, k) such that (- (px(///w)) S - /x;' and hence the thesis.
D PROOF OF LEMMA 3.3
Lemma. Let be a set of constraints in the variables Tı,..., Zz. II, = and II are equisatis- fiable, and each assignment the variables 21,..., - satisfying I1 be extended in order to satisfy II.
Proof. Clearly, given the soundness of the CP resolution rule, II, satisfiable, then also II;-1 satisfiable (each constraint in II;-1 and not in II, is entailed by II,).
remains to show that if 3 is an assignment to the variables Pi....,22: satisfying II;-1 the set of constraints / (TT,) is satisfiable. Similarly to the notation used in the proof of lemma 3.2 in Appendix C given set of constraints ТI, the expression z (*T(II) denotes the set of constraints the
variable v, obtained by substituting each variable xy (j < i) with the corresponding value 45 in the constraints II.
Assume F (II,) is not satisfiable. Then, there exist two constraints v and v/ in (II,) equivalent to (z, 2 r.) and (x; S 4t), respectively, and
1. either 4 ris
2. or 4 2 ri and there exists 2 constraints (V,, 22 Vn in (ll,))
with each
v equivalent to (æ; S v (a;
2 ri
wi) and I. p2=
1
vn
r r
W1
22
..... Pa such
that 1
< T
11 V2
v r
V < r I l
r
wn and thus
( ri <4
However, 4 < is not possible because CPres, U') belongs to 3 (II,-1) and is equivalent to (ri < 4.). Regarding the second case, F f(It)) contains the constraints = denotes logical equiva- lence)
T, CPres (V, 11) = (s4 r (1 (r. ) =
T2 = CPres 222 (z; 2 ri 22) v (r) 41 2 = x 2
r,= CPres; (Y, n- V = n;
and thus 3-t (II,1) contains CPres, V( = = < 4, thus reaching a contradiction.
E PROOF OF THEOREM 3.5
Theorem. Let IT be a finite and satisfiable set of constraints. For any sample and variable ordering, the corresponding sample DRL(ā) satisfies II.
Proof. We prove the statement by induction over the number n of variables appearing in IT.
Let 0. In this case II is satisfied by any sample and DRL(G) T.
16

---

Published as a conference paper at ICLR 2025
Let 1. Let 2; be the last variable in the ordering occurring in II. Since II contains (n 1) variables, DRL(ā) satisfies 11 by the inductive hypothesis. From Lemma 3.3 we know that 11, is satisfiable, and hence the thesis follows from Lemma 3.1
F PROOF OF THEOREM 3.6
Theorem. Let II be a finite and satisfiable set of constraints. For any sample t and variable ordering, the corresponding sample DRL(²) is optimal wrt i, II and the variable ordering.
Proof. We prove the statement by induction over the number n of variables occurring in II.
Let = O. In this case 1I is satisfied by any sample ã, and DRL(3) = T.
Let 1. Let T; be the last variable in the ordering occurring II. Since 11,-1 contains only the variables 21. 22,... I;-1, we know that for any sample ē, DRL(2) is optimal with respect to 2, 11 and the variable ordering for the inductive hypothesis. From Lemma 3.3 know that IT, is satisfiable. From Lemma 3.1 we know that for every I, DRL(2) is optimal wrt i and II, and hence the thesis.
G DATASETS
Below we provide brief description for each dataset and the links the pages where they can be downloaded.
URL (Hannousse & Yahiouche 2021 is used to perform webpage phishing detection with features describing statistical properties of the URL itself well as the content of the page.
CCS is used to identify individuals at high risk of cervical cancer from features describing the patients' demographic and medical history, including age, sexual behavior, contracep- tive use, and various medical test results.
LCLD is used to predict whether the debt lent is unlikely to be collected from features related to the loan as well as client history. In particular, we use the feature-engineered dataset from Simonetto et al. 2022). inspired from the LendingClub loan data.
HELOC is dataset from FICO used predict whether customers will repay their credit lines within 2 years from features related to the credit line and the client's 's history.
House F was used to predict the prices of houses in King County (USA) and contains data collected from May 2014 to May 2015. The features describe various features of the sold houses, including the date of sale, house prices, the number of bedrooms and bathrooms, square footage, condition, grade, year built, and location, among others.
For each dataset above, Table 6 shows the number of samples in the train, validation and test parti- tions, along with the number of features and the number of constraints. Regarding the constraints, they were already included in some of the original datasets. This is true for URL, Heloc and LCLD.. Regarding CCS and House, we manually annotated the constraints using our background knowledge about the problem, then we checked whether the data were compliant with our constraints and fi- nally retained only those constraints that were satisfied by all the datapoints. Simple examples of these constraints (from CCS) state simple facts like "if the feature capturing the number of cigarettes packs per day is greater than 0 then the feature capturing whether the patient smokes not should be equal to 1" or "the feature representing the age should always be higher than the age at the first intercourse".
Linkk to dataset:
$ *Link to
'Link to dataset: http:://gggaaccccoommmssccccccep
'Link to dataset:
Link to dataset: http://wwwww.aoooooooooooooooooo
17

---

Published as a conference paper at ICLR 2025
Table 6: Dataset statisties.
Dataset
# #Train
# Val
# Test
# Features
# Constraints
Task (# classes)
URL
7K
2K
2K
18
Binary classification
CCS
1K
0.02K
0.15K
36
12
Binary classification
LCLD
494K
199K
431K
29
16
Binary classification
Heloc
8K
2K
0.2K
24
12
Binary classification
House
17K
0.5K
4K
20
13
Regression
H
MODELS
Below we give a brief description of each of the models used in our experimental analysis:
WGAN (Arjovsky et al. 2017): it is GAN based model which has been trained by using the Wasserstein distance loss, which improves the stability of learning.
TableGAN (Par et al. 2018 is a GAN-based model designed for generating realistic tabular data. It uses a convolutional neural network (CNN) discriminator to better the capture dependencies among features.
CTGAN (Xu al., 2019) is again a GAN-based model which uses a conditional gener- ator to model feature distributions and applies mode-specific normalisation technique to improve the generation of imbalanced categorical data.
TVAE (u et al. 2019 uses variational autoencoder architecture to capture both continu- ous and categorical feature distributions, learning a probabilistic latent space representation of the data. By optimising the evidence lower bound, it balances reconstruction accuracy and regularisation.
GOGGLE (Liu 2022 ; uses a VAE framework combined with a graph neural network, which allows the model to learn complex feature relationships by representing the data as graph, where each node corresponds to feature, and edges capture dependencies between features.
I EFFICACY EVALUATION PROTOCOL
In order evaluate the efficacy of the models, we closely follow the protocol outlined in (Kim al. 2023 For clarity, we describe the protocol below.
First, we generate synthetic dataset and split it into training, validation, and test sets, maintaining the same proportions as in the real dataset. Next, we conduct a hyperparameter search using the synthetic training set to train various classifiers and regressors. Specifically, for the classification datasets (i.e.. URL, CCS, LCLD, Heloc), we use the following classifiers: AdaBoost (Schapire)
2013) Decision Tree (Wu al. 2008), Logistic Regression (Cox 1958) Multi-layer Perceptron
(MLP) Haykin 1994), Random Forest (Ho 1995), and XGBoost (Chen & Guestrin 2016). For the regression dataset (i.e., House), we use: Linear Regression, MLP, Random Forest regressors, and XGBoost. Across all classifiers and regressors,
use the same hyperparameter settings as those in Table 26 of (Kim et al., 2023). Then, based on the F1-score obtained on the real validation set, we select the best hyperparameter configuration. As last step, evaluate the selected models on the real test set and average the performance across all classifiers/regressors. The results for all models reported using three metrics (i.e., Fi-score, weighted F1-score, and the Area Under the ROC Curve) for the classification datasets, and two metrics (i.e., Mean Absolute Error and Root Mean Square Error) for the regression dataset.
We run the entire process five times for each model, then compute the average results for each metric individually across the repetitions.
18

---

Published as a conference paper at ICLR 2025
Table 7: Best hyperparameter settings used for DGMs (and also for DGMs+LL and DGMs+DRL) in our experiments.
Model/Dataset
Hyperparameter
URL.
CCS
L.C.D
Heloc
House
Batch size
510
256
510
510
510
Optimiser
Adam
Adam
Adam
Adam
Adam
Learning rate
0.001
0.001
0.001
0.001
0.0002
WGAN
Epochs
150
1250
15
150
100
Discriminator iters
5
LL Ordering
Corr
KDE
Rnd
Corr
Rnd
DRL Ordering
Rnd
Rnd
KDE
Rnd
KDE
Batch size
128
128
510
128
256
Optimiser
Adam
Adam
Adam
Adam
Adam
TableGAN
Learning rate
0.001
0.0002
0.01
0.001
0.0001
Epochs
300
2000
20
200
50
LL Ordering
Corr
KDE
KDE
Corr
KDE
DRL Ordering
KDE
KDE
KDE
Corr
Rnd
Batch size
500
70
500
500
500
Optimiser
Adam
Adam
Adam
Adam
Adam
CTGAN
Learning rate
0.0002
0.001
0.0002
0.0002
0.0002
Epochs
150
1000
20
500
150
LL Ordering
KDE
KDE
KDE
Corr
Rnd
DRL Ordering
KDE
Corr
Corr
Corr
Rnd
Batch size
70
70
500
500
70
Optimiser
Adam
Adam
Adam
Adam
Adam
TVAE
Learning
0.0002
0.0001
0.00001
0.000005
0.0002
Epochs
150
1500
40
150
150
Loss factor
2
4
2
LL Ordering
KDE
Rnd
Corr
KDE
Rnd
DRL Ordering
Rnd
Rnd
Corr
Corr
KDE
Batch size
128
32
128
64
64
Optimiser
Adam
Adam
Adam
Adam
Adam
GOGGLE
Learning rate
0.005
0.01
0.001
0.001
0.01
Epochs
1000
500
60
1000
400
Threshold
0.1
0,2
0,2
0.1
0,2
Patience
50
50
50
50
50
LL Ordering
KDE
Rnd
Rnd
Rnd
Rnd
DRL Ordering
Rnd
Rnd
Rnd
Rnd
Rnd
J HYPERPARAMETER SEARCH
We carried out an extensive hyperparameter search identify the optimal configurations for each DGM. We selected these configurations based on the efficacy performance: for the classification datasets (i.e., URL, CCS, LCLD, and Heloc), used the average of the Fl-score, weighted F1- score, and Area Under the ROC Curve (AUC), whereas for the regression dataset (i.e., House), we used the average of the Mean Absolute Error and Root Mean Square Error.
For clarity. we describe the hyperparameter search space below, for each of the considered models. For the GOGGLE model, we adopted the same optimiser and learning rate settings as in (Liu al. 2022). Specifically, we used the Adam optimizer (Kingma & Ba 2015) with five learning rates: x 10 10-3,5 x
10 10-2 Additionally, we experimented with a set of values for the threshold parameter: {1 x 10 2 10-1] J. For the TVAE model, Adam was used again, but with different set five learning rates: x 10 x 10 -5,05___ 10 4 5,2 x 10 4, x 10-3, 33 For the other three models (i.e.. WGAN, TableGAN, and CTGAN), tested three different optimisers: Adam, RM- SProp (Hinton 2014), and SGD (Ruder 2016). each paired with its own set of learning rates, as follows:
19

---

Published as a conference paper at ICLR 2025
Table Constraint violation rate (CVR) for each unconstrained DGM model and each dataset.
Constraint Type
Model/Dataset
URL
CCS
LCLD
Heloc
House
WGAN
17.9
15.0=5.6
28.3+11.3
69.1
100.0:0.00
TableGAN
5,4114
14.5=3.6
19.143.7
45.6:116.3
100.011.00
CTGAN
3.8+1.3
56.1=7.5
1.91111
55.8=10.2
100.0=0.0
Linear
TVAE
3.0
8.6=1.9
3.9
1..
100.0=0.0
GOGGLE
47.3+6.9
42.5=3.9
16.5+13.2
67.3+6.9
99 99.9+0.1
All models DRL
0.0:0.00
0.0=0.0
0.0+0.00
0.0+0.00
0.0 #0.0
WGAN
8.6+1.7
38.3=10.0
26.8+5.2
47.3+15.5
74.2++144
TableGAN
3.9
56.1
16.013.5
33.8114.7
73.7114.8
CTGAN
6.3
58.8
5.340.8
1.741.4
42.8:123.1
Disjunctive
TVAE GOGGLE
7.610.7
11.8=0.7
6.641..5
0.0+0.0
52.7+24.7
2.0+2.88
37.1=15.8
65.3+11.8
35.2+4..2
20.4::20.5
All models DRL
0.0
0.0=0.0
0.0.10.0
0.010.00
0.0 1:0.0
WGAN: Adam: {1 10 10-4. 2x10 1x110 R.SProp:: f5x10 SGD: {1 x 10 x 10 10-3)
1111100,11x10---11111111 1x10 3),
1111100**
TableGAN; Adam: {5 x 10 1 x 10 10-4,11111 10-$,1 x 10-3), RMSProp:
x1x 10-4 2 10 10-1,1 ")-"), x1 10 x 10
CTGAN: Adam: {5 x 100
10-4,144-00 11-4,,10011111111011111111,,11,1 10 10-3),101000001
RMSProp: {1 10-4,
4,
10 10-4, x 10 10-3) 3]. SGD: {1 x 10 10x0,,1100
Additionally, we explored various batch sizes for each model:
WGAN: {64, 128, 256, 510}
TableGAN: 2128,256, 22,,25560 510}
CTGAN and TVAE: {70, 280, 500}
GOGGLE: (32, 64, 128)
We did not separately tune our DGM+DRL models, nor the DGM+LL models. However, for each of the DGM+DRL and DGM+LL models, we ran three versions corresponding to three different ways of ordering the variables that decided the order which the layers (LL and DRL) change values of the features. More precisely, we tried: (i) random (Rnd) ordering, (ii) the correlation (Corr)-based ordering. (iii) the Kernel Density Estimation (KDE)-based ordering. The last two orderings proposed by /Stoian al. (2024), and are defined in the Appendix of their paper. For completeness, we also define them here. The Corr-based ordering is computed as follows: for each feature, the absolute difference is taken between the pairwise feature correlations (with respect to all other features) of samples generated by the unconstrained DGM and the real data. The features then ranked in ascending order based on these scores. This ensures that features with the most similar correlations between the generated and real data are prioritised by DRL (and LL) first. The KDE-based ordering is computed by first fitting Kernel Density Estimator (KDE) on the real data and estimating the log-likelihood for each real and synthetic sample. In a discrete setting, regardless of the variable domains, two marginal probability mass functions are approximated for each variable using the real and synthetic data, respectively. The variables are then ranked by computing the Bellback-Leibler divergence between these two and sorting the results ascending order. We then selected the best ordering separately for each DGM+DRL and DGM+LL model. In Table 7 we report the optimal hyperparameter configurations, which we use in all experiments presented in our paper that involve the DGM, DGM+LL and DGM+DRL models.
K SYNTHETIC DATA QUALITY.
Background knowledge alignment. In addition the CVR metric reported in the main paper, we also compute the samplewise constraints violation coverage (sCVC) and the constraint violation coverage (CVC), where sCVC indicates the average percentage of constraints violated per sample,
20

---

Published as a conference paper at ICLR 2025
Table 9: Samplewise constraints violation coverage (sCVC) for each unconstrained DGM model and each dataset.
Constraint Type
Model/Dataset
URL
CCS
LCLD
Heloc
House
WGAN
2.2:
7.922999
15.1
14.3+:2.5
50.0=0.0
TableGAN
0.7:u..2
7.5
9.81199
9.043.3
50.0
CTGAN
0.5
32.1
1.010.5
9,9422.8
50,0
Linear
TVAE
0,4+0.1
4.4
2.0+0.3
7.4
50.0=0.0
GOGGLE
17.2+4.9
22.0+2.3
8.647.1
17.2+4.9
50.0=0.0
All models + DRL
0.0+0.0
0.0+0.0
0.0+0.0
0.0
0.0 =0.0
WGAN
0.9+0.2
5.3+1.7
2.2+0.5
11,5444
7,0=0.1
TableGAN
0,4+0.1
6.8+119
1.5+0.3
7.943.7
6,7+1.3
CTGAN
0.640.1
8.0.1.2
0.410.1
0.340.3
3.942.1
Disjunctive
TVAE GOGGLE
0.840.1
1.340.1
0.540.0
0.040.0
4.8
0.2+0.3
5.042.1
10.043.4
7.140.9
1.9
All models + DRL
0.0
0.0
0.0+0.0
0.0+0.0
0.0 =0.0
Table 10: Constraints violation coverage (CVC) for each unconstrained DGM model and each
dataset.
Constraint Type
Model/Datasct
URL
CCS
LCLD
Heloc
House
WGAN
45.0=5.0
100.0=0.0
100.0=0.0
100.0*0.0
100.0=0.00
TableGAN
34.0+-1.2
100.0+0.0
100.0+0.0
100.0+0.0
100.0=0.0
CTGAN
17.5+41.3
100.0+0.0
100.0::000
100.0+0.0
100.0+0.00
Linear
TVAE
12.5+0.0
100.010.0
100.0.0.00
9..411.3
100.040.0
GOGGLE
100.040.0
100.040.0
100.0.0.00
100.01:.00
100.040.0
All models DRI.
0.0+0.00
0.0+0.00
0.0
0.0
#0.0
WGAN
50.0+0.0
40.0+0.0
28.0+11.3
80.0+0.0
34.942.4
TableGAN
53.6+3.8
40.0+0.0
51.343..
80.040.0
36.4+0.0
CTGAN
56.4115.0
40.010.0
22.31:222
55.216.6
34.913.3
Disjunctive
TVAE
55.214.6
40.010.0
20.313.4
24.8111.11
36.0.1.8
GOGGLE
28.41:21.3
40.010.0
49.118.1
48.8111.6
33.3
All models DRI.
0.0.10.0
0.010.0
0.0.1..0
0.0.10.0
0.0 .10.0
across all samples, and CVC represents the percentage constraints violated at least once by the samples. In Tables 8 9 and 10 we provide the CVR, sCVC, and CVC, respectively, for the unconstrained DGM models, according to the two possible types: linear and disjunctive constraints. Indeed, constraints expressed linear inequalities
special case of the QFLRA constraints, we have also partitioned the constraints between those that can be captured as linear inequalities and those that cannot. Then, checked how much each of the partitions contributes the high CVR results, and found that in 13 (resp. 6) out of 25 cases, the CVR was greater than 25% (resp. 50%) on the constraints presenting disjunctions alone.
Efficacy. Tables II 12 show the efficacy, along with the standard deviations from the mean, for each unconstrained DGM model and the corresponding DGM+DRL. models on every classification dataset, using the F1, wF1, and AUC metrics, respectively. Similarly, in Table [14 we report MAE and RMSE for each unconstrained DGM model and the corresponding DGM+DRL. models on the regression dataset (i.e., House).
L LINEAR VS. QFLRA CONSTRAINTS
Background knowledge alignment. In Tables 15 16 and provide the CVR, sCVC, and CVC, respectively, for the DGM+LL models, according to the two possible types: linear and disjunctive constraints.
21

---

Published as a conference paper at ICLR 2025
Table 11: Efficacy comparison between the unconstrained DGM models and their DRL counterparts in terms of FI. The performance, along with the standard deviation, is reported for each classification dataset.
URL
CC
LCLD
HELOC
WGAN
0.794
0.303 0660
0.139=00.55
0.665=0.000
WGAN+RS
0.792++0.031
0.051+0.037
0.156=00007
0.628=0.013
WGAN+DRI.
0.800
0.313
0.19710.060
0.721 #0.027
TableGAN
0.562=00.051
0.196:10.08
0.259
0.593
TableGAN+RS
0.544
0.138:+0.02 025
0.251
0.568 00707
TableGAN+DRL
0.619
0.163++0.07
0.269
0.628
CTGAN
0.822#0.001
0.145 040
0.247
0.736=0.035
CTGAN+RS
0.817
0.086
0.201 10.066
0.706
CTGAN+DRL.
0.80610..00
0.288 1...1.1.11. 116
0.288 10.013
0.744:0.000
TVAE
0.81010.008
0.325
0.185 0.185
0.717 0.0013
TVAE+RS
0.78810.023
0.024
0.237 0.237 10.018
0.420:10.007
TVAE+DRL
0.835
0.467 100
0.189 +0.022
0.731 00.009
GOGGLE
0.622:10.04
0.0391100016
0. 156
0.596
GOGGLE+RS
0.608+10.098
0.047:+0.02
0.235 1.1149
0.577 09.0
GOGGLE+DRL
0.720 00.066
0.253
0.298
0.698 00003
Table 12: Efficacy comparison between the unconstrained DGM models and their DRL counter- parts in terms of wF1. The performance, along with the standard deviation, reported for each classification dataset.
URL
CC
LCLD
HELOC
WGAN
0.796##0002
0.330 057
0.296=000037
0.648
WGAN+RS
0.794+0.002
0.088+0.00 035
0.312+0.050
0.617=0.018
WGAN+DRL
0.801#0.011
0.340 122
0.33910.019
0.652
TableGAN
0.659+0.033
0.228 035
0.393+0.010
0.615=0.030
TableGAN+RS
0.64810.006
0.172.10.024
0.389
0.599 0.006
TahleGAN+DRI
0.69310.028
0.196:10.076
0.401 10.018
20.88
CTGAN
0.79940.033
0.159+0.004 042
0.379
0.675
CTGAN+RS
0.795+10001
0.095
0.342+0.054
0.650=0.009
CTGAN+DRL
0.81510.011
0.308 1..188
0.409
0.680
TVAE
0,8021100022
0.351 1881
10.001
0.686110.004
TVAE+RS
0.77810.026
0.061 100010
0,28310.000
10.007
0.465 10.001
TVAE+DRL
0.83210.014
0.487-10.099
0.330 10.014
0.69410.006
GOGGLE
0.64810.074
0.076.10.015
0.296 10.066
0.566110.00
GOGGLE+RS
0.639+0.066.
0,084+0.022
0.322
0.549
GOGGLE+DRL
0.673+0.039
0.281+0.00 139
0.310+0.057
0.636=0.022
Efficacy. Tables 18 for each DGM+LL model and the corresponding DGM+DRL. models on every classification dataset, models
20 show the efficacy, along with the standard deviations from the mean,
using the F1, wF1, and AUC metrics, respectively. Similarly, in Table 21 we report MAE and RMSE for each unconstrained DGM+LL model and the corresponding DGM+DRL models on the regression dataset (i.e., House).
M BACKGROUND KNOWLEDGE ALIGNMENT: A QUALITATIVE ANALYSIS
Figure 6 accompanies Figure 4 from the main body of our paper and shows the relevant sample space for the same two constraints from the House dataset: if the Zipcode is 98004 or 98005 then the Price is greater than 400K USD and the Zipcode is between 98006 and 98008 then the Price exceeds 225K USD. As comply with the constraints. Unlike the synthetic data from the unconstrained DGMs, the samples generated using DRL layer never cross into the areas that mark regions where datapoints violate
see, in all cases, the unconstrained DGMs and the DGMs+LL fail to
22

---

Published as a conference paper at ICLR 2025
Table 13: Efficacy comparison between the unconstrained DGM models and their DRL counter- parts in terms of AUC. The performance, along with the standard deviation, is reported for each classification dataset.
URL
CC
LCLD
HELOC
WGAN
0.870=#.0012
0.814=00007
0.605=0.010
0.717=0.02 00201
WGAN+RS
0.862#0.009
0.570
0.611=0.022
0.685#0.02
WGAN+DRI.
0.875:10.007
0.88510.050
0.62310.023
0.717=0.029
TableGAN
0.843+20000
0.802
0.655
0.707=0.00 00000
TableGAN+RS
0.854
0.682:+0.08
0.653+0.010
0..00
TableGAN+DRL
0.865+0.022
0.742+0.008
0.657=00007
0.709
CTGAN
0.859#0.040
0.914
0.651 =0.00
0.744=0.009
CTGAN+RS
0.85610..010
0.515
0.61510.031
0.706.1.014
CTGAN+DRL.
1..03
0.955 10.022
0.643 0.0019
0.745 0.0008
TVAE
0.86310.011
0.858
0.631 10.004
0.750.10.004
TVAE+RS
0.84610.024
0.522.10.04
1..008
0.497 100006
TVAE+DRL
0.09010.010
0.92611.009
0.635 00.002
0..52
GOGGLE
0.742
0.549
0.551 20.034
0.600
GOGGLE+RS
0.727:100060
0.57 1.571+00077
0.532:20.049
0.592=1.0053
GOGGLE+DRL
0.747 0..099
0..09:
0.563=20.027
0.691 03.0
Table 14: Efficacy performance comparison between DGM and DGM+DRL. models trained on House, using MAE and RMSE.
MAE
RMSE
WGAN
547652.6:1..1
688130. 1:4.8
WGAN+DRL
547637.5=17.4
688118.0=14.9
TableGAN
547655.3
688132.7=4499
TableGAN+DRL
547653.6#22.88
688131.4#18.7
CTGAN
547652.9
088130.4=2.0
CTGAN+DRL.
547642.9.1188
688122.1 1:1.00
TVAE
547650.0
688128.5:14.2
TVAE+DRI
547645.41311
688124.6 222.8
GOGGLE
547639 1166395511133 13.8
688119.6 11.5
GOGGLE+DRI.
547633.9 16.00
688115.7
the constraints and, in addition, their distribution resembles closely the real data in all five cases.
In addition, we show a similar comparison but for a different dataset and different constraints. Specifically, we consider the following two constraints from the URL dataset: if the Number of Subdomains less than 2 then the Hostname length is less than 30 and if the Number of Subdo- mains is less than 3 then the Hostname length is less than 55. The two constraints capture the relation between the two features (i.e., Number of Subdomains and Hostname length) and, differently from the two constraints from the House dataset mentioned above, their respective violation space inter- sects, as shown in red Figure Nevertheless, our constrained models never violate any of the constraints, unlike the unconstrained and DGM+LL models.
Finally, in order to show the unintended consequences using rejection sampling, we visualise us- ing t-SNE (van der Maaten & Hinton 2008) the differences between the distributions of the samples generated by the standard DGMs and by DGMs+RS (i.e, with rejection sampling). These visuali- sations be found in Figure 8 As
in the Figure, there can be cases where rejection sampling actually creates some changes in the distribution, which can then affect the machine learn ing efficacy.
23

---

Published as a conference paper at ICLR 2025
7-
Rea
oWGAN
о ++LL
+DLRL
Price (1e5)
N w R.
98004
98005
98006
98007
98008
98004
98005
98006
98007
98008
98004
98005
98006
98007
98008
9800d
98005
98006
98007
98008
Zipcode
Zipcode
Zipcode
Zipcode
Rea
TableGAN
++LL
+DRR
Price (1e5)
10
H
98004
98005
98006
98007
98008
98004
98005
98006
98007
98008
98004
98005
98006
98007
98008
98004
98005
96006
98007
98008
Zipcode
Zipcode
Zipcode
Zipcode
Reaa
oCTGAN
O+LL
+DRR
Price (1e5)
98004
98005
98006
98007
98008
98004
98005
98006
98007
98008
98004
98005
98006
98007
98008
98004
98005
98006
98007
98008
Zipcode
Zipcode
Zipcode
Zipcode
oReal
TVVE
O+LL
+DRR
Price (1e5)
D 0S0
98004
98005
98006
98007
98008
98004
98005
98006
98007
98008
98004
98005
98006
98007
98008
98004
98005
98006
98007
98008
Zipcode
Zipcode
Zipcode
Zipcode
Price (1e5)
5n 6
oRea
GOGGLE
O+LL
+DRL
2 9.
98004
98005
98006
98007
98008
98004
98005
98006
98007
98008
98004
98005
98006
98007
98008
98004
98005
98006
98007
88008
Zipcode
Zipcode
Zipcode
Zipcode
Figure 6: Comparison of sample distributions between real data and synthetic data generated by the unconstrained DGM models and their corresponding DGM+LL and DGM+DRL models, using the Zipcode and Price features from the House dataset. In order (from the top to the bottom row), the DGM models used in the plots are: WGAN, TableGAN, CTGAN, TVAE, and GOGGLE. The areas red indicate regions where samples violate the constraints on the given features.
24

---

Published as a conference paper at ICLR 2025
Hostname length
70
60
50
40
Real
WGAN
+LL
DRL
1
subdomains
No. subdomains
No. subdomains
No. subdomains
Hostname length
70
Real
TableGAN
o +
+ DRL
No. subdomains
No. subdomains
No. subdomains
No. subdomains
mlsstname length
30
20
10
Real
cCTAN
+LL
+DRL
0
No. subdomains
No. subdomains
No. subdomains
No. subdomains
Hostname length
70
60
50
40
30
20
Real
TVVA
+LL
DDL
No. subdomains
No. subdomains
No. subdomains
No. subdomains
Hostname length
70
60
50
40
Real
GGGGGE
o ++LL
DRL
1
3
1
3
No. subdomains
No. subdomains
No. subdomains
No. subdomains
Figure 7: Comparison of sample distributions between real data and synthetic data generated by the unconstrained DGM models and their corresponding DGM+LL and DGM+DRL models, using No. subdomains and Hostname length features from the URL dataset. In order (from the top to the bottom row), the DGM models used in the plots are: WGAN, TableGAN, CTGAN, TVAE, and GOGGLE. The areas in red indicate regions where samples violate the constraints on the given features.
25

---

Published as a conference paper at ICLR 2025
Table 15: Constraint violation rate (CVR) for each DGM+LL model and dataset.
Constraint Type
Model/Dataset
URL
CCS
LCLD
Heloc
House
All models LL
0.0
0.0
0.0:
0.0
0.0 1:000
Linear
All models DRL
0.0
0.0.10.0
0.0.10.0
0.0
0.0 1:0.0
WGAN+LL
8.913.2
51.5
27.0113.6
20.616..3
100.011..0
TableGAN+LL
3.620.8
54.0.11.88
11.3.10.9
26.611.77
23.9.2.7
CTGAN+LL
7.0=2.6
55.7=16.3
2.6+1.11
2,6+2.4
10.8
Disjunctive
TVAE+LL
6.8=0.6
8.4+2.0
5,8+0.88
0.0+0.0
13.0#12.6
GOGGLE+LL
6.5=7.0
23.0=10.7
81.946.5
11.5+7.1
2.6+2.6
All models DRL
0.0=0.0
0.0:2.00
0.0:t.00
0.0:0.0
0.0 #0.0
Table 16: Samplewise constraints violation coverage (sCVC) for each DGM+LL model and each
dataset.
Constraint Type Model/Dataset
URL
CCS
LCLD
Heloc
House
All models LLL
0.0
0.0
0.0.10.0
0.0.10.0
10000
Linear
All models DRL.
0.0
0.0
0.0
0.0
0.0 +0.0
WGAN+LL
0.940.4
6.941.1
2.2+10.4
4.2+:..4
9.1
TableGAN+LL
0.4+0.1
6.542.3
1.040.1
5.9
2.2+0.3
CTGAN+LL
0.740.33
7.0
0.2:10.1
0.5
1,0+0.7
Disjunctive
TVAE+LL
0.7::0.1
0,910.2
0.4::0.1
0.010.00
1.211.2
GOGGLE+LL
0.710.7
2.611.4
11.2:1211
2.34114
0,210.2
All models DRL
0.0.0.0
0.0.0.0
0.0.10.0
0.0.10.0
0.0 10.0
N SAMPLE GENERATION TIME
As we can see from Table 22 the DGM+DRL models bring additional time to the sample gener- ation runtimes. However, this is often much smaller than the additional time required to use unconstrained model and then doing rejection sampling. Indeed, the largest runtime difference reg- istered between unconstrained DGM and its constrained counterpart is of only 0.12 seconds (i.e., for CTGAN on the URL and LCLD datasets). Notably, in 20 out of 25 cases, the overhead for the DGM+DRL models is less than 0.1 seconds. On the other hand, for the DGM+RS models, the sam- ple generation procedure timed out after 24h for all the models tested on the House dataset (where we had 100% CVR). The registered times are also not very promising for any of the other datasets when using DGM+RS, where the minimum absolute difference registered equals 0.07 seconds and the maximum equals 5.55 seconds (notice that no DGM nor DGM+DRL model has sampling time above 0.29 seconds).
O REAL DATA PERFORMANCE
In Table 25 we report the average Fl-score, weighted F1-score and Area Under the ROC Curve obtained by training the same six classifiers (resp. four regressors) on the four real classification datasets (resp. real regression dataset). This allows us to compare the machine learning efficacy of the synthetic data with one of the real data. As we can see from the results, the synthetic data generated with all the DGMs (unconstrained, +LL. and +DRL) manage to obtain very good results. In multiple occasions, the models trained on the synthetic data not only get results comparable with the ones obtained with the real data, but actually perform better than them. In spite of this, the classification models trained on synthetic data never manage to get better performance than those trained on the real data for all metrics. On the other hand, the regressors trained on the synthetic version of the House dataset always manage to get lower MAE and RMSE, matter the DGM used to generate the synthetic data (with the exception TVAE+LL which got slightly higher MAE than the obtained with the real data).
26

---

Published as a conference paper at ICLR 2025
Table 17: Constraints violation coverage (CVC) for each DGM+LL model and each dataset.
Constraint Type
Model/Dataset
URL
CCS
LCLD
Heloc
House
All models LL
0.0
0.0
0.010.0
0,0
0.0 00.0
Linear
All models + DRL
0.0
0.0
0.0.:0.0
0.0
0.0 0.00
WGAN+LL TableGAN+LL
50.010.0
40.0.10.0
28.011..
1..2
13.1 1411
54.0114.7
30.01:0.0
27.310000
84.81111.1
36.7 0.8
CTGAN+LL
52.8+2.3
30.0=0.0
25,4+2.7
41.6++2..8
32.7=5.0
Disjunctive
TVAE+LL GOGGLE+LL
51.2+11
29.2+11.
22.6+0.6
20.8+11.3
32.0=1.6
33.6412.4
27.5+5.0
46,2+0.0
28.0#16.7
17.6=1.0
All models DRL
0.0
0.0::000
0.0+0.0
0.0
0.0
Table 18: Efficacy comparison between the DGM+LL models and their DRL counterparts in terms
F1. The performance, along with the standard deviation, is reported for each classification dataset.
URL
CC
LCLD
HELOC
WGAN+LL
0.803+0.035
0.359 0906
0.183+0.000
0.694
WGAN+DRL
0.800+0.011
0.313+0.11 127
0.197=0.060
0.721
TableGAN+LL
0.612+t0.11
0.169
0.232++00226
0.638=0.061
TableGAN+DRL
0.619#0.016
0.163:+0.079
0.269=00025
0.628=00.08%
CTGAN+LL
0.836+0.002
0.250 0801
0.265
0.729=0.027
CTGAN+DRI.
0.83610.000
0.288 116
0..033
0..22
TVAE+LL
0.82440.004
0.413+
0.158
0.730
TVAE+DRL
0.835+0.009
0.467 100
0.1092
0.731 00.009
GOGGLE+LL 0.787+0.014
0.233++.118
0.284
0.723=#0018
GOGGLE+DRL
0.720+0.086
0.253 144
0.298=+0.168
0.698=0.022 023
P COMPARISON BETWEEN DGMs+DRL AND LLM-BASED TABULAR DATA GENERATION
recent trend the tabular data generation field has been to use LLMs to perform the task. While these models are very promising, they are also not exempt from the problems that affect the other DGMs. In this section we thus compare the performance of the standard DGMs equipped with our DRL and GreAT (Borisov et al. 2023 a state-of-the thrt---rttttt LLM- based tabular data generator. For all datasets considered, GreAT was trained and run on an A100 GPU with 40GB of RAM, using the pre-defined hyperparameters.
Table 23: CVR for each model and dataset. Cases with CVR250% underlined. Best results are in bold.
URL
CCS
LCLD
Heloc
House
GreAT
0.7
98.010.3 1.1
9.60.1.5
15.7#1.4
All + DRL 0.0 0.040.0
0.0=0.0
0.0+0.0
0.0+0.0
Firstly, in Table 2 can see from the Table, even though in two out of five cases GreAT manages to get very low CVR, for CCS its CVR shoots up to 98.0%. As CCS is by far our smallest dataset (with only 1K datapoints in the training set), this hints to the fact that M-b--d models struggle to learn the distribution from datasets with few datapoints.
report the CVR obtained with GreAT and with all the models+DRL. As we
Secondly, we generate 1,000 samples with GreAT and we report the average runtime Table 24 together with the average sampling time obtained with the DGMs+DRL. As we from the Table, GreAT takes two ders of magnitude longer to perform the sam- pling than the average DGM+DRL. model.
Table 24: Sample generation time in seconds.
URL.
CCS
LCLD
Heloc
House
GreAT
51.8
28.4
22.3
26.3
14.4
DGM+DRL
0.22 0.13
0.14
0.10
0.13
27

---

Published as a conference paper at ICLR 2025
Table 19: Efficacy comparison between the DGM+LL models and their DRL counterparts in terms
of wF1. The performance, along with the standard deviation, is reported for each classification dataset.
URL
CC
LCLD
HELOC
WGAN+LL
0.799=t.0022
0.383 0.09
0.330=0.006
0.662
WGAN+DRL
0.801#0.011
0.340#0.12
0.309
0.652#0.036
TableGAN+LL
0.695:0.077
0.203
0.373=
0.633 03666
TablcGAN+DRL
0.693#0.028
0.196::0.076
0.401 =0.018
0.628 00008
CTGAN+LL
0.820=10.008
0.271 083
0.392=00.030
0.688=0.010
CTGAN+DRL
0.815+0.011
0.308 118
0.409=0.007
0.680=0.011
TVAE+LL
0.816#0.000
0.436:+0.00 055
0.310=0.011
0.691=0.007
TVAE+DRL
0.832#0.014
0.487=0.006
0.330=0.011
0.694=0.006
GOGGLE+LL
0.749
0.262 173
0.310==0.030
0.663=0.012
GOGGLE+DRL 0.673#0.009
0.281 0.119
0.310=0.057
0.636=0.000
Table 20: Efficacy comparison between the DGM+LL models and their DRL counterparts in terms
of AUC. The performance, along with the standard deviation, reported for each classification dataset.
URL
CC
LCLD
HELOC
WGAN+LL
0.869:t0.014
0.857:10005
0.608
0.732#0.013
WGAN+DRL
0.875
0.885
0.623+t0.023
0.717
TableGAN+LL
0.868
0.794:10.01
0.640:10.05
0.704:0.030
TableGAN+DRL
0.865:t0.022
0.742:10.089
10.007
0.709:0.011
CTGAN+LL
0.800
0.959 10.027
0.641 10.015
0.755 0.0007
CTGAN+DRL
0.883+0.009
0.955+0.00 0.2
0.643=0.019
0.745
TVAE+LL
0.878+0.007
0.933+0.008
0.633
0.747 00000
TVAE+DRL
0.893+0.010
0.926#0.008 0.99
0.635=0.002
0.752=#0.00
GOGGLE+LL
0.802
0.765
0.554
0.719=00.00
GOGGLE+DRL
0.747:0.009
0.758+00099
0.563+0.027
0.691 +0.039
Table 21: Efficacy performance comparison between DGM+LL and DGM+DRL models trained on House, using MAE and RMSE.
MAE
RMSE
WGAN+LL
547638.5#11.4 688118.2+11.0
WGAN+DRL
547637.5+17.4
688118.0=11.9
TableGAN+LL
547638.0=18.9
688118.3=17.9
TableGAN+DRL
547653.6+22.
688131.4=18.7
CIGAN+LL
547642.3:11.6
688121.4:14.6
CTGAN+DRI.
547642.9418.1
688122.1
TVAE+LL
547658. 1.9.8
688137.429.4
TVAE+DRL
547645.4431.8
688124.6=22.88
GOGGLE+LL
547651.9
688129.6
GOGGLE+DRL
547633.9=16.0
688115.7=11.3
This difference is particularly striking given that GreAT was the only model requiring an A100 to run.
This analysis shows that not only LLM-based are still prone the violation of the constraints, but also that they require more powerful hardware and much longer time to sample.
28

---

Published as a conference paper at ICLR 2025
WICAN WIGNHRRS
TacleGAN
(b)
CTGAN CIGANARS
150
TiKE TUAE-R5
(c)
GOGGLE
Figure 8: t-SNE visualisations of the distribution of the samples generated for the CCS dataset by (i) WGAN and WGAN+RS in Figure Sal (ii) TableGAN and TableGAN+RS in Figure 8b (iii) CTGAN and CTGAN+RS in Figure Sc
(iv) TVAE and TVAE+RS in Figure Sd and (v) GOGGLE and GOGGLE+RS in Figure Se Note that changes in distribution are marked with red contours.
29

---

Published as a conference paper at ICLR 2025
Table 22: Sample generation time (in seconds) for all DGMs and their respective DGM+DRL mod- els and DGM+RS models (i.e, DGMs with rejection sampling), for all datasets. The hyphen indi- cates timeout after 24h.
URL
CCS
LCLD
Heloc
House
WGAN
0.01 0.01
0.01
0.01
0.00
WGAN+RS
0,08
0.11
0.12
0.25
WGAN+DRI.
0.08
0.07
0.08
0.04
0.09
TableGAN
0.21
0.10
0.09
0.10
0.07
TableGAN+RS
0,43
1.57
0.38
0.78
TableGAN+DRL
0.28
0.14
0.19
0.15
0.18
CTGAN
0.16 0.11
0.10
0.09
0.07
CTGAN+RS
0.45 1.71
0.37
1.01
CTGAN+DRL
0.28 0.19
0.22
0.14
0.16
TVAE
0.14 0.08
0.07
0.06
0.05
TVAE+RS
0.37 0.31
1.08
0.96
TVAE+DRL
0.25 0.16
0.20
0.11
0.14
GOGGLE
0.22 0.08
0.08
0.06
0.06
GOGGLE+RS
0,48
0.35
5.63
0,25
GOGGLE+DRL
0.29
0.14
0.11
0.09
0.16
Table 25: Efficacy scores calculated on real data. For classification datasets URL, CCS and LCLD, we used F1 Fl-score, weighted Fl-score and Area Under the Curve to measure the performance, while for the regression dataset House, we used the Mean Absolute Error metrics and the Root Mean Square Error.
F1
wF1
AUC
MAE
RMSE
URL
0.88410.007
0.875+0.004
0.903
CCS
0..29
0.547:0.011
0.948=0.010
LCLD
00.00
0.31620.013
0.645
Heloc
0.702
0.66210.011
0.707
House
547655.7+38.2
688133, 1
30


Reviews:
[{"review_id": "i29DTzecCM", "reviewer": "Reviewer_oZm4", "summary": "The authors of this paper study the problem of generating tabular data where complex constraints expressed by QFLRA should be satisfied. The authors propose a neuro-symbolic AI layer to tackle this problem and prove its effectiveness in respecting the constraints. Experimental results on several datasets clearly demonstrate the advantages of the proposed model over the baselines.", "questions": "Please refer to W1-W3.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1. The problem of satisfying QFLRA in building tabular data generative models is new.\nS2. The proposed technique can be integrated into any deep model.\nS3. The proposed method is sound as the authors prove its correctness.", "weaknesses": "W1. The authors mainly focus on proving that their proposed Neuro-symbolic AI layer can respect the complex constraints expressed by QFLRA. What about the impact of this new layer on the learned distribution? Is it possible that such a new layer may harm the learned distribution for satisfying the constraints? \n\nW2. It seems to me that the constraints of QFLRA may be very complex and even deciding if a QFLRA is satisfiable is a difficult problem. What would happen to the proposed model if the input QFLRA is not satisfiable? \n\nW3. In the experiments, the authors adopt some simple QFLRA constraints, where the feasible region has a couple of disconnected parts. For these relatively simple QFLRA, can we first identify each disconnected subregion, then use traditional models to learn the distribution over each subregion and exploit a mixture model to combine distributions over all subregions to have an overall generative model?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "m2MjYilP8N", "reviewer": "Reviewer_TebA", "summary": "This paper introduces a novel Disjunctive Refinement Layer (DRL) for Deep Generative Models (DGMs), aiming to generate synthetic tabular data that 1) the sample space of the model distribution is aligned with what is stated in the background knowledge. 2) The model distribution approximates the ground truth distribution. This paper extends constraint enforcement in data generation by leveraging Quantifier-Free Linear Real Arithmetic (QFLRA) formulas, enabling more complicated (e.g. non-convex constraints). The theorem proves the proposed DRL fully satisfies the constraints and is optimal in Euclidean distance.", "questions": "1. What is the computation complexity of DRL?\n\n2. What are the constraints for different datasets? Are they manually chosen by authors or are included in the original datasets?\n\n2. Why does satisfying the constraints help improve machine learning efficiency? Is that because we also transform the test dataset so that they satisfy the constraints?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The use of QFLRA constraints in DRL is a substantial advancement, enabling the generation of data that complies with complex, realistic conditions. This advancement is especially relevant for applications requiring high data fidelity, such as healthcare or finance.\n\n2. Theory analysis proves the DRL will transform the generated data to fully satisfy the constraints while attaining optimal Euclidean distance to the original sample.\n\n3. The paper includes extensive testing across five generative models and datasets, showing consistent improvements in constraint satisfaction and downstream model performance.", "weaknesses": "The main method section (Sec 3) is a little bit dense and hard to read. I think providing a pseudo code is helpful here.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "22uLPVfTUX", "reviewer": "Reviewer_vt7B", "summary": "This paper proposes a post-processing method to ensure that already generated tabular data satisfies conditions defined as a union of feasible sets specified by linear inequalities. The constraints include complex forms, such as disconnected and non-convex sets, which are rarely addressed in traditional constraint problems. The proposed method, a variation of Fourier-Motzkin, progressively reduces constraints to derive a broader feasible region. If the generated data fails to meet predefined constraints, the original data point is adjusted to the one with the minimum Euclidean distance within the relaxed constraint region.", "questions": "1. In Lemma 3.1, what does \" optimal \" mean?\n\n2. What is the meaning of \"The CP resolution rule is sound\"?\n\n3. In Example 4, does the result depend on the ordering of variables?\n\n4. Does the proposed method adjust the generated data to satisfy the given constraints in the problem, or does it use more relaxed constraints?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "When the feasible region is a disconnected, non-convex set, it is challenging to directly apply projection methods for post-adjusting generated data. This paper is significant because it demonstrates the existence of complex constraints in tabular data generation and proposes a solution to address this issue.", "weaknesses": "The method proposed in this paper for post-processing already generated data may require further theoretical support. It isn't easy to assert that finding the data point with the shortest Euclidean distance within the constraint-satisfying set is the most reasonable approach. For example, the Mahalanobis distance might be a more effective metric. \n\nFurthermore, the paper does not compare information leakage and reproduction accuracy, essential performance metrics for generated data. It would be beneficial to demonstrate the trade-off between these two critical metrics in generative models. For example, it is helpful to report the Membership Inference Accuracy and Attribute Disclosure Risk of the considered models.\n\nAnother limitation of this model is that it requires ranking the variables included in the constraints to be satisfied for high-dimensional data. The proposed method seeks solutions within a feasible region larger than the region required to satisfy the constraints. Additionally, the constraint composed of the union of linear constraints covered in this paper could potentially be avoided during optimization using barrier methods. Although this approach differs from the method proposed in the paper, one could consider normalizing the model by using a penalty function to ensure the generated table model remains within the predefined feasible region. For instance, adding a log barrier function to the risk function can enforce the necessary constraints on the output of the VAE decoder. Alternatively, for computational stability, ReLU could be used instead of the log barrier function. This approach may serve as an alternative that fulfills both the objective function for model training and the given constraints, potentially bypassing the issue of defining metrics for post-processing generated data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Q8Fc2hewFe", "reviewer": "Reviewer_KTDt", "summary": "This paper proposes a novel Disjunctive Refinement Layer added to neural network synthetic data generators (to replace linear constraint layer) that naturally enforces complex (including non-convex and disconnected) QFLRA constraints on tabular data without rejection sampling, and also improves the synthetic data's utility.", "questions": "As mentioned in Weaknesses.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "+ It uses a disjunction of linear inequalities to decompose complex non-convex and disconnected constraints into a combination of simple constraints, whose satisfiability implies the entire inequality system's satisfiability.\n+ It derives and resolves implicit constraints among previous variables that ensure the satisfiability of constraints on the latter variable, and hence allows satisfiability in the entire system.", "weaknesses": "- Mathematical statements need clarification:\n\n    1. Line 127: If $w_k=0,\\forall k$ and $b < 0$, which satisfies your constraint that $w_k,b\\in\\mathbb{R}$, would the claim that $\\Omega(\\Phi)$ is non-empty still be true? (Although similar case is addressed in Line 275-277, it does not validate Line 127.)\n    2. Line 170-172: What if $\\Psi$s in $\\Pi$ are not simplified and somewhat standardized? \n            1) If for some $\\Psi$, $l^\\Psi_i >= r^\\Psi_i$ (violating the constraint \"whenever ...\" in Line 159, if it could be violated), would the definitions of $l^\\Pi_i$ and $r^\\Pi_i$ still be valid?\n            2) If for some $\\Psi,\\Psi'\\in\\Pi$, $[l^\\Psi_i, r^\\Psi_i]\\cap$[l^{\\Psi'}_i,r^{\\Psi'}_i]\\ne\\emptyset$ (so that less than $|\\Pi|+1$ disjoint intervals are created), would the definitions of $l^\\Pi_i$ and $r^\\Pi_i$ still be valid?\n    3. Line 259-260: Ambiguous (although can be guessed from textual explanation), it's $(A\\setminus B)\\cup C$ or $A\\setminus(B\\cup C)$?\n    4. Line 285-287: Similar problem as in Line 170-172\n\n- Some implementation details not mentioned in paper and appendix:\n\n    1. How the DRL is applied in all the experimented models is not explicitly stated, including Appendix.\n    2. How constraints are determined (or if manually determined, what are they) is not mentioned in the main content nor appendix.\n\n- The paper claims the additional sampling time is negligible, but I would like to doubt this on the following basis:\n\n    1. 1000 samples could be a bit too small, taking the shared overhead of initializing the task and data transformation into account. \n    2. The sampling time is expected to be approximately in linear correlation to the number of samples, so that ~0.1s for 1000 samples mean ~1s for 10000 samples, ~10s for 100000 samples, etc. What matters more is not the absolute additional time per 1000 samples, but the relative additional time (e.g. how many times more needed). According to the reported generation time in Table 5, in 3/5 cases DRL needs more than twice the time than non-DRL. This might not be negligible.\n    3. DRL is also added during training, given that it could need ~2x times the time to generate samples, it naturally leads to the question that does it need also ~2x times to train.\n\n- Although the method is novel and sound, the utility of the proposed method is doubtful:\n\n    1. On tabular data generation literature, a recently influential trend of the use of language models and large language models is not discussed, while they are better-performing than GANs, likely to have a very low CVR (may not be 0% in theory but could be very close to it in practice). If these recent models naturally achieves near-perfect result on constraints, then the proposed method may not be useful in practice. Thus, it needs to be discussed.\n    2. The authors did not compare the proposed method versus direct rejection sampling. Rejection sampling also guarantees 100% satisfaction of all constraints, and according to the reported timing in Section 4.3 (Table 5), it is not necessarily worse. e.g. for URL dataset, non-WGAN models has a CVR of about 10%. This means ~0.15/0.9=0.17s < 0.27s is needed, so it's even faster. Moreover, rejection sampling preserves naturally generated samples from generators instead of editing them, and there may be a chance that its utility can be even better. \n    3. Putting the above two points together, there could be a model with <1% violation rate, so <+1% time for generation, that obtains better result than +DRL in the older models.\n\n\n- A serious issue is that the page limit is 10 but section 7 and 8 in this paper is in the page 11, which can potentially lead to desk reject.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)