Harpoon: Generalised Manifold Guidance for Conditional Tabular Diffusion
Authors: Aditya Shankar
Yuandou Wang
Rihan Hai
Lydia Y. Chen
Venue: ICLR 2026 Poster (2026)
Keywords: Diffusion models
Conditional generation
Tabular diffusion
Manifold learning
Primary area: generative models
TL;DR: Extends manifold theory for handling arbitrary tabular conditional generation tasks at inference time.

Abstract: Generating tabular data under conditions is critical to applications requiring precise control over the generative process. Existing methods rely on training-time strategies that do not generalise to unseen constraints during inference, and struggle to handle conditional tasks beyond tabular imputation. While manifold theory offers a principled way to guide generation, current formulations are tied to specific inference-time objectives and are limited to continuous domains. We extend manifold theory to tabular data and expand its scope to handle diverse inference-time objectives. On this foundation, we introduce Harpoon, a  tabular diffusion method that guides unconstrained samples along the manifold geometry to satisfy diverse tabular conditions at inference. We validate our theoretical contributions empirically on tasks such as imputation and enforcing inequality constraints, demonstrating Harpoon's strong performance across diverse datasets and the practical benefits of manifold-aware guidance for tabular data. Code URL: https://github.com/adis98/Harpoon

Published as a conference paper at ICLR 2026
HARPOON: GENERALISED MANIFOLD GUIDANCE FOR CONDITIONAL TABULAR DIFFUSION
Aditya Shankar' Yuandou Wang' Rihan Hai' Lydia Chen Depparment of Computer Science, Delft University of Technology *Department of Computer Science, Université de Neuchâtel (a. shankar, y. wang-45, r. hai @ @tudell ft. .nl 1 lydiaychen leee. org
ABSTRACT
Generating tabular data under conditions is critical to applications requiring precise control over the generative process. Existing methods rely on training- time strategies that do not generalise to unseen constraints during inference, and struggle to handle conditional tasks beyond tabular imputation. While manifold theory offers principled way to guide generation, current formulations are tied to specific inference-time objectives and are limited to continuous domains. We extend manifold theory to tabular data and expand its scope to handle diverse inference-time objectives. On this foundation, we introduce HARPOON, a tabular diffusion method that guides unconstrained samples along the manifold geometry to satisfy diverse tabular conditions at inference. We validate our theoretical contributions empirically tasks such imputation and enforcing inequality constraints, demonstrating HARPOON'S strong performance across diverse datasets and the practical benefits of manifold-aware guidance for tabular data. Code URL: https: / //ggtttuu.. com adis 988 /HHrrpon
INTRODUCTION
Generating tabular data subject to conditions (constraints) is critical to many tasks, such im-
puting missing values given partial observations (Zhang et al. 2025) or simulating hypothetical
"whattif" scenarios for decision-making (Narasimhan et al.] 2024). Such tasks require strong gen- crative models, among which diffusson models (Ho et al. 2020 Song et al. (202I) have emerged as state-of-the-art (SOTA) across many modalities (Dhariwal & Nichol 202 Kotelnikov et al. [2023 Kollovieh et al., 2023). However, conditional tabular diffusion models (Zheng & Charoenphakdee
2022 Ouyang al. [023] Villaizán- Vallelado al. 2025 Zhang et al. (2025) often rely on training-
time strategies that cannot generalise to unseen conditions at inference, or cannot efficiently handle
diverse objectives, such as inequality constraints on features (e.g. Age >= 10) (Stoian al. 2024).
Recent works in image diffusion offer a geometric view of the diffusion process (Chung et al. 2022 He et al., (2024). treating data lying on a low-dimensional manifold (Gorban & Tyukin 2018 Bengio et al. [0033] PPope et al. 2021). They show that naively enforcing conditions can push generated samples off the manifold, producing unrealistic results. These works propose constraining samples to stay close to the manifold's surface until converging to the desired conditions. However, these theories are limited to continuous domains and assume flat geometries, which may not hold for mixed-type tabular data. They also only analyse specific inference-ting losses (e.g., squared-error) without generalising to diverse objectives, such as inequality constraints.
We extend manifold theory to tabular diffusion models operating in the data space. We first prove that diffusion models implicitly learn an orthogonal mapping onto the underlying manifold when trained using the standard squared-error loss, even without strong assumptions on the curvature. We then prove that the gradient of any differentiable inference- time loss lies in the tangent space of the manifold. With this powerful result, we introduce HARPOON, conditional tabular diffusion method that requires training once and adapts to diverse conditions at inference. HARPOON interleaves tan- gential gradient corrections with unconstrained denoising steps, gradually guiding samples along the

---

Published as a conference paper at ICLR 2026
manifold's surface towards regions satisfying diverse inference- objectives. such imputation or inequality constraints on features. We summarise our contributions as follows:
Theory. We provide the first theoretical results linking manifolds to tabular diffusion, including curved geometries. We prove that diffusion models act as orthogonal projectors, guaranteeing the tangent-space behaviour of gradients from any differentiable inference-time objective.
Algorithm. Building on our theoretical results, we design manifold-awan conditional tabular dif- fusion method, HARPOON, that guides unconstrained samples towards satisfying diverse inference- time tabular generative constraints.
Empirical validation. We empirically validate the geometric alignment of the inference- gra- dients with the underlying manifold, demonstrating HARPOON'S ' strong performance across diverse datasets and conditional tasks, including imputation and inequality conditions.
2 BACKGROUND AND RELATED WORK
We review diffusion models for tabular data and their conditional extensions. We then turn towards manifold-based perspectives from image diffusion motivate our contributions. The key notations used from this section onwards are summarised in Appendix A
Tabular Diffusion. Denoising Diffusion Probabilistic Models (DDPMs) (Ho al. 2020 de- fine a generative procedure by reversing a Markovian noising process, p(Z JT Too
T(=ap(ra which progressively corrupts clean sample To over T steps. For tabular data containing a mix of discrete and continuous features, To is represented as [zgont Ti ..... To (k)
(1)
with continuous features zgnnt and categorical vectors x(( that can be represented using one-hot, integer, or binary encoding schemes. The standard approach for the forward (noising) process across data modalities is to add continuous Gaussian noise to samples, as follows:
V vo xo + 5ce, E e~ I),
where ā. = II s- a is a cumulative noise schedule, with ā₂ + as T Ho et al. resulting noisy distribution is p(zz) :- (z)))))) with p(x,(=0) N (VaNoo (I l.).).
2020 The
The backward process iteratively reconstructs xp from ær using a neural network (a.k.a denoiser) with parameters 0 that predicts (0 (x2) each step by estimating the added noise e. The standard training loss is the Mean Mcan-Squared-Erre Error (MSE) (Ho et al. 2020) using estimates Eg(Ie, t):
Liaain (Exorp(xa),ttee (-U1,,,21, [|| cx 1) 1)
For tabular data, the general standard is model continuous and discrete variables using separate diffusion processes, such as masked (Shi et al. 2025 Austin et al.) 202T] or multinomial (Kotelnikoy) et al. 2023 Hoogeboom al., 2021) diffusion for categorical features. However, these approaches introduce additional training complexity, requiring separate loss functions, such as MSE and cross- entropy for continuous and discrete features, respectively. Another alternative is to stick with single continuous diffusion process and squared-error objective, but instead apply it on smooth latent space embeddings of samples using autoencoders (Zhang et al. 2024 Shankar al. 2024).
Conditional Tabular Diffusion. Conditional generation introduces a variable e to guide the gener- ation process. We use e to broadly denote any user-specified constraint, ranging from binary masks that indicate partially observed entries in imputation tasks (Xu et al.) 2019 Alcaraz & Strodthoff] 2022) to inequality constraints on features (Stoian et al. (2024). A key challenge is that these condi- tions often correspond to diverse and non-linear objectives (Donti et al. (02T). As result, enforc- ing them at inference scales poorly, since existing approaches typically require separate retraining or fine-tuning of the generative model for each new condition.
Conditional tabular diffusion methods typically fall into two categories: training-time and inference- time solutions. Training-time methods learn the conditional distribution pe(2x-1 c) during train- ing, using c as model inputs (Zheng & Charoenphakdee [022 Villaizán- Vallclado et al. 2025] Ouyang et al. 2023). However, this limits generalisation to unseen conditions at inference. Clas- sifier guidance (Dhariwal & Nichol 2021), trains an auxiliary classifier to predict the gradient
We use p for the forward process; Po denotes the learned denoising process parameterized by 0.
2

---

Published as a conference paper at ICLR 2026
VE, log p(c(ar)). to steer sampling (Liu al. 2024 While this is effective when denotes discrete labels, it is not applicable to real-valued constraints. Moreover, the pre- pre-trained classifiers cannot guide samples towards conditions unseen during training. Classifier-free guidance (Ho & Salimans 2021] avoids the need for separate classifiers by repurposing the diffusion backhone. It jointly trains and interpolates between conditional and unconditional model to control the degree of guidance. However, it is still limited by the conditional branch, which requires training on constraints known in advance, affecting inference-tine generalisation.
In contrast, inference-time methods for imputation (Zhagg et al. 2025) guide generation by for- ward noising observed values and jointly denoising observed and missing values, similar image inpainting (Lugmayr et al.] (022). While effective for imputation, they require partially observed ground truths and cannot handle inequality constraints. Rejection sampling (Casella et al. 2004) general fallback, but is impractical due to high violation rates.
Finally, we note that conditional generative methods typically operate the data space, where constraints are specified directly on features. Conditioning latent methods is challenging because it requires translating data space constraints into their latent counterparts, which is only possible when there is deterministic mapping between the two spaces (Zhang et al. 2024).
The Geometry of (Image) Diffusion. Recent works in image dif- fusion interpret the diffusion process as sequence of manifold transitions (Chung et al. 2022 He et al. 2024). A clean sample low-dimensional manifold Mo Rd. The forward process gradually perturbs MM into a family of sur- rounding shells {M. where MT approximates pure Gaus- sian noise (see [Figure T). Noisy samples concentrate near these shells, which the model learns denoise. Enforcing constraints by pushing samples orthogonally toward Mo risks "skipping" shells, yielding off oiffiistribution inputs where model estimates become un- reliable (Chung et al. 2022). Crucially, Chung et al. (2022) show that the gradients of inference-time squared-error objectives lie in the tangent space of MM. and leverage these tangential corrections to help keep samples within the current shell while steering them along the manifold geometry enable condition -ondition-awwre aware denoising.
To is assumed lie
M
#t
M
M,
N(,,(
Figure 1: Geometry of forward
(1) and backward (1) diffusion.
Extending these insights to tabular diffusion is not trivial. First, ex-
isting theories assume continuous features, while tabular models have both continuous and discrete features (Hoogehoom et al., 2021 Kotelnikov et al., 2023). thereby violating these assumptions. Second, current theoretical proots implicitly assume flar manifold geometries (Chung et al. [022] He et al. 2024), which need not hold for mixed-type tabular data. Third, the tangent-space guar- antees only hold for squared-error losses (Theorem 1, Chung al. (2022)). In contrast, tabular conditions, such as inequalities or imputation masks, need diverse, often non-linear differentiable losses (see eqs. and 10 Appendix Dj, which is beyond the current theoretical scope.
3 MANIFOLD GUIDANCE FOR TABULAR DIFFUSION
We first derive the theoretical foundations for HARPOON and then detail its algorithm for conditional tabular generation. We define the necessary notations and assumptions, leading up to our main theoretical contribution: a manifold guidance technique for any differentiable inference-time loss.
3.1 PRELIMINARIES AND ASSUMPTIONS
We represent a tabular sample xp E IRd as concatenation of continuous and one-hot encoded categorical features, same Sec. We assume that the support of the data distribution p(xo) lies on a continuous smooth manifold Mo C IRd (see Remark [). Here, d denotes the ambient space dimension, and dim(Mo) is the intrinsic dimension of the data manifold, with n < d. We formalise this with standard geometric assumptions (Chung et al. 2022).
Assumption 1 (Smooth Embedding). The support of the clean data distribution p(zo) lies on smooth, connected, n- n-dimensional manifold Mn embedded in the ambient space R', where n < d. Formally, there exists smooth embedding yp: R IRd such that Mo (R().).
3

---

Published as a conference paper at ICLR 2026
Assumption 2 (Local Linearity). For any point zo MMg the manifold is locally well well-approximated by its tangent space T sphere B, fx E R2 || zoll2 <, the manifold coincides with its tangent plane:
Tz, Mo at Zg. That is. there exists a radius r >> 0 such that within the hyper-
Mon (x)) = Mo n (z0)))
Intuitively, sufficiently zooming in around Zo makes the manifold appear locally flat.
Remark 1. [Tabular representation] sample 2o Fd consists of continuous and categorical and 2 to tabular data, we assume the
variables in the tabular setting. To extend Assumptions I support of the encoded points resides in a continuous space. For example, a discrete feature with K classes can be encoded as a one-hot vector, or as "soft" probability distribution over the classes. This relaxes categories into points within the (K-I)-simplex, giving continuous embedding. While such continuous approximation may be suboptimal, we later show that our theoretical derivations provide a way to overcome this limitation for tabular diffusion.
Building on these assumptions, we connect the forward process to the geometry of noisy manifolds. From eq. [ each 2, that is the outcome of the forward process can be viewed lying near scaled copy of the clean data manifold, perturbed by Gaussian noise (see Figure I). We formalise this intuition in Proposition T formally proved in He et al. (2024).
Proposition 1 (Shell Structure). During the forward diffusion process, noisy samples at diffusion step probabilistically concentrate on (d. - dimmensiil manifold Mr forming shell around a scaled copy of the n-dimensionaa data manifold Mo:
M == fo ER" d(z, varMo) =V(1===(1-- (1)
Rd
Here, d(x, Va,Mo) == in -|=- Vazz'l denotes the Euclidean distance from æ to the closest point on the scaled manifold VarMo. Note that + 1 as t 0. As increases, the shells drift further away from Mo. giving the structure shown in Figure 1
Under the standard training objective in eq. 2 the model learns to denoise points concentrated on the noisy manifolds Mr, iteratively reconstructing clean data To from Gaussian noise.
3.2 GENERALISED MANIFOLD THEORY
Based on the Gaussian noising process and denoiser training objective introduced in Sec. 3.1 now develop a generalised manifold theory for conditional tabular diffusion. We first define the denoiser mapping Qt, which predicts dirty estimates from noisy samples x; by reversing equation !
Definition ("Dirty" Estimate via Diffusion Model). Let ea c0(2, t) denote the diffusion model's 's noise estimate. We define the dirty estimate of a noisy sample x, by reversing eq. (forward process).
1 Vor
to = Qi(x.) =
[1 vI 11--a āe ()(
(3)
where Q Rdd + Rd maps a noisy input to point dn MM. Intuitively, Qe(xe) lies near the clean data manifold Ma. since is a reversal of the forward process (eq. 7 that noises a clean sample.
We show that for a model trained under Gaussian noising process using eq, 2 Qi naturally aligns with the orthogonal projection of x, onto Mg. as formalised in the following theorem.
Theorem 3.1 (Limiting behaviour of dirty estimates). Let eœ be a denoiser trained to predict Gaus- sian noise under the framework of egs. [ and 2 for a clean sample zo. Then, for the mapping Qt defined in Definition! | we have lima.-,1 Qx(r.) (Ix), where x(xx) denotes the orthogonal projection of xt onto the data manifold Mo-
The formal proof is in Appendix B Theorem 3.1 establishes a general and practically relevant guar- antee that the dirty estimates Q (Zz) map the orthogonal projection T(x;)) on the data manifold as 5e - 1. /Fiuure 2a intuitively explains this behaviour, where the "spotlight" for the dirty estimate's density distribution sharpens around the orthogonal projection F, approaches Mo-
We note that Chung et al. (2022) present a related claim (their Proposition 2), which does not require the limiting Or -> This is because their proof implicitly assumes the manifold is globally
4

---

Published as a conference paper at ICLR 2026
21
1
m)
(.(c.)
[.as.
Mr
A A
Mp- Nio,
(a) Spotlight of 2: projections
(b) Imputation constraints
(c) Inequality conditions
Figure 2: (a) Shows the orthogonal behaviour dirty estimates projected from Ht to Eg. Figs. (a) and (b) show HARPOON' s guidance mechanism, interleaving unconditional denoising (-) and tan- gential updates (t) for (b) imputation constraints of the form (1 m) @ øe. and (c) inequality constraints. Yellow regions indicate the areas on the manifold Mo matching the constraint.
flat, allowing them to invoke symmetry arguments to explain the orthogonal behaviour (detailed in Appendix B). We generalise their result, proving that the orthogonal behaviour arises without such restrictive assumptions, but in the limit as â₂ + 1. We use Theorem 3.1 to rigorously justify the assumptions used in the following theorem.
inference-time gradients). Let e denote arbitrary condition- ing information and assume Q, from Definition / acts as orthogonal projection onto the data manifold Mo at åg = Qu(xe). Then, for any differentiable inference-tine time loss function Cint Fdd x C E. defined by (fo(ef the gradient lies in the tangent space of Mo at To. Le., Vz, Lin c) Mo.
Theorem 3.2 -Manifod-constraainee
We provide the proof in Appendix B Theorem es- tablishes powerful result that the gradients any dif- ferentiable objective at inference (Lint) are aligned with the tangent space of the data manifold. This gener- alises prior work, which proved the tangential behaviour only for squared-error inference-time losses the form ||W - H (1 under restrictive symmetry and manifold-lincarit assumptions (Theorem 1, Chung et al. )022). Our formulation is strietly more general, cov- ering curved geometries. It applies to arbitrary differ- entiable losses, supports rich spectrum of inference conditions c, ranging from partially observed values (e.g. imputation masks) prompts, metadata, inequality constraints. Importantly, the inference-tine losses (Cint) can be different from the training loss (Cersin. 24.
with MSE Cross Entropy
with MAE Cross Entropy
Angles in degrees
only
with MAE only
90 degrees
42-
22
1.0
0.2
Diffusion step
Figure 3: Avg. angle between gradients (100 samples) and dirty estimates So for various loss functions on Adult.
Tabular adaptation. Theorem 3.2 is critical for tabular data. Even if training relies on continuous Gaussian noising framework to avoid separating dis- crete and continuous diffusion, inference can incorporate losses tailored to discrete variables, such as cross-entropy, inequality constraints, or even L -based penalties with sparsity-inducing proper- ties Bach et al. (2012) suited to tabular domains.
Empirical orthogonality. Strietly speaking, the tangential behaviour of gradients requires the dirty estimate Qz(xz) to be orthogonal, which is guaranteed only in the limiting case ār - i.c., as t O. However, practice, we observe that the gradients remain close to 90" with Qx(xr. even larger time steps, suggesting that the orthogonality property extends well beyond the limit (see Figure 3). Additionally, Figure 3) shsows a consistent behaviour across varying inference-time losses under the same training objective (MSE), giving empirical verification of Theorem 3.2
3.2 together with the observations in Figure 3 directly motivate HARPOON'S 's de- sign, which functions like high-precision guidance system. The denoiser estimate (x.) acts like
Theorems 3.1 and and
5

---

Published as a conference paper at ICLR 2026
compass, pointing toward the orthogonal projection onto Mu- Meanwhile, the manifold-aligne gradients act as anchors, steering updates tangentially along the manifold surface, keeping samples within the manifold shells for reliable model estimates. The idea is analogous to a ship navigating against a current through steady movements along the manifold without drifting off course.
Leveraging this intuition, HARPOON interleaves unconditional denoising steps with tangential gra- dient corrections, guiding samples towards the specified conditions. Since consecutive manifolds are nearly parallel (Proposition I] we can (unconditionally) denoise one step followed by a tangential correction using gradients from the previous step, gradually converging towards the target region (see Figs. 2b(2c). This guidance system naturally adapts a variety of tabular conditions. For inequality constraints (Figure 2c). the target areas span region on the manifold's surface, and Har- poon guides samples "land" within this region. For imputation-styles tasks (Figure 2b). the con- straints instead anchor samples to the partial observations on the manifold of the form - m.) 20 (see (Appendix D). lying on a constraint line for a given missing value binary mask m.
At a high level, this interleaved sampling is structurally similar to prior training-frre guidance meth- ods (Chung et al. 2023; 2022 He et al., 2024| [aasall et al. 2023 Song et al. 2023 Yu et al.. 2023). The distinction in the class of theoretically justified objectives. Existing work either justifies guid-
ance only under restrictive flat-manifold assumptions (Chung et al. 2022 2023 He et al. (024), or
by applying inference-time losses without analysis of geometric compatibility with the underlying
manifold (Song et al.] 2023) Bansal et al., 2023] Yu et al.] 2023). We remove this dichotomy by generalising the tangent-space behaviour of guidance even to arbitrary inference-time objectives.
3.3 HARPOON ALGORITIIM
We concretely detail the guidance process in Algorithm [ At each step i, we first compute the dirty estimate using equa- tion 3 We then pre-compute the tangen- tial gradient corrections by applying the the inference-time objective (Lint) on the dirty estimates, i.e., G). Then, the sample is unconditionally denoised one step using the standard re- verse process (Ho et al.) 2020) (line K). As the denoising step is unconstrained, we ap- ply the pre-computed tangential correction to guide the sample towards the objective using a guidance strength 1) (line
Algorithm 1 Sampling with HARPOON
1: Initialize .r ~ ~N(0
2: for i
... T do 3: Sample ~ .V ift ift > 1, else = = 0 Estimate noise c0( t)
c) (line
5: Obtain dirty estimate du (eq.
6:
Get tangential gradient
2. (in c)
Set (1-a))
7:
(1-ac)
8:
Denoise one step:
2/-1 va. 2, VI-ai
1-a co( t)) + 022
Apply tangential update:
It-1 x -7 -9
Importantly, the inference-time loss Cinr (4o, c) can be different from the training objective, including imputation or ReLU-based inequality losses (Donti et al. 202 (see Appendix D). handling rich variety of generative tabular conditions. Moreover, since tabular data is represented using sparse one-hot Aean-Abbolute-Ere (MAE) loss for
10: return 20
encoded discrete features, we use the sparsity-inde -nnduiiginnuuinn improved convergence at inference time by default (Bach et al. 2012).
4 EXPERIMENTS
We evaluate the practical benefits of HARPOON's manifold-guin updates for conditional gen- cration across two representative tasks: (i) imputation of missing values from partially observed samples, and (ii) inequality constraints, which enforce conditions without relying ground-truth targets. We further analyse the effect of tabular-specific losses at inference time, study different guidance schedules, and measure runtime efficiency. Additional experiments, covering alternative mask patterns, encoding schemes, and evaluation metrics, are reported in AAppendix
4.1 EXPERIMENTAL SETTINGS
Datasets. We evaluate on eight widely used benchmarks, mostly from the UCI repository (Dua et al. 2017). Five with only continuous features (Gesture, Magic, California, Letter, and Bean), and

---

Published as a conference paper at ICLR 2026
Table I: Imputation MSE on continuous features for MAR mask. Standard deviation in subscript.
Ratio
Method
Adult
Bean
California
Default
Gesture
Letter
Magic
Shoppers
GAIN
1.860.04
1.4
15.06
6,861.81
1.22o
1.050.04
1.27
1.670.26
Miracle
2.17a.34
0.31o.12
0.50g.11
1.51g.3e
0.94g.10 10
1.320.11
0.88g.14
2.210.76
GRea'T
3.70
1.820.05
1.55g.05
11.3915.12
>10
1.670.04
1.64g.15
1.51o.10
0.25
Remasker
0.92a.04
0.230.06
0.890.03
0.56
0.70p
0.576.03
0.81o.1
0.72o.12
DiffPuter
1.04
0.18p
0.470.04
0.62o.o6
0,40p
0.420.04
0.770.18
0.760.10
HARPOON
0.990.08
15.06.
0.424.04
0.5 1g.0
0.5
0.52p.04
0.97g.20
0.73p.06
GAIN
3.376.21
1.940.12
18.69
23.177.05
2.97
1..03
1.366.45
3.
Miracle
2.080.40
0.490.06
0.92a.07
1.03a.19
1.04g.00
0.83o.03
1.07a.14
1.360.28
GReaT
1.61a.21
4.944.66
1.41a.19
1.84
>10
1.390.04
1.24g.11
>10
0.50
Remasker
0.94
0.420.06
0.90g.os
0.59g.06
0.97
0.78p.03
0.77g.0s
0.82
DiffPuter
1.180.09
0.280.06
0.79u.12
0.640.07
0,480.06
0.750.04
0.80g.os
0.88
HARPOON
1.080.08
0.190.05
0.72g.07
0.53g.08
0.63o
0.860.05
0.86g.
0.81
GAIN
7.440.21
3.100.07
22.741.30 33
62.2310.77
9.701.1r
1.299.03
2.04g.39
14.85
Miracle
1.270.04
0.750.06
0,980.02
>10
1.23
1.00..02
1.02a.03
1.100
GReaT
1.70
4.552.46
1.43
2.10g.58
>10
1.440.04
1.199.0s
1.28
0.75
Remasker
1.03p.01
1.140.u7
1.13a.0s
0.89p.us
3.98p.63
1.150.02
1.30u.10
1.170.03
DiftPuter
1.499.09
0.610.04
1.10p.08
0.89
0.71
1.220.04
1.08
1.260.08
HARPOON
1.26p.07
0.370.03
1.02a.07
0.72
0.84p 08
1.300.05
1.00g.08
1.10p.05
three with both discrete and categorical features (Adult, Default, and Shoppers). The preprocessing steps are detailed in Appendix C
Each dataset is randomly split into 70% training and 30% test sets.
Baselines. We compare against SOTA tabular methods spanning causal graphs. deep learning, gen- erative, and large language model based (LLM) approaches. The baselines include: (i) MIRA- CLE (Kyono et al. 202I], causal graph-based imputation method, (ii) DifiPuter (Zhang et al.)
2025
SOTA conditional tabular diffusion model, (iii) GReaTT (Borisov et al. (023), an L.M-- based method for tabular generation and imputation, (iv) Remasker (Du et al. 2024), a masked au- toencoder imputation method, and (v) GAIN (Yoon et al. (2018), conditional tabular GAN model. For fairness, all diffusion- approaches, including ours, share the same backbone architecture. Details on the baseline implementations and hyperparameters are provided in in/ppendix G
Evaluation Protocol. Since our goal is to enable conditioning at inference, all models are pre- trained once and then evaluated strictly at inference time across different tasks, including impu- tation and inequality conditions. For the imputation tasks, we evaluate under missingness ratios of 0.25, 0.5, and 0.75. Continuous features are evaluated using the Seean-Squreed-FFee (MSH) between imputed values and the ground truths, while categorical features are assessed using the match accuracy. We consider three missingness patterns following prior work (Muzellec et al., 2020]: Missing-AI-Random (MAR) (main text), and Missing- Completely-At-Raane Missing-Not- Not-AAt-ntt (MNAR) (Appendix I.T). Dataset preprocessing and mask generation de- tails are provided in AAppnddxx E, For the general mequality conditions, we consider four scenarios: (i) range constraints on continuous feature (e.g. 12 Age >= 10); (ii) categorical constraints, where a discrete feature is fixed to target value (e.g. Gender = 'Male'); )); (iii) conjunctions (AND), where both the range and categorical conditions are applicd simultancously; and (iv) disjunctions (OR) of range and categorical constraints, where either one may be satisfied. Details of the con- straint specification and implementation are provided in Appendix E For these tasks, we report the constraint violation rate (Stoian et al.] 2024). measuring the fraction of samples that break the enforced restriction, the a-score (Alaa et al., (2022), which quantifies fidelity, and the utility using a downstream XGBoost (Chen & Guestrin 2016) classifier trained the synthetic data and evaluated against a holdout test set (additional details in Appendix IT) We also provide results using additional metrics in Appendix L.5
(MCAR) and
All results are averaged across five trials to account for variance.
4.2
IMPUTATION RESULTS
Tables and report the MSE and imputation accuracy for continuous and categorical features respectively. For continuous features, HARPOON (ours), DiffPuter, and Remasker are generally the best best- performing. While the inference-time methods (DiffPuter and HARPOON) remain robust even under high missingness scenarios, training-time methods, such as Remasker and GAIN, de-
7

---

Published as a conference paper at ICLR 2026
grade sharply under increased missing ratios (e.g.. Gesture at 75%), probably because they rely on training- time mask patterns and missing ratios, which limits generalisation at inference. LLM- based generation with GReaT is unreliable, occasionally producing extremely high MSE values (e.g. gesture), indicating their limited effectiveness for tabular tasks. While the causal-learning method, MIRACLE, sometimes achieves low errors, its performance under high missingness (c.g., Default at 75%). This likely arises because MIRACLE constructs a fresh imputer per task at inference rather than leveraging a pre-trained model, which can fail when insufficient information is available under high missing ratios.
be unstable, collapsing entirely
For categorical features, HARPOON often achieves the highest accuracy across nearly all datasets and missingness ratios, outperforming DtfrPuterr, Remasker, and GReaT. While DiffPuter remains competitive, its RePaint strategy (Lugmayr et al., 2022) of independently noising observed and masked entries can introduce semantic mismatches, reducing alignment with observed categories. Most other methods, except for GReaT, typically only model continuous features, resulting poor performance on categorical imputation. Overall, we see that HARPOON provides robust and reliable inference-time imputation across both continuous and categorical features.
4.3 RESULTS ON INEQUALITY CONSTRAINTS
Table 3 compares the best generative baselines from Tables and 2 (HARPOON, DiftPuter, and GReaT) on general tabular inequality constraints. Determin- istic imputers such as Remasker are excluded, as they produce identical outputs when inputs are iden- tical, making them unsuitable for such tasks which do not have partially observable ground-truths.
Table 2: Categorical imputation accuracy (t) for MAR mask.
Ratio
Method
Adult
Default
Shoppers
GAIN
22.11
51.302.60
41.257.70
Miracle
42.10s 85
58.513.05
27.22, su
GReaT
22.881.60
44.130.40
37 37.69%.7
0.25
Remasker
11.81
69.63
39,833.00
For range constraints, both DiffPuter and GReaT exhibit high violation rates (74-96%) This likely stems from their approach of treating conditional tasks as standard imputation problems, which is ill- suited to inequality constraints because there is no observed ground truth, only a plausible range. As a result, these methods rely on rejection sampling. In contrast, HARPOON handles these tasks using dif- ferentiable loss specifically designed for inequalities (see eq. 9 Appendix D). This provides greater flex- ibility, reducing violations to below 8% on Adult and Default, and 20% on Shoppers, while generally achieving the fidelity and utility scores.
DiffPuter
69.473.05
71.58:.5
56.19,.o
HARPOON
69.444.11
73.49h.5
51.17, nu
GAIN
21 21.855.01
42.28g.24
27. 27.06e.u
Miracle
33.73x.80
55.1 1199
30.09,
GReaT
45.972.50
55.81..7
37.12,
0.50
Remasker
45.1611.05
63.00
38.44,
DiffPuter
63.33,.56
62.92g.0
52.14,
HARPOON
64,80;
69,55
48.39, on
GAIN
22.411.45
20.
17.614.59
Miracle
35.46..51
41.74...87
38.33,.e
GReaT 50.164.07
56.93
37,894.8
0.75
Remasker
38.67, ..4
50.28:118
44.15, 79
DiffPuter
56.99,.a1
49.24g,
50.94,
HARPOON
58.81
61.48,.88
48. 48.374.30
For categorical constraints, all methods achieve near-zero violations. However, HARPOON gen- erally attains the highest precision, despite a slightly higher violation rate than the others. This is because both DiffPuter and GReaT handle categorical conditions like imputation tasks and enforce ground-truth conditions (e.g., color 'red') using hard substitutions, which can push samples off- manifold. In contrast, HARPOON enforces constraints through soft differentiable losses, allowing all features to be coherently adapted via gradient steps at the cost of small violation rate (52%), Hence, samples remain aligned with the manifold giving outputs with high fidelity and utility scores.
Under conjunctions, DiffPuttrr and GReaT frequently fail, with violation rates between 78-97%. In contrast, HARPOON maintains low violations (2-12%) while maintaining strong fidelity and utility across datasets. This demonstrates its ability to flexibly satisfy diverse types of constraints without sacrificing sample quality. Disjunctions show that HARPOON has lowest violation rates amongst methods, and is also consistently lower than the worst its range or categorical constraint in isolation. This is intuitive, since the conjunctions increase the feasible region, making it easier to satisfy. However, fidelity scores sometimes show a stark drop (Default, Shoppers). We hypothesise that this is because the model's gradient updates may prioritize one condition over another, skewing the distribution toward certain feasible regions (see/Figure 4) Appendix().. Nevertheless, high utility scores indicate that the generated samples remain useful for downstream tasks.
8

---

Published as a conference paper at ICLR 2026
Table 3: Constraint Violation rate (4), fidelity (a-score (t)) and downstream utility (1) of methods
Constr.
Method
Adult a-score
Default
Shoppers a-score
Viol. %
Utility
Viol.
c-score
Utility
Viol. %
Utility
DiffPuter
73.861.00 03
0.83c.o2
0.940
88.531.21
0.65p.02
0.78
76.69..0
0.688.oz
0.89
Range
GReaT
83.58p.c9
0.92
0.95o.00
95.37a.a3
0.670.01
0. 0.70o.15
96.533.46
0.88p.co
0.35g.43
Harpoon
3.060.26
0.92p.01
0.95o.00
7.780.60
0.79p.02
0.85o.01
20.55o.95
0.60p.o1
0.89..01
DiffPuter
0.00p.00
0.83p 02
0.870.00 m1
0.00g.00
0.85p.02
0.81
0.00p.co
0.52p.02
0.82p.
Category
GRea'
0.00,.00
0.72p.02 02
0.87 01
0.00p.00
0.85p.o1
0.83
0.00p.co
0.86
0.71p.35
Harpoon
2.18
0.86
0.88
0.91p 19
0.92p.o1
0.83
p.00
0.
0.88
Range
DiffPuter
78 78.88
0.73p.04
0.920.03 .0
91.30p.87
0.59p.os
0.56o.15
78.68
0.76p.02
0.85
AND
GReaT
80.22
0.90p.o1
0.91o.02 02
94.79
0.61p.02
0.52
95.231.00
0.73p.02
0.84p.01
Category
Harpoon
3.37
0.82p.os
0.919.as
2.98, 77
0.87
0.81
12.05
0.81p.04
0.85
Range
DiffPuter
63.22p.o7
0.88
0.93,
23.03
0.88p
0.82
50.566.az
0.53g
0.88
OR Category
GReaT
67.14g.31
0.88g.01
0.94.
24.04..14
0.90p.00
0.84
65.766.a1
0.82
0.35g
Harpoon
3.040.07
0.87p.01
0.933
4.23
0.65p.01
0.8
20.20
0.47p.01
0.89
Table 4: Harpoon's 's imputation performance (MAR) under different inference-time losses using a model pre- trainrried on MSE loss.
Adult
Default
Shoppers
Ratio
Loss
MSE (1)
Acc. (f)
MSE (1)
Acc. (t)
MSE (1)
Acc. (f)
MAB
0.99
69.44a.11
0.51o.o6
73.492.53
0.73p.00
51.175.20
MSE
1.15g.09
68.393.04
0.68
71.572.16
0.86p.11
51.515.32
0.25
MAE + CE
1.050.10
69.303.o2
0.540.07
73.012.48
0.77p.08
51.133.35
MSE CE
1.3
66.622.90
0.770.12
71.392.45
1.04p.1e
51.275.28
MAB
1.08g.o0
64.80
0.53 06
69.55
0,81
48.39;.00
MSE
1.19g.07
63. 63.283.07
0.670.0e
67. 67.601.76
0.90p. 10
48 24.244,,
0.50
MAE + CE
1.14g.0s
64.569.15
0.55g.07
69.261.00
0.85p.06
48.05 48.054.98
MSE + CE
1.32g.08
62. 17.90
0.73g.00
67.721.76
1.07
48.045.06
MAE
1.266.07
58.81
0.72g.o6
61.483.83
1.10p.05
48.37
MSE
1.38g.
57.183.41
0.88p.07
58.813.93
1.170.05
47.76,..10
0.75
MAE CE
1.30
58.733..8
0.73n.07
61.38
1.150.04
48. 12
MSE CE
1.44g.04
57.20
0.9 1.91
59.634.11
1.29
47.684.58
4.4 ABLATION: CHOICE OF INFERENCE-TIME LOSS
Table compares the imputation scores of various inference-time loss functions for HARPOON. Across all scenarios, the best performance, both in MSE and imputation accuracy scores, con- sistently comes from MAE MAE with cross-entropy (CE). In contrast, the MSE loss variants perform worse, despite MSE being the training-time time objective. This is because MAE is sparsity- inducing, which aligns well with the structure of discrete features represented as sparse one-hot vectors, while MSE "spreads" errors across dimensions, giving imprecise categorical predictions.
Remark The ablation results Table 4 have important implications. It shows that using tabular-specific loss at inference (e.g. Saarsity-nnuucig MAE) can even outperform the loss used training, directly verifying Theorem Once the denoiser learns to project orthogonally onto the data manifold, the gradients of any differentiable inference-time objective lie in the tangent space of the manifold, enabling guidance with respect to arbitrary loss choices. Notably, this does not require special handling of discrete features during training or separate discrete and continuous diffusion processes, like the ones introduced in Sec.
4.5 ABLATION: CHOICE OF GUIDANCE SCHEDULE
Motivated by the discussions Sec 3.3 and the observations in Figure 3| Harpoon injects tangential corrections at every denoising step during inference, since the gradients remain nearly orthogonal for the entire denoising process. But Theorem 3.2 says that orthogonality strictly holds only as outputs approach the manifold Mo. Therefore, compare two strategies: a linear schedule, which

---

Published as a conference paper at ICLR 2026
Table 5: Linear vs. fixed guidance strength across denoising steps (MAR).
Adult
Default
Shoppers
Ratio
Type
MSE (L)
Acc. (1)
MSE (L)
Acc. (1)
MSE (4)
Acc. (1)
Linear
0.990.0s
68.912.92
0.510.06
73.21
0.68p.07
50.70%.61
0.25
Fixed
0.990.08
69.443.11
0.510.06
73.492.53
0.73p.06
51.175.20
Linear
1.090.07
63.463.19
0.54g.0s
67.162.33
0.80
47.305.33
0.50
Fixed
1.080.06
64.80
0.53p.06
69.551.91
0.8
48.395.00
Linear 1.370.07
55.49
0. 0.780.07
44.644.29
1.160.07
46.704.76
0.75
Fixed
1.260.07
3.88
0.72g.06
61.483,83
1.10p.05
48.374.39
gradually increases the guidance strength (17) up to maximum of 0.2 during denoising, and our default constant schedule, which maintains a guidance strength 0.2 for steps (Table 5). Across all scenarios, the linear schedule consistently leads to lower imputation accuracy and higher MSE. We hypothesize that this occurs because even approximate guidance early in the denoising process helps steer the model toward feasible regions. Delaying guidance until later steps may leave the model insufficient time to converge to regions consistent with the constraints.
4.6 RUNTIME COMPARISONS
Table 6 compares the runtimes of various methods on three datasets (more datasets in Appendix L.4) using 12-core AMD Ryzen 9 5900 processor with an NVIDIA
RTX 3090 GPU, CUDA 12.1, Ubuntu 20.04.6, and Pytorch 2.2.1 with Python 3.12.
We observe that HARPOON has runtime within roughly 2x of its "unconditional" proxy, DiffPuter. This is expected because DiffPuter performs only the forward pass and does not propagate gradients, whereas HARPOON interleaves denoising steps with tangential gradient updates, which doubles the func- tion calls to the backbone network. Nevertheless, HARPOON remains extremely efficient, taking less than 5 seconds even on the largest dataset, Adult. In contrast, the LLM autoregressive paradigm (GReaT) and the causal graph-based method (Miracle) are significantly slower and have lower imputation quality. fast, but produces poor-quality generations compared to diffusion- -asee approaches.
Table 6: Sampling runtimes (seconds) of methods (MAR, 0.25 mask ratio).
Method
Adult
Default
Gesture
GAIN Miracle
0.14g.26
0.15
0.14p.
33.030.99
44.540.29
23.09
GReaT
65.47...7
202.717.44
131.72
Remasker
2.685.24
0.33o.53
0.30p 54
DifiPuter
2.830.27
2.62a.28
0.98p
Harpoon
4.430.34
4.05a.34
1.54p.36
The GAN-based method, GAIN, is very
CONCLUSION
We propose HARPOON, a tabular diffusion method for guiding generation to meet diverse con- straints. With geometric perspective on tabular diffusion, we prove that denoisers act orthogonal projectors, guaranteeing the tangent-space behaviour inference-time gradients. This allows train- ing models with simple losses, while allowing for task-specific objectives be applied at inference.
HARPOON'S ' unlocks practical benefits for real-world tabular modelling. handles imputation, inequality constraints, and heterogencous feature types without special-case training procedures. Its manifold-guides updates provide general guidance mechanism for enforcing constraints at inference, making generation robust and adaptable across diverse tasks. We empirically validate our theoretical findings and show HARPOON'S s adaptability in handling diverse constraints.
Limitations and Future Work. We model tabular diffusion continuous process fit existing manifold theories. However, extending the theory itself to discrete noising mechanisms (Kotelnikov al. 2023 Shi al. 2025) would broaden applicability to other diffusion designs. Beyond tables, applying inference-time guidance to other modalities is a natural next step for improvement.
Overall, our work introduces new design principle for conditional tabular diffusion, bridging the gap between theory and practice: keep training simple, scale at inference.
10

---

Published as a conference paper at ICLR 2026
Acknowledgements. This research is partly funded by Priv-GSyn project, 200021E 229204 of Swiss National Science Foundation, the DEPMAT project, P20-22 / N21022, of the research pro- gramme Perspectief of the Dutch Research Council, and ASM international NV. This publication was also supported by the Dutch Research Council (VI. feni.222.439) the Smart Networks and Services Joint Undertaking (SNS JU) under the European Union's 's Horizon Europe Research and Innovation programme (Grant Agreement No. 101192)
Reproducibility Statement. We provide full implementation details, including model architec- tures, training hyperparameters, and evaluation protocols in the Appendix. All experiments are run with fixed random seeds and repeated across five independent trials report mean and standard deviation. Public datasets are used for all experiments, and we open-source our anonymised code: https: / thub. com/ adi s 98/ /Harrr Harpoon We additionally also provide the full proofs for our theoretical contributions in the Appendix.
Ethics Statement. This work advances theoretical and methodological foundations for conditional tabular generation. By enabling flexible inference-time guidance, our framework can support appli- cations where respecting domain constraints essential, such as enforcing physical laws in scientific simulations, adhering to medical ranges in healthcare analytics, or satisfying financial regulations in risk modelling. All experiments are conducted on standard, publicly available benchmark datasets, and our approach is intended to be building block for reliable, constraint-aware generative mod- elling in practical domains. Regarding Al use, Large language models were used to assist us with writing and editing, but all the ideas, designs, and analyses are own.
REFERENCES
Ahmed Alaa, Boris Van Breugel, Evgeny S Saveliev, and Mihaela Van Der Schaar. How faithful is your synthetic data? sample-level metries for evaluating and auditing generative models. In International conference on machine learning, pp. 290-306. PMLR, 2022.
Juan Lopez Alcaraz and Nils Strodthoff. Diffusion-based time series imputation and forecasting with structured state space models. Transactions on Machine Learning Research, 2022. ISSN 2835-8856. URL httt::::///peeeeiiww. wttp://www..eee//ommddamd-a.aaa
Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in neural information processing systems, 34: 194:17981-11 1999999999, 2021.
Francis Bach, Rodolphe Jenatton, Julien Mairal, Guillaume Obozinski, et al. Optimization with sparsity-induche penalties. Foundations and Trends@ in Machine Learning, 4(1):1-106, 2012.
Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Universal guidance for diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 843- 852, 2023.
Yoshua Bengio, Aaron Courville, and Pascal Vincent, Representation learning: A review and new perspectives. IEEE transactions pattern analysis and machine intelligence, 1798-187888888, 2013.
Vadim Borisov, Kathrin Sessler, Tobias Leemann, Martin Pawelczyk, and Gjergii Kasneci. Lan- guage models are realistic tabular data generators. In The Eleventh International Confer- ence on Learning Representations, 2023. URL https://////.....tt. http///pppeeeeeeeeeeeeeeeeee net net/forum?id IEygmONOeI
George Casella, Christian P Robert, and Martin T Wells. Generalized accept-reject sampling schemes. Lecture notes-monograph series, pp. 342-347, 2004.
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp. 785-794, 2016.
Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, and Jong Chul Ye. Improving diffusion models for inverse problems using manifold constraints. Advances in Neural Information Processing Systems, 15:25683-256966 2022.
11

---

Published as a conference paper at ICLR 2026
Hyungjin Chung, Jeongsol Kim, Michael Thompson Mccann, Mare Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh Interna- tional Conference Learning Representations, 2023. URL httpps://pperreveew
forum? Darum?id=OnD d=OnD9 7
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8799 87-4874,22 2021.
Priya L. Donti, David Rolnick, and J. Zico Kolter. DC3: A learning method for optimization with hard constraints. In 9th International Conference on Learning Representations, ICLR 2021, Vir- tual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL htttttst///peeeeviiiie
net/ Forum? d=Vi ZHVxJ 66SS5
Tianyu Du, Luca Melis, and Ting Wang. Remasker: Imputing tabular data with masked autoen- coding. In The Twelfth International Conference Learning Representations, ICLR 2024, Vi- enna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL htttps::/ppenreeiiew..co
[Eorumid=-KI LLV
Dheeru Dua, Casey Graff, et. UCI machine learning repository. 2017. URL/http:/arcchive ics. uci. edumml
AN Gorban and IY Tyukin. Blessing of dimensionality: mathematical foundations of the statistical physics of data. Philosophical transactions. Series A, Mathematical, physical, and engineering sciences, (1022111)20100233 2018
Yutong He, Naoki Murata, Chich-Hsin Lai, Yuhta Takida, Toshimitsu Uesaka, Dongjun Kim, Wei- Hsiang Liao, Yuki Mitsufuji, J Zico Kolter, Ruslan Salakhutdinov, and Stefano Ermon. Manifold preserving guided diffusion. In The Twelfth International Conference on Learning Representa- tions, 2024. URL htttps://ppeneeeiee wwwpeeiiwwweweeeeeeeeeeee bett//orommlddooo00l....
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NewrIPS 2021 Workshop on Deep Generative Models and Downstream Application,, 2021. URLitttps::/oppereviai net/ www.eooriiiie........... www.uunnnnddaaaAAaxeeee
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 0866400-8511 2020.
Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling, Argmax flows and multinomial diffusion: Learning categorical distributions. Advances in neural information processing systems, 34:1 14:44444-14 12465, 2021.
Marcel Kollovich, Abdul Fatir Ansari, Michael Bohikc-Schneider, Jasper Zschiegner, Hao Wang, and Yuyang Bernie Wang. Predict, refine, synthesize: Self-guiding diffusion models for proba- bilistic time series forecasting. Advances Neural Information Processing Systems, 36:28341- 28364, 2023.
Akim Kotelnikov, Dmitry Baranchuk, Ivan Rubachev, and Artem Babenko. Tabddpm: Modelling tabular data with diffusion models. In International conference on machine learning, pp. 17564- 17579. PMLR, 2023.
Trent Kyono, Yao Zhang, Alexis Bellot, and Mihaela van der Schaar. Miracle: aausally-aware imputation via learning missing data mechanisms. Advances in Neural Information Processing Systems, 34:23806-222 228177, 2021.
Tongyu Liu, Ju Fan, Nan Tang, Guoliang Li, and Xiaoyong Du Controllable tabular data synthesis using diffusion models. Proceedings of the ACM on Management of Data, 2(1):1-29, 2024.
Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11461-11471, 2022.
Boris Muzellec, Julie Josse, Claire Boyer, and Marco Cuturi. Missing data imputation using optimal transport. In International Conference Machine Learning, pp. 7130-7140. PMLR, 2020.
12

---

Published as a conference paper at ICLR 2026
Sai Shankar Narasimhan, Shubhankar Agarwal, Oguzhan Akein, Sujay Sanghavi, and Sandeep P. Chinchali. Time weaver: A conditional time series generation model. In Forty- first Interna- tional Conference Machine Learning. ICML 2024, Vienna, Austria, July 21-27, 2024. Open- Review.net, 2024.
net ner/forum? d=woxDe xmFr
Yidong Ouyang, Liyan Xie, Chongxuan Li, and Guang Cheng. Missdiff: Training diffusion models on tabular data with missing values. arXiv preprint arXiv:2307.00466, 2023.
Phillip Pope, Chen Zhu, Ahmed Abdelkader, Micah Goldblum, and Tom Goldstein. The intrinsic dimension of images and its impact on learning. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL hattps://ppenrevile net/ forum? 1txrummm22dx
Aditya Shankar, Hans Brouwer, Rihan Hai, and Lydia Chen. Silofuse: Cross-silo synthetic data generation with latent tabular diffusion models. In 2024 IEEE 40th International Conference on Data Engineering (ICDE), pp. 110-123, 2024.
doi: 10.1 1..1109//DE001 00W.E60001446224.0000000000
Juntong Shi, Minkai Xu, Harper Hua, Hengrui Zhang, Stefano Ermon, and Jure Leskovec. Tabdiff: mixed-type diffusion model for tabular data generation. In The Thirteenth International Confer- ence on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025. httt:t:///ppeeeeeeee
Jiaming Song, Qinsheng Zhang, Hongxu Yin, Morteza Mardani, Ming- Yu Liu, Jan Kautz, Yongxin Chen, and Arash Vahdat. Loss-guided diffusion models for plug-and-play controllable generation. In International Conference on Machine Learning, pp. 32483-32498, PMLR, 2023.
Yang Song. Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL https::/
net forum?ice
EXTIG12RREE
Jan Pawel Stanczuk, Georgios Batzolis, Teo Deveney, and Carola-Bibiane Schönlieb. Diffusion models encode the intrinsic dimension of data manifolds. In International Conference on Machine Learning, pp. 46412-46440. PMLR, 2024.
Mihaela C. Stoian, Salijona Dyrmishi, Maxime Cordy, Thomas Lukasiewicz. and Elconora Giunchiglia. How realistic is your synthetic data? constraining deep generative models for tabu- lar data. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vi- enna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL URLphtttps///eeeeeeeee
net/
Fooumn?dd-
Mario Villaizán-Vallelado, Matteo Salvatori, Carlos Segura, and Ioannis Arapakis. Diffusion mod- els for tabular data imputation and synthetic data generation. ACM Transactions on Knowledge Discovery from Duta, 19(6):133,, 2025.
Lei Xu, Maria Skoularidou, Alfredo Cuesta Infante, and Kalyan Veeramachaneni. Modeling tabu- lar data using conditional gan. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, pp. 3345-7345, 2019.
Jinsung Yoon, James Jordon, and Mihacla Schaar. Gain: Missing data imputation using generative adversarial nets. In International conference on machine learning, pp. 5689-5698. PMLR, 2018.
Jinsung Yoon, Lydia N. Drumright, and Mihaela van der Schaar. Anonymization through data syn- thesis using generative adversarial networks (ads-gan). IEEE Journal of Biomedical and Health Informatics, 24(8))2378-2388. 2020. doi: 10. 109//BBH..202022988222
Jiwen Yu, Yinhuai Wang, Chen Zhao, Bernard Ghanem, and Jian Zhang. Freedom: Training-fre free energy-guided conditional diffusion model. In Proceedings of the IEEE/CVF International Con- ference Computer Vision, pp. 23174-23184, 2023.
13

---

Published as a conference paper at ICLR 2026
Hengrui Zhang, Jiani Zhang, Zhengyuan Shen, Balasubramaniam Srinivasan, Xiao Qin, Christos Faloutsos, Huzefa Rangwala, and George Karypis. Mixed-type tabular data synthesis with score- based diffusion in latent space. In The Twelfth International Conference Learning Represen- tations, 2024, htt.ttts::///
net/ forum??d-=4xy233ee
Hengrui Zhang, Liancheng Fang, Qitian Wu, and Philip S. Yu. Diffiputer An EM-driven diffusion model for missing data imputation. The Thirteenth International Conference on Learning Representations, 2025. URL / wttp:://wppe.eee.eee/ooo net/ http:///wweetttorruuooooCCSSSSS-
He Zhao, Ke Sun, Amir Dezfouli, and Edwin V Bonilla. Transformed distribution matching for missing value imputation. International Conference on Machine Learning, pp. 42159-42188 PMLR, 2023.
Shuhan Zheng and Nontawat Charoenphakdee. Diffusion models for missing value imputation in tabular data. CoRR, Abs/2210.17128., 2022. doi: 1048500ARX11:210.1122 URL https: 7/doi. 4ttp//10.45555/axxxxxx 2210. 17128
14

---

T (2,) Published as a conference paper at ICLR 2026
A
NOTATIONS
Table 7: Summary of key notations.
Symbol
Explanation
(A) Data & Manifold Setup
Ambient space dimension
Zo E Rdd p(xo)
Clean (un-noised) sample
Clean data distribution
Mo C Rd
Smooth, connected data manifold
n
Intrinsic dimension of Mo with n <
:: R_ + Rd
P:R
Smooth manifold embedding function (Assump. !
T Mo
Tangent space of My at To
(To) {x: | 1u--xpl xolla <r}
Local hypersphere radius r about zo (Assump.
(B) Forward Diffusion & Noisy Shells
E .T
Diffusion step index; T total steps
Qz
Noise schedule at step t
a
II as
Cumulative noise schedule
Noisy sample at diffusion step t Sampled white noise Noisy shell manifold at step t (Prop. I
Mr
(C) Denoiser & Training Objective
co (2t,)) Ltrain
Denoiser's noise estimate for sample at step MSE training loss eq. 2
(D) Projection & Dirty Estimate Qx(-)
Dirty estimate mapping function (Def. T Dirty estimate point on Mo Orthogonal projection of x, onto My (Thm. 3.1
c0 = Q+(.)
(E) Guidance & Tangency
E C
Arbitrary conditioning information
(int(-)
Arbitrary differentiable inference-time loss
(F) HARPOON Sampling (Alg.
Sampled noise for stochastic denoising Noise scale for denoising step Guidance step size (strength)
4/-1
Unconditional one-step denoised estimate
B PROOFS
Theorem 3.1 (Limiting behaviour of dirty estimates) Let en be denoiser trained to predict Gaus- sian noise under the framework of eqs. [Tndn] 2 for a clean sample zo. Then, for the mapping Qi defined in Definition I we have lima.,+1 Qe(xs) = *(x)), where x(x) denotes the orthogonal projection of 24 onto the data manifold Mo.
Proof. Consider the training objective for a denoiser with parameters 0 under continuous Gaussian noising framework equation
Cissn -eo (1) -e
Substituting from equation
1
we get:
15

---

Published as a conference paper at ICLR 2026
cp((xst) - Vā
Liaain = Luuide
v1 di
Ltaain -La co(os))))
pl prtpeeo.DDDpooopo(ddddddddddddd
vI at
For fixed #a and 6, ea( 1) optimized by minimizing the loss, i.e. setting V Strain O:
() = S. 2 (co(2221)) 2t - (a
plæclzo, t)p(zo)dro 0
v1 ai
c0( 1) I plaskkro ()p((r))d2g)
/
varto
plar ()p(ra)dro
vI ai
Dividing both sides by Sag p(ze|zo: t)p(zo) we get:
2t Vār Sa nop(rr/ (opelroo))((t (p(
vI az VI
Sm p( (ro, () dooo
Applying Bayes theorem, we can rewrite this as follows:
It
VarF
where F(x,, () Se, sop(rolor Dlferddoo Sao p(zol 2e, t) t)p(
c0(22, 1) vI -
(4)
VI - ai
We partition F t) into contributions from points Zo in region E MMs chosen inside
ball radius E E (0, [x, (xx))l2) around the orthogonal projection (.))), and its complement Mo \ E. (Stanczuk al. 2024). Under the Gaussian noising framework, the conditional distribution
satisfies p( T4, 1) ~ (xp 24, T)
JB To Nanlat of dæo
+ SMOYE To-N (2ol of da SMO N ( of1) dæ
t(
JM, (1 ofl) dxp We then divide both the numerator and denominator by SE N (mollre, of dzo:
L no (N 2 dey
SMayE No:N(aola)..o date
Te N (ora (e o2 dayg
Ig (zulee, dat
F(x,,t)
fo. ( (o daa
[maye Ni JR oz
N N in (f day
Applying Lemma D.9 (Stanczuk et al. 2024), for a general Gaussian distribution, the contributions from Mg E become negligibly small as ar This simplifies the expression to:
lim F(x,,)) SE vo -N Ir, of dro
JB N 1) dæo
Subtracting TIx) from both sides, we get:
lim F(r,,t) ((x))= f8 ((x))) -V (Zola, of dro o1
Nooo(z of 1) drg
Since ||ri (xx))) S s for all zo E E, we can bound the difference:
< (l),)) -(x)) S SE NE of A It, of1) dxo
As is user-defined parameter that be set arbitrarily small when constructing F,, get:
16

---

Published as a conference paper at ICLR 2026
lim F(z,, t) = T(1)). o-
Upon substituting this result for F(r,; 8) in equation
we get:
Jt
v1 a
lim eo c t) 0.-0
r (2t)
v1
ai
a
Since of -ū,, we have:
lim Qx(xs)
Zt
VI
aieo(T t)
T (2))
Vāx
Vār
Hence (x())) points the orthogonal projection on the data manifold Mao
Key difference from Chung et al. (2022): While our result is similar to Proposition 2 from Chung al. (2022). the key difference lies how we treat the integral in equation Chung et al. (2022) assume this integral behaves as a weighted average over all points on the manifold, relying on the symmetry of the normal distribution to argue that the denoised estimate points to the orthogonal projection. However, this assumption requires the manifold to be globally linear, not just locally linear, since the integral spans the entire manifold. In contrast, our proof makes no such assumption on the manifold's geometry, generalizing the result to non-linear manifolds.
Theorem 3.2 (Manifold-constrained inference-time gradients) Let с denote arbitrary condition- orthogonal projection onto the
ing information and assume Qs from Definition 7 acts data manifold Mo at ãn = Qe(x)). Then, for any differentiable inference-tine loss function Lint Rdd x C + R. defined by (fn the gradient lies in the tangent space of Ma at Zn. Le., VR, Lint (in c) E Mo-
Proof. Let Rdd be an arbitrary vector. We decompose the tangential component and UN 1 (e. is the normal component.
UN, where or TTo(oe))Moo
By Assumption 2 the local behaviour of Q, satisfies the following:
lim Q+(x, + sv) lim (Q(x)) + SUT) s-0
s-0
Taking the partial derivative with respect to and evaluating at gives the following:
Qx + sw)
(Qx(a+) + SCT)
Ös
-=0 Ōs
==0
The left-hand side Jq.0 v (where Jq. is the Jacobian), so the expression evaluates to the following:
JQ. = UT.
(5)
Now, apply the chain rule for differentiable loss Lint(fa, c):
V Lint c) @Line(To, c) (Cin:()
Or.
Or.
dLint
V Cint c) -ō.
J0. Dão
0Linl
Va, ar (2o, c) JQ.W (since dān
some w E Rdd)
(6)
From equation we have:
TT JT vt
J JT w U
T
JJ JTJ vfE
U Jo w + JJ 10 =TTTT= + vf
+
The term of wN since the vectors perpendicular one another, yielding:
J. w + w= ofwr Upon matching terms, since uN and Vr are perpendicular one another, we have the following:
J/.== WT
Substituting this in equation 6 we get the final result:
Lint( wT E Mo
17

---

Published as a conference paper at ICLR 2026
C DATASETS
We used eight widely adopted benchmark datasets: Adult Default of Credit Card Clients
Magic
Gamma Telescope" Online Shoppers Purchasing Intention Gesture Phase Segmentation Letter Recognition and Dry Bear Additionally, we included the California Housing dataset from Kaggle" The statistics of these datasets presented in Table each of which is partitioned using a random 70-30 train-test split. Prior to splitting the datasets, we filtered out the rows having missing values, similar to Zhang et al. (2025). We also remove the target column from training and evaluation to avoid leaking label-feature correlations, following Zhang et al. (2025).
Table Statistics datasets. Num stands for the number of numerical columns, and Cat stands for the number of categorical columns.
Dataset
# Rows
# Num
Cat
f Train (In-sample)
Test (Out-of-Sample)
California Housing
20,640
14,447
6,193
Letter Recognition
20,000
16
14,000
6,000
Gesture Phase Segmentation
9,522
32
6,665
2,857
Magic Gamma Telescope
19,020
10
13,314
5,706
Dry Bean
13,610
17
9,527
4,083
Adult Income
32,561
6
22,792
9,769
Default of Credit Card Clients
30,000
14
10
21,000
9,000
Online Shoppers Purchase
12,330
10
7
8,631
3,699
D TABULAR CONDITIONS: IMPUTATION AND INEQUALITY CONSTRAINTS
While the manifold assumptions describe the geometry of noisy data, ultimate goal is to do conditional tabular generation. Such conditions arise several forms:
Imputation under masks. A binary mask
{0, 1 specifies observed (m 0) and miss- ing/masked (m = 1) entries for a ground-truth sample zo E MMo The conditional target given by (1 m) Ho, i.e., the subset of ground truth values reconstruct. We can measure the likelihood of denoised estimate (0n) adhering to the conditions through a reconstruction loss such as
mmp ( To m) = [[(1 m) @ (øo mo)llp
for E {1,2}, corresponding to the Mean Absolute Error (MAE) or MSE, respectively.
Inequality constraints. Beyond imputation tasks, conditions can also impose inequality constraints:
min f (āo) s.t. S g(0o) S u. h(to) U.
ZoERd
where C, u are elementwise lower and upper bounds (e.g., Age E [0, 120). Salary 2 0), g() allows transformations subset selection. The function h(-) could be selector that enforces features to match a particular discrete or continuous value v. Feasibility can be approximated using soft write in more general form as follows:
penalties as proposed by (Donti et al. 2021), which
Csott (20) f(0) (
||ReLU(g(0)) - w)]bs))
A (((()) 2||k,
where i,j. are suitably chosen norms (e.g., 1, 2, or oo). While useful in principle, the norm to incorporate these constraints directly training- time objective for model (Donti et al.
ht http:///eraicccaaaaa///////////t
https:/ http:////hht...tea.////////////a
c1 ciients22pp
žip
Segmentaticn.17
18

---

Published as a conference paper at ICLR 2026
Lt
Figure 4: Disconnected submanifolds under disjunctive constraints, e.g. (colourzred OR colour=blue). Point Ta likely favours the region indicated by red arrow due to the proximity and larger feasible area imposed by the constraint.
2021]. This practice presents two limitations: (i) some constraints may correspond to rare events or infrequent classes, making it difficult to get enough samples for reliable training, and (ii) constraints may change at inference (e.g., Age > 30 replaced by Age < 15), necessitating retraining.
We that different conditional tasks naturally correspond to different optimisation objectives at inference time, motivating the central question our work.
Conjunctions and disjunctions of conditions. Equation naturally handles conjunctions (AND) of multiple constraints since the overall loss drops to zero when all individual constraints are satis- fied. To handle disjunctions (OR), modify the loss by taking the product of individual constraint violations. For instance, for disjunction such as to >> a or do < b, we can formulate it as
Laig (o)) [|ReLU((a fo)l ||eeLU bb]l
(10)
which drops to zero if either condition is satisfied.
Interestingly, disjunctive constraints correspond disconnected sub-manifolds of the feasible region. As Figure 4 illlustraess the model may end up favouring one condition over the other de- pending on which reasible region covers larger portion of the data manifold, or which region the noisy point 2t is closer to during denoising.
E EXPERIMENTAL CONFIGURATION OF TABULAR CONDITIONAL TASKS
We describe the setup of the conditional tasks used in experiments. We first detail the imputation tasks under different missingness mechanisms, followed by the design of incquality constraints.
E.1 IMPUTATION TASKS
For imputation, we simulate missingness patterns under three standard assumptions:
MCAR (Missing Completely at Random). Entries removed uniformly random, indepen- dent the observed or unobserved values. We implement this by masking random subset of entries at given missingness ratio, following Zhang al. (2025).
19

---

Published as a conference paper at ICLR 2026
MAR (Missing at Random). Entries removed conditionally observed variables. This sures missingness depends on observed data but not on the missing entries themselves. We imple- ment MAR scenario following the approach in Zhang al. (2023); Zhao et al. (202)). Specifically, we keep aside a number of columns as the fixed input features and train a logistic model on these features generate the missing values for the rest. We follow line of search to tune the bias term to get the desired degree of missingness.
MNAR (Missing Not at Random). Entries are removed in way that depends their (un- observed) values. Following Zhang et al. (2025), we implement this by separating the input features into two groups and generating the missing values for the second group using the logistic model, similar to the MAR tasks. We then mask out values from the first group following the MCAR setting, creating tasks that are more challenging than both MAR and MCAR.
E.2 INEQUALITY CONSTRAINTS
In addition to imputation, we evaluate conditional generation under inequality constraints. We con- sider three types of conditions: range, categorical, and their combination.
Range Constraints. We first select a continuous feature at random and restrict it to a region in the tail of its empirical distribution, i.e., region with relatively few samples in the training data. This design ensures that the constraint is non-trivial and represents a realistic stress test for the model. For evaluation, the test set. We then generate an equal number of samples from the model under the specified constraint. Performance assessed by measuring (i) the fraction of generated rows that violate the range constraint, (ii) downstream utility using an XGBoost classifier, and (iii) the a-score (Alaa al. 2022) of the generated samples, quantifying their fidelity.
construct "ground truth" reference
by applying the same filter on
Categorical Constraints. For categorical constraints, we fix the value of a randomly chosen dis- crete feature a particular class, deliberately avoiding the majority class from the training data. As with range constraints, we construct a ground truth by filtering the test set for rows that satisfy the categorical restriction, and we generate an equal number of samples under this condition. Evaluation follows the same protocol: violation rate and alpha-precision relative to the ground truth.
Conjunetive Constraints. We also consider conjunctions, where we consider tasks where both range and categorical constraints are applied jointly. In this case, any generated row that violates at least one of the specified constraints counted as violation. Ground truth samples are constructed by filtering the test set with both conditions applied simultaneously.
Disjunetive Constraints. Finally, we evaluate disjunctive (OR) constraints, where a generated row only needs to satisfy at least one of multiple specified conditions. For example, constraint may require that either continuous feature A exceeds a threshold a, or a categorical feature takes particular value b. Ground truth samples are constructed by filtering the test set for rows that satisfy any of the specified conditions. Such constraints can result in disconnected submanifolds, as illustrated in Figure 4, since each constraint likely spans a different feasible region the data manifold.
The specific values and classes chosen for constraints, along with the proportion of training samples covering them, are provided in Table demonstrating that these setups constitute a challenging stress cases rather than trivial ones.
F EVALUATION METRICS
In addition to standard imputation metrics such as MSE and accuracy, we assess the quality of gen- erated tabular data using combination of fidelity, privacy, and utility metrics. We implement all our metrics using the code available at SynthCity: https:: /ttp:://///htbbb/// wcw./vanderschaarl.... synthcity
Fidelity quantifies how well the generated samples match the distribution of the real data. We use the following:
20

---

Published as a conference paper at ICLR 2026
Table 9: Constraint specifications for range, categorical, conjunctive, and disjunctive tasks across datasets, together with the proportion of samples seen during training satisfying each constraint.
Dataset
Constraint Type
Feature(s)
Restriction
Train Proportion (%)
Range
Age
50
21.73
Categorical
Workelass
State-gov
3,9
Adult
Conjunctive
Age, Workclass
50; State-gov
0.87
Disjunctive
Age, Workclass
2 50 or State-gov
24.83
Range
AGE
2 50
8.9
Categorical
EDUCATION
High School
16.37
Default
Conjunctive
AGE, EDUCATION
Wwy 50 or High School
50; High School
3,4
Disjunctive
AGE, EDUCATION
21.88
Range
Administrative
25.36
Categorical
VisitorType
New Visitor
14.25
Shoppers
Conjunctive
Administrative, VisitorType
4: New Visitor
New_Visitor
4.19
Disjunctive
Administrative, VisitorType
4 or New Visitor
35.42
a-score (Alaa et al. 2022); The core idea is that real data naturally concentrates in "typical" high-density region, with the remaining a fraction treated as outliers. The method therefore constructs the minimum- volume o-support of the real distribution, i.e., the smallest region that contains the most typical fraction a of real samples. The score then measures the proportion of generated samples that fall inside this typical region. In our experiments, higher a-scores indicate better alignment of generated samples with the high-density core of the real data, reflecting strong sample fidelity.
KS score Saalmogoro--Smimon test): The Kolmogorov-Smimmt (KS) score measures the maximum difference between the cumulative distribution functions (CDFs) of real and synthetic samples. Intuitively, it quantifies how closely the synthetic data reproduces the marginal distribution of each feature. In the SynthCity implementation, higher KS scores indicate stronger similarity between the real and synthetic distributions, while lower scores indicate substantial divergence.
The detection score measures how easily classifier can distinguish real data from synthetic data. We train linear classifier the pooled dataset containing both real and synthetic samples, using K-fold cross-validation to estimate robustness. The met- ric reports the average AUC-ROC obtained when predicting whether sample real or synthetic. Lower scores (close to zero) indicate that the sets are indistinguishable.
Detection score:
Privacy metrics assess the risk of identifying real data points from the synthetic dataset. We use the Identifiability metric, which estimates how likely is that synthetic sample is "too similar" to real one, indicating potential privacy leakage. Intuitively, the metric checks whether any synthetic record lies closer real sample than the distance between real samples with each other. so, the synthetic sample may inadvertently reveal an individual. Higher scores indicate a greater risk of recovering real data (Yoon al. 2020).
Utility quantifies how useful the generated data is for downstream tasks. We train an XGBoost clas- sifier on the generated dataset and evaluate accuracy on the real test set. This reflects whether the synthetic data preserves predictive relationships present in the original dataset. In our experiments, we predict the gender, the marital status, and the operating system as the targets, for Adult, Default, and Shoppers respectively.
G BASELINES AND IMPLEMENTATION DETAILS
G.1 NON-DIFFUSION BASELINES
We implement most of the baseline methods according to the publicly available codebases.
Remasker (Du et al.
2024): https: / http//gittttas thub. com http://yduskyyyy wew.kk//remakkeeee The framework
automatically masks traction of values during training (50%), which leave unchanged.
21

---

Published as a conference paper at ICLR 2026
MIRACLE (yono al. 2021) and GAIN (Yoon al. 2018
https:: /ttp/bgittbbbbbbb com/ wanderschaar ab/ http:aabbhhhhpraee imppre For GAIN, we updated the plugin code to separate pre- training from inference- timeencc-ttime time masking. Following the paper (Yonn et al. 2018), we pre-train the model using 20% missing data under an MCAR mask, and evaluate at inference on various masks.
2023): https::/pypt wttppi.orgppooeeeeee....pppppppp /bctt/beeeee /beee/eeeeee watt///ggeee////// Since the model sometimes fails generate all missing one go, we give up five retries to fill table. if still fails to generate some values, we replace them with the mean for numerical columns. Categorical columns were fixed using random replacement.
GReaT (Borisov
whole
All the above baselines (except for GReaT)) require tabular data consisting of purely numerical in- puts, so we use integer encoding represent discrete features as numbers. The train and splits are then standardized using the mean and standard deviation of the training set. For GReaT, no special encoding is required as the tokenizer automatically creates a suitable representation for the values. For evaluation, MSE scores are reported on the standardized data (for numerical columns) for consistency across all baselines and datasets.
G.2 DIFFUSION BACKBONES AND HYPERPARAMETERS
For all diffusion-based baselines (HARPOON, DiffPuter (Zhang et al. 2025), we adopt shared MLP architecture as the backbone for consistency. The network consists of linear layers with swish activations. For the timestep embeddings, positional embedding. We use one-hot encoding for the categorical features and we standardize all numerical features similar the other baselines in Sec. G.1 Evaluation is also performed in similar manner for consistency across basclines.
2-layer MLP with 1024 channels for the
The diffusion process is configured with 0j = 0.9999 and aT 0.98, consistent with prior work (Ho et al. 2020).. We use 7' 200 diffusion steps for both noising and denoising using batch size 1024 trained for 1000 epochs. We use learning rate of 10 and guidance step size 0.2 (HARPOOR only) for all datasets and task configurations. When sampling using HARPOON, we use the MAE loss for imputation tasks. For general inequality constraints (equation 8). we use
coefficients Xg. Å 1, the 2-norm for i, and the 1-norm for k.
H TRAINING AND SAMPLING ALGORITHMS
H.1 TRAINING
we adopt the standard diffusion objective. At each iteration, we draw clean sample zo from the data distribution p(ro)) uniformly sampled timestep / ~ U(1, [1,,7], and Gaussian noise ~ (N, I). We forward noise the sample using equation and then predict the injected noise using the denoiser сş. Parameters updated by minimizing the mean squared error with the MSE loss, following the standard training protocol, as proposed in Ho et al. 2020).
During training (see Algorithm 2,
Algorithm Training of Unconditional Diffusion Model
1: repeat
2: Sample data point xi ~ p(zn)
3: Sample timestep ~ Uniform (1...., T})
4: Sample noise ~~ 10,
5: Predict noise using backbone (arzo + /I āe t,
6:
Update parameters with gradient descent on
Vell= - (
7: until convergence
H.2 SAMPLING WITH HARPOON
At inference time, HARPOON augments the standard denoising procedure with manifold-aware ditional guidance (see Algorithm l). We initialize the sample with Gaussian noise 27 ~ (N I). At denoising each step, we predict an estimate do of the clean sample. A task-specific conditional
22

---

Published as a conference paper at ICLR 2026
loss C( c) is then evaluated, and its gradient with respect to 2; is computed. This gradient serves as an approximate tangential direction along the data manifold M,. Since the noisy manifolds at successive timesteps approximately parallel, the gradient computed at timestep t provides good local approximation of the tangent space at & - 1. H HARPOON therefore reuses these gradients across steps to maintain manifold alignment. Concretely, after performing the standard denoising update to obtain a provisional sample -/- HARPOON applies an additional tangential correction:
Tt-1 = _ 1) Va, ((xr, c);
where is a step size hyperparameter. This ensures that updates both (i) respect the manifold structure learned during training and (ii) steer the sample toward satisfying the imposed condition.
ADDITIONAL EXPERIMENTS
1.1 IMPUTATION RESULTS UNDER MCAR AND MNAR MASKS
Beyond the main results with MAR masks (Sec. 4.2 we further evaluate Harpoon and baseline methods under MCAR and MNAR missingness Tables 12 and 13 We notice that the overall trends remain consistent with the MAR setting. In terms of MSE and accuracy, DiffPuter and HARPOON generally perform the best, with HARPOON generally being the better among the two. While methods such as Remasker and Miracle are also strong baselines, we see that their bias towards numeric-type data limits their accuracy when imputing discrete features (see Tables 11 13
Table 10: Imputation MSE for MCAR mask. Standard deviation in subscript.
Ratio
Method
Adult
Bean
California
Default
Gesture
Letter
Magic
Shoppers
GAIN
1.770.04
1.27a.o2
13.75
5.67p.03
1.16g.0 02
1.03p.01
1.46p.02
1.82n.03
Miracle
1.120.04
0.08
0.38p.o2
0.83p.11
0.86 os
0.45p.03
0.510.03
1.270.10
GReaT
5.124.38
2.731.85
1.670.04
1.75p
>10
1.746.01
1.60p.0
1.630.07
0.25
Remasker
0.940.05
0.260.09
1.01o.1
0.51 03
0.67o.or
0.65o.07
1.70, .so
0.86g.
DiffPuter
1.05o.05
0.160.01
0.490.02
0.53 03
0.34o. 01
0.396.01
0.67p.02
0.80o.05
Harpoon
1.01p.06
0.15..o1
0.44g.0z
0.45
0.47a.01
0.47p.01
0.79p.o2
0.74
GAIN
3.30p.02
1.790.01
17.30p.13
19.16p.07
3,06g .2
1.11
1.82p.00
4.640.03
Miracle
4.00b.58
0.16u.02
0.83g.a5
1.37p.us
1. 1.21
0.89p.04
1.63u.25
2.280.20
GReaT
2.50
4.093.74
1.55
2.30
>10*
1.550.02
1.40p.02
1.580.34
0.50
Remasker
1.06..02
1.02a.19
1.40u.02
0.86p.us
1.91
1.120.05
4.37p.02
1.230.04
DiftPuter
1.22p.03
0.27
0.77g.02
0.64
0.54g.
0.70,.02
0.81
0.930.03
Harpoon
1.120.02
0.18
0.70g.01
0.53p.02
0.70 02
0.830.02
0.87p.m
0.870.02
GAIN
6.630.01
2.83u.01
21
21.660.08
58.09u.22
9.59g.u6
1.299.00
2.360.01
13.07
Miracle
1.470.14
0.53u.o2
0.89
>10*
1.23
0.86
0.96..05
1.13g.0n
GReaT
1.790.58
4.21
1.50o.o1
2.342.0
>10
1.50p.01
1.27p.0
1.330.00
0.75
Remasker
1.070.01
1.3
1.400.00
1.01
6.66g 01
1.110.00
4.38o.02
1.130.01
DiftPuter
1.47
0.63
1.190.01
0.90p.uo
0.82 02
1.18p.01
1.12p.0
1.200.03
Harpoon
1.28p.m
0.39
1.1
0.72p.
0.97a.02
1.276.02
1.09p.m
1.06
I.2 ABLATION: EFFECT OF ENCODING
In Tables 14 and 15 we compare different encoding schemes on imputation performance, measured by MSE (For continuous features) and accuracy (for discrete features). Specifically, we consider one-hot encoding versus integer encoding. where the latter assigns a unique integer to each category of a feature. Our results show that one-hot encoding consistently outperforms integer encoding, both in terms of MSE and discrete imputation accuracy. We attribute this to the fact that integer encoding implicitly imposes artificial ordering among categories, which may bias the model towards predicting categories with numerically closer values, even they are semantically unrelated. In contrast, one-hot encoding avoids this issue by representing categories in an unordered manner.
I.3 CHOICE OF PREPROCESSING
Table 16 shows the imputation performance under MAR missingness for three datasets when us- ing either standard scaling (Z-normalization) quantile standardization to preprocess continuous
23

---

Published as a conference paper at ICLR 2026
Table 11: Imputation Accuracy for MCAR mask. Standard deviation in subscript.
Ratio
Method
Adult
Default
Shoppers
GAIN
20.77u.21
47.770.43
45,430.43
Miracle
43.48p.89
65.232.07
40.220.47
GReaT 15.90p.77
41.740.27
40.51..46
0.25
Remasker 41.36p.57
67.03o.71
42. 190.34
DiffPuter
66.470.20
69.91o.26
59.110.37
Harpoon
66.540.19
72.220.30
54.670.55
GAIN
19.94..15
41.140.12
32.450.20
Miracle
30.241.19
56.03
32.50
GReaT 31.08u.s7
49.996.44
39.790.35
0.50
Remasker
33.283.21
51.232.31
43.761.00
DiffPuter
60.59u.21
61.71o.46
55,900.23
Harpoon
61.950.21
68.25
52.260.31
GAIN
21.220.04
19.99
19.270.13
Miracle
33.680.74
49.0210.05
38.020.56
GRea'T' 42.620.25
54.910.33
38. 17..3
0.75
Remasker
29.170.04
44.450.14
46.590.09
DiftPuter
53.82n.17
49.640.34
51.770.40
Harpoon
55.320.06
60.55o.27
49.650.36
Table 12: Imputation MSE for MNAR mask. Standard deviation subscript.
Ratio
Method
adult
bean
california
default
gesture
letter
magic
shoppers
GAIN
1.86
1.350.0s
13.92o.48
5.79
1.30p.o 06
1.03p.03
1.48p.10
1.770.13
Miracle
1.33
0.27o.15
0.55g
0.95p.15
1.04p.00
0.54p.05
0.65
1.600.30
GReaT
2.83
1.96n.15
1.76g.11
1.990.20
>10
1.75p.03
1.670.0s
5.267.18
0.25
Remasker
0.96g.09
0.270.09
1.18g.29
0.61g.11
0.77
0.690.11
1.09
0.90p.13
DillPuter
1.05
0.190.o8
0.51
0.64p.09
0.46
0.39
0.73
0.84u.07
Harpoon
1.02p.07
0.15u.ua
0.46u.04
0,51g.10
0.60p.11
0.490.02
0,88p.15
0.780.08
GAIN
3.32g.08
1.76p.05
17.400.34
18.82a.
3.08p.08
1.09
1.80p.07
4.600.18
Miracle
3.96u.52
0.39o.13
0.92a.17
1.30g.17
1.23p.00
0.96n.06
1.470.15
2.370.23
GRea'T
2.931.77
2.52p.75
2.00g.35
1.73
>10
1.55
1.40g.06
1.410.06
0.50
Remasker
1.04g.07
1.12p.22
1.42g.09
0.790.08
2. 2.140.30
1.140.04
4. 4.35g.15
1.180.03
DillPuter
1.22o.07
0.30p
0.76a 06
0.66p.0 .77
0.570.06
0.690.02
0.816.06
0.910.04
Harpoon
1.12
0.19b.uz
0.71
0.54p.07
0.74g.u
0.82p.0z
0.85
0.83
GAIN
6.83g.
2.85o.07
21.700.24
58.61o.70
9.81
1.280.os
2.33g.03
13.42p.56
Miracle
1.71
0.80p.05
0.95g.00
0,94g.08
1.25p.00
1.00
1.05g.00
1.170.06
GReaT
1.48g.06
7.914.18
1.62a.28
1.890.31
>10
1.50p.02
I.25
1.30g.05
0.75
Remasker
1.05g.03
1.299.08
1.390.02
0,95p.06
6.63p.04
1.10v.01
4.33g.06
1.130.02
DifffPuter
1.470.06
0.61p
1.18
0.85 .8
0.79p.ns
1.170.01
1.099.06
1.220.04
Harpoon
1.290.05
0.38p.05
1.10p.m os
0.67g.05
0.91g.05
1.250.02
1.04g.0e
1.070.03
features Across all datasets and missingness ratios, we observe that standard scaling provides
10
consistently lower MSE on continuous features compared to the quantile counterpart, sometimes by more than order of magnitude. In contrast, categorical imputation accuracy remains largely unchanged, since categorical variables are not affected by feature feature scaling.
The large errors with quantile scaling are probably because of highly non-linear mappings that shape features to match a uniform (or normal) distribution, which can produce large jumps in the data space for small differences in the transformed space, especially for sparse heavy-tailed regions of the distribution. Hence, we see that for Adult, where continuous features are relatively well-behaved, quantile scaling only slightly worsens performance. However, in the Default and Shoppers datasets, both of which contain skewed or heavy-tailed numerical variables, the quantile mapping results in large MSE errors.
10We the default preprocessor from httre generated/ssleernn.co
24

---

Published as a conference paper at ICLR 2026
Table 13: Imputation Accuracy for MNAR mask. Standard deviation in subscript.
Ratio
Method
adult
default
shoppers
GAIN
20.620.29
49.722..1
44.560.94
Miracle
43.11 1.76
63.904.92
39.79h.71
GReaT
16.496.96
42.28n.62
39,43p.s0
0.25
Remasker
40. 1 2.94
66.651.47
42.01
DiffPuter
67.021.33
70.50o.66 66
58.85
Harpoon
66.861.20
72.580.64
53.940.71
GAIN
19.89..12
42.031.43
32.060.50
Miracle
31.11
54.32
33.151.47
GReaT
31.660.90
50.266.47
38.94 38.940.36
0.50
Remasker
35.41o.16
55.231.81
44.291..5
DiffPuter
60.570.84
62.100.65
55.610.90
Harpoon
61.920.85
68.480.45
51.85p.80
GAIN
20.94..11
20.22
19.130.56
Miracle
31.502.80
53.44
39.181.m
GRea'T 42.760.83
54.850.36
38.35p.38
0.75
Remasker
28.95..30
43.89
46.33
DiffPuter
53.330.03
49.290.44
51.560.61
Harpoon
54.930.79
60.40p.c4
49.53g.43
Table 14: Imputation MSE (MAR mask) under different encodings for HARPOON.
Ratio
Method One-Hot
Adult
Default
Shoppers
0.990.08
0.51p.06
0.73p.u6
0.25
Integer
1.61o.10
0.67
1.05o.14
One-Hot Integer
1.08p.06
0.53g.u6
0.81p.u6
0.50
1 1.58p.0s
0.68p.os
1.090.12
One-Hot 1.260.07
0.72
1.10p.05
0.75
Integer
1.54p.05
0.82
1.26p.01 04
1.4 RUNTIME COSTS
Table shows runtime costs of methods on all datasets. The observations are consistent with what was reported in the main text. We that HARPOON's still stays within 2x the runtime cost of DiffPuter due to the doubling of function calls to the denoiser backbone for gradient propagation.
1.5 RESULTS WITH ADDITIONAL METRICS
Table 18 reports the KS, detectability, and identifiability scores of synthetic datasets generated under different constraints: Range, Category, Conjunctions, and Disjunctions. Overall, KS scores remain consistently high across all methods and constraints, indicating that the synthetic data effectively preserves the marginal distributions of the original datasets.
Detectability, which measures how easily a classifier can distinguish synthetic from real data, varies with the type of constraint. Under stricter constraints, such as conjunctions and categorical, de- tectability is generally lower, indicating that samples are more indistinguishable from the real data. This is expected, as enforeing categorical or conjunctive constraints restricts the model to subset of discrete labels or consistent combinations, producing samples that more closely resemble real data. In contrast, Range constraints allow more flexibility, since values can vary continuously within plau- sible bounds, which increase detectability. For HARPOON in particular, detectability noticeably decreases under conjunctive and categorical constraints, suggesting that the model effectively guides samples toward regions satisfying both objectives.
Identifiability, which quantifies the risk of exposing real records, remains generally low across all methods. However, we interesting trade-off with the detectability. The more sample resem-
25

---

Published as a conference paper at ICLR 2026
Table 15: Imputation Accuracy (MAR mask) under different encodings for HARPOON.
Ratio
Method One-Hot
Adult
Default
Shoppers
69.443..1
73.492.52
51.175.20
0.25
Integer
65.04 65.043.22
73.022.48
45.124.48
One-Hot
64.80
69.55
48.395.00
0.50
Integer 59.822.71
68.861.62
43.16
One-Hot
38.813.85
61.483.83
48.37 1.39
0.75
Integer
55.212.54
62.762.73
45.453.50
Imputation MSE (continuous features) and Accuracy (categorical features) under MAR mask and various preprocessing scalers.
Table 16:
Adult
Default
Shoppers
Ratio
Method
Avg. MSE
Avg. Ace
Avg. MSE
Avg. Ace
Avg. MSE
Avg. Acc
Standard
0.990.08
69.443.11
0.51o.0s
73.492.5a
0.73g.06
51.175.20
0.25
Quantile
1. 1.03a.10
66.60a.34
12.192.52
74.882.50
5.79
50.865.40
Standard
1.08
64.80
0.53g.08
69.55
0.81
48.395.00
0.50
Quantile
1.14
63.253.36
13.693.47
70.782.08
5.561.55
48.305.66
Standard
1.260.07
58.813.6
0.72
61.4 61.483.85 83
1.10g.05
48.37.1.39
0.75
Quantile
1 1.430.16
57.814.24
20.459.37
61.924.16
4.78p.99
48.604.78
bles real data (lower detectability), the higher the potential risk of revealing private records (higher identifiability). Range constraints enforce numerical consistency, resulting low identifiability but moderate detectability due to skewed distributions of generated outputs. On the other hand, cate- gory constraints preserve categorical labels, lowering detectability while increasing identifiabiltty. Combining both constraints in the AND setting increases the identifiability of HARROON,, but lowers the detectability. The OR setting provides a balance, producing realistic data with moderate privacy protection.
Table 17: Runtimes (seconds) of methods across datasets (MAR mask 0.25 missing ratio).
Method
Adult
Shoppers
Default
Gesture
Magic
Bean
Califoria
Letter
GAIN
0. 0.14g.28
0. 0.140.28
0.15g.28
0. 0.14g.27
0. 0.140.27
0. 0.14g.27
0.15g.29
0.14g.27
Miracle
33.03
18.27
44.54p.ca
23.09p.13
19.99..11
19.86n.18
21.31
26.39
GReaT
65.472.57
51.132.29
202.717.44
131.72
19. 19.84..2
75.294.32
25.05
103. 680.68
Remasker
2.685.24
0.28p.51
0.33g.53
0.30p.54
0.30p.53
0.290.53
0.29p.52
0.30p.52
DiffPuter
2.83
1.190.25
2.620.28
0,98p.27
1.680.28
1.32
1.86g
1.79 25
Harpoon
4.430.34
1.890.38
4.05g.34
1.54p.35
2.65p.34
2.08o.31
2.88g.34
2.80p.53
26

---

Published as a conference paper at ICLR 2026
Table KS, detectability and identifiability scores.
Constr.
Method
Adult
Default Det. (.)
Shoppers
(t) %
Det. (1)
(L)
(1)
Iden. (1)
(t)
(1)
Iden. (1)
Dif'Puter
0.940.00
0.940.30
0.170.01
0,93g.00
0.94n.c1
0.150.0
0.91c.00
0.940.00
0.29
Range
GReaT
0.96,
0.950.00
0.20
0.92g.0
0.99u.00
0.08g.r
0.91g.00
0.99)
0.07
Harpoon
0.95p
0.900.00
0.266.00
0.94g.00
0.89p.00
0.3
0.92p.00
0.876.00
0.44p.0:
DiffPuter
0.95o.00
0.85o.01
0.41o.01
0.94g.00
0.75
0.44o.01
0.89,.00
0. 0.89
0.41o.02
Category
0.970.00
0.750.01
0.440.03
0,93g.00
0.79p.c1
0.29..01
0.93p.00
0.910.00
0.288.02
Harpoon
0.97,
0.730.01
0.46
0.95g.00
0.8
0.40g.0
0.90
0.866
0.47
Range
DiftPuter
0.946
0.910.01
0.30
0.93
0.95g.01
0.15u.0
0.88p.c
0.94p.a
0.27
AND
GReaT
0.960.00
0.92o.01
0.310.04
0.92g.00
0.98p.01
0.07a.02
0.92a.00
0.990.00
0.07p.ol
Category
Harpoon
0.966.00
0.64p.02
0.470.06
0.94g.00
0.82g.00
0.38g.04
0.89.00
0.82p.01
0.53g.0
Range
DiftPuter
0.94..00
0.92,.00
0.22p
0.94g.u
0.80p.00
0.37
0.90,.oo
0.936.00
0.30p.ot
GReaT
0.97
0.90g.00
0.24..00
0.93
0.84g.01
0.24g.01
0.92g.00
0.98p
0.14
Category
Harpoon
0.95
0.920.00
0.236.01
0.92g.00
0.94g.00
0.20
0.91p.c0
0.90p.00
0.37
27


Reviews:
[{"review_id": "JGTkm7EtfJ", "reviewer": "Reviewer_kUVE", "summary": "The paper proposes a new way to guide diffusion models for tabular generation at inference time. In particular, they show how at every time step $t$ most of the datapoints lie on a shell of dimension $d-1$ ($d$ being the original dimensionality of the datapoints) and that while the denoising process creates an orthogonal projection onto the shell of the datapoint $x_t$, the gradient of any loss function defined  starting from a condition $c$ is tangent to such shell. Exploiting these geometric results, they are able to create a new procedure for guiding the diffusion process at inference time, where essentially they interleave the diffusion process which (as they nicely put it) acts as a \"compass\" towards the manifold of the original datapoints and the update wrt the loss function that encodes the constraints which moves along the shell to guide the model towards the right region of the manifold.", "questions": "1. In order to impose the linear inequality constraints you had to devise an ad-hoc loss function. Do you see this as a possible bottleneck for the widespread application of your solution? \n\n2. Aside from categorical constraints, it is very reasonable to assume that tabular data have disconnected support. This makes me wonder how realistic is assumption 1 and also how important it is that assumption 1 is met in practice. Would it be possible to have an ablation study where a multiple datasets are created with each of them violating the assumption in different degrees and then studying how the method performs on them?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is very well written and provides a lot of intuitions about the results they propose. I really appreciated the visualisations of the gradients and the updates. \n\nThe experimental analysis is extensive and with very positive results. \n\nThe authors give a very nice geometric explanation of why their method works.\n\n**Note:** It is difficult for me to assess the novelty of this work wrt the previous works on diffusion models as I am not familiar with them.", "weaknesses": "1.  The authors do not report the sampling generation time. As this is an important metric for tabular data generation, it would be nice to have it\n\n2. In theorem 3.2 $\\mathcal{C}$ is not defined. Also it is not clear which conditions we have on $\\mathcal{C}$. Can it really be any arbitrary information? For example (Stoian & Giunchiglia, 2025) has extended the work cited in your paper to constraints expressed as disjunctions over linear inequalities. This defines non-convex and disconnected spaces hence violating your assumption 1. Would this time of conditioning be allowed? What about polynomials? \n\n3. A better definition of the alpha metric is needed. \n\n\n\nMinor things: \n\n1. Citation for Borisov et al. is missing the year \n2. Sometimes equation is written with capitol E and sometimes with lower case e \n\nReferences: \n\nStoian & Giunchiglia. Beyond the convexity assumption: Realistic tabular data generation under quantifier-free real linear constraints, ICLR, 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YUMwjR7gCI", "reviewer": "Reviewer_c4MN", "summary": "This paper introduces a method that can perform conditional tabular diffusion named HARPOON. It can generate new tabular data with specific conditions. In the theory contribution section, the authors show: 1. diffusion de-noiser acts as orthogonal projectors to data manifold 2. any differentiable inference-time loss has gradient in the tangent space of the manifold and therefore updates using this gradient can preserve realism. HARPOON is a method of combining unconditional de-noising step and the constraint-guided tangent gradient step. The experiments show that HARPOON outperforms other baseline methods.", "questions": "1. Does the tabular data generated from HARPOON improve the downstream model?\n2. What does the run time of HARPOON look like against other methods?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper is well written and provides both theoretical and experimental contributions\n2. The theory is novel and is the first to use diffusion model's orthogonal projection to manifolds in tabular setting\n3. HARPOON can handle mixed data types and has much lower constraint violation rate when doing conditional generation", "weaknesses": "1. Computation time might be an issue but this is not discussed in the experiments\n2. The utility, fidelity and privacy aspects of the generated tabular data is not discussed. It would be great to see where HARPOON stands among these 3 aspects.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3Jnewrrtpo", "reviewer": "Reviewer_1jxC", "summary": "The paper generalizes previous ideas from manifold theory to diffusion models for tabular data. This allows the usage of differentiable losses at inference time to guide samples along the data manifold at a given diffusion time $t$. Depending on the objective, this produces samples that satisfy constraints based on partially observed data and specified inequality constraints without the need for re-training the diffusion model.", "questions": "In addition to the suggestions and questions stated in Weaknesses, I kindly ask the following questions:\n\n- Could this manifold guidance lead to a skewed (conditional) data distribution?\n- Your guidance mechanism leads to *soft* constraints but not cannot impose *hard* constraints. This should be made clear in the text. Does that lead to issues in the imputation task? Since the non-missing values are not strictly fixed but still updated during sampling, could it lead to the final samples actually deviating from the partially observed information?\n- Does the guidance also work for latent diffusion models (TabSyn, CDTD, etc.) or is a data-space model necessarily needed?\nWhy is it a problem if models push samples orthogonally towards $\\mathcal{M}_0$ (line 104/105)? Even if it skips a shell, in the next update step, the model, which is conditioned on $t$, should have no problem continuing the path if trained properly. Is this an artifact from a model that is not capable / complex enough?\n- How does your guidance generalize to high dimensions, in particular when increasing the number of features or categories in a dataset? One-hot encoding is known to be very inefficient and will blow up the dimensions when applied to a categorical feature with 100s or 1000s of categories. \n- Can relative or logical constraints be accommodated? For instance, consider a data table of order and delivery dates. Naturally, we would expect that order date < delivery date.\n- Figure 2 b) and c) give a good intuition of moving a sample along a shell. Can there be situations in practice where this fails? For example, if at a particular level the manifolds are no longer connected, e.g., $\\mathcal{M}_t$ is not a single piece but two separate ones?\n- Are you using the QuantileTransformer to transform the continuous features for the TabDDPM model? From own experience this appears to be a crucial step for performance and is ubiquitous in tabular diffusion models.\n- In line 420: \"not just the constrained ones [...]\", do you actually mean the *un*constrained features? \n- Figure 3 is not well-designed. It does not does not include the red or green lines due to unfortunate overlaps. It would yield a more consistent argument if the x-axis would indicate $t \\in [0,1]$ and not the denoising step.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The authors generalize previous results, extending the usefulness of manifold-based insights to tabular data. In particular, they remove the necessity of squared losses, which extends the effective conditioning capabilities.\n- The ability to condition on certain features or constraints at inference time without the need for training the model on that specific conditional generation task is very valuable, in particular for tabular data.\n- The illustrations mostly paint an intuitive picture of the underlying mechanisms.\n- I like the effort of making the model more comparable to other diffusion-based approaches by adjusting the backend architecture. \n- The presented results show competitive or better performance in practice. Great improvements can be seen in scenarios that impose inequality constraints.", "weaknesses": "- The methodological background on diffusion is basically non-existent and the overview of the tabular diffusion models is severely underdeveloped. Since TabDDPM, many models have been proposed that 1) often considerably outperform TabDDPM and 2) do not rely on multinomial diffusion. In fact, models that treat categorical data in some continuous space exist (e.g., TabSyn [1] or CDTD [2]). It is unclear how the results extend to such models.\n- The missingness rates of 0.5 and 0.75 in the main results seem very extreme. The cited DiffPuter paper uses 30%. In reality, it is questionable why we would want to impute data when 75% of the data is actually missing. For more moderate missing rates, the performance advantage of the proposed method is not clear but it remains mostly competitive.\n-  Only using a single metric to evaluate sample quality of tabular data (in the case of inequality constraints) is not enough. Metrics like the detection score (see, e.g., metrics used in [1] and [2]) which evaluate the joint distribution more holistically would be interesting to see and give more insight into how the manifold guidance impacts samples.\n- The examined constraints for the conditional generation tasks are rather simple and focus on a maximum of two features (one categorical, one continuous), so the proportion that training samples are valid is still quite high. It is unclear how the framework performs under more constraints, which make the guidance more difficult. An ablation study should investigate what happens if the number of constraints is increased, such that the number of valid training samples shrinks. In the extreme case, is the method able to recover a single valid observation?\n- The paper makes the claim in line 112 that extending the results from Chung et al. to tabular diffusion is not trivial. One reason is that tabular data contains discrete features. This, however, is then solved by the rather trivial solution of simply one-hot encoding discrete features and treating them as continuous. Besides this encoding, there is no tabular-data-specific modeling in the paper.\n- Figure 2a) does not actually illustrate the behavior of a \"spotlight\" that becomes sharper near $\\mathcal{M}_0$.  In the Figure the position of $x_t$ relative to $\\mathcal{M}_0$ never changes. When the spotlight becomes sharper, I would expect an $x_s$ closer to $\\mathcal{M}_0$ where $s < t$. Note also that the orthogonal projection of $x_t$ and $x_s$ need not be the same.\n- Making the assumption in line 253 that $Q_t(x_t)$ is approximately orthogonal for all $t$ based on only a single empirical observation seems not well-founded. Could this assumption not be weakened by making $\\eta$ time-dependent, such that the guidance term has more impact when it is also more likely to hold, i.e., as $t\\rightarrow 0$?\n- It is not clear what tabular diffusion framework is assumed. It appears to be very similar to to StaSy model [3], which also one-hot encodes categorical data. The authors should be more specific about the framework they are using. If it can be used on any or most tabular diffusion frameworks (maybe under certain conditions) this should be stated as a strength of the framework. Coming up with an entirely new generative model does not highlight the strength of the potentially more general approach.\n- Getting the tangential gradient requires backpropagating through $\\epsilon_\\theta$. It should be clarified how costly the method is, in particular at inference time and relative to the other baselines.\n- Based on the inference time losses and the results, it should be highlighted by the authors that it is not guaranteed that samples satisfy the specified constraints. The approach is more similar to penalization than imposing hard constraints.\n- The authors do not discuss how their approach differs from classifier-guidance when using similar losses.\n\n---\n\n\n[1] Zhang, et al. (2024) Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Space. ICLR.\n\n[2] Mueller, et al. (2025) Continuous Diffusion for Mixed-Type Tabular Data. ICLR.\n\n[3] Kim, et al. (2023) STaSy: Score-based Tabular data Synthesis. ICLR.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)