ICML 2025

TabNAT: A Continuous-Discrete Joint Generative Framework for Tabular Data
Hengrui Zhang * 1 Liancheng Fang * 1 Qitian Wu 2 Philip S. Yu 1
Abstract
While autoregressive models dominate natural lan-
guage generation, their application to tabular data
remains limited due to two challenges: 1) tabular
data contains heterogeneous types, whereas au-
toregressive next-token (distribution) prediction
is designed for discrete data, and 2) tabular data
is column permutation-invariant, requiring flexi-
ble generation orders. Traditional autoregressive
models, with their fixed generation order, struggle
with tasks like missing data imputation, where the
target and conditioning columns vary. To address
these issues, we propose Diffusion-nested Non-
autoregressive Transformer (TabNAT), a hybrid
model combining diffusion processes and masked
generative modeling. For continuous columns,
TabNAT uses a diffusion model to parameterize
their conditional distributions, while for discrete
columns, it employs next-token prediction with
KL divergence minimization. A masked Trans-
former with bi-directional attention enables order-
agnostic generation, allowing it to learn the distri-
bution of target columns conditioned on arbitrary
observed columns. Extensive experiments on ten
datasets with diverse properties demonstrate Tab-
NAT’s superiority in both unconditional tabular
data generation and conditional missing data im-
putation tasks.
1. Introduction
The growing demand for synthetic tabular data in critical
applications such as privacy-preserving data sharing and
augmented machine learning pipelines (Fonseca & Bacao,
2023; Hernandez et al., 2022) has spurred significant inter-
est in deep generative models. While multiple architectures,
*Equal contribution
1Computer Science Department, Univer-
sity of Illinois at Chicago, Chicago, United States 2Eric and
Wendy Schmidt Center, Broad Institute of MIT and Harvard, Cam-
bridge, United States. Correspondence to: Qitian Wu <wuqi-
tian@broadi...