Using  maximal information auxiliary variables to improve synthetic data generation based on TabPFN foundation models
Authors: Elias Chaibub Neto
Venue: ICLR 2026 Poster (2026)
Keywords: tabular synthetic data generation
in-context learning
tabular foundation models
Primary area: generative models
TL;DR: We introduce MIAV, a strategy for enhancing synthetic data generation with TabPFN and other PFN-based tabular models. MIAV is theoretically grounded, improves weakly-associated variable handling, boosts efficiency, is order-invariant, and competitive

Abstract: Synthetic data generation for tabular datasets is shifting toward the use of large, general-purpose foundation models. TabPFN, a state-of-the-art example, uses in-context learning to generate probabilistic predictions conditioned on observed examples in a single forward pass. However, when variables are only weakly associated with others, the model's ability to generate realistic synthetic data deteriorates, as the context examples provide little predictive signal. To address this, we introduce the maximal information auxiliary variable (MIAV) strategy, which increases context information with auxiliary variables constructed by rank-matching random noise variables to real data. We establish theoretical properties of the approach which explain its good performance for weakly associated variables. Additional practical advantages of the MIAV approach include improved computational efficiency and invariance to variable order during the synthetic data generation process. Empirical evaluations, on simulated and real datasets, illustrate how the MIAV strategy improves data generation when compared to direct application of TabPFN, and is competitive against other baselines. To illustrate the generality of the MIAV approach we also present an implementation based on the TabICL model (a more scalable tabular foundation model restricted to classification tasks) for performing synthetic data generation on categorical datasets. Overall, MIAV offers an effective foundation model–based alternative to bespoke synthetic data generators.

Published as a conference paper at ICLR 2026
USING MAXIMAL INFORMATION AUXILIARY VARI- ABLES TO IMPROVE SYNTHETIC DATA GENERATION BASED ON TABPFN FOUNDATION MODELS
Elias Chaibub Neto
Sage Bionetworks, Seattle, WA 98121, USA elias. chaibub. wtw..uu....ttsssgeeebeeeec.. org
ABSTRACT
Synthetic data generation for tabular datasets is shifting toward the use of large, general-purpose foundation models. TabPEN, state-of-the-art example, uses in-context learning to generate probabilistic predictions conditioned on observed examples in single forward pass. However, when variables are only weakly associ- ated with others, the model's ability to generate realistic synthetic data deteriorates, as the context examples provide little predictive signal. To address this, we intro- duce the maximal information auxiliary variable (MIAV) strategy, which increases context information with auxiliary variables constructed by rank-matching random noise variables to real data. We establish theoretical properties of the approach which explain its good performance for weakly associated variables. Additional practical advantages of the MIAV approach include improved computational effi- ciency and invariance to variable order during the synthetic data generation process. Empirical evaluations, on simulated and real datasets, illustrate how the MIAV strategy improves data generation when compared to direct application of TabPFN, and competitive against other baselines. To illustrate the generality of the MIAV approach we also present an implementation based on the TabICL model (a more scalable tabular foundation model restricted to classification tasks) for performing synthetic data generation categorical datasets. Overall, MIAV offers effective foundation mosel- alternative to bespoke synthetic data generators.
INTRODUCTION
Accessible data is crucial for advancing machine learning research. In practice, however, real-world datasets often contain sensitive information, restricting their open distribution within the research community. A promising solution is the generation of synthetic datasets that closely replicate the properties real data while avoiding direct disclosure (Lu et al., 2023).
While synthetic data has long been explored through bespoke statistical models and machine learning algorithms, the field is now undergoing a paradigm shift driven by advances in large-scale, general- purpose models. Traditional approaches, such those by Borisov al. (2023), Cresswell and Kim (2024), Jolicoeur-Martineau et al. (2024), Kotelnikov et al. (2023), Nowok et al. (2016), Reiter (2005), Shi al. (2025), Watson et al. (2023), Xu et al. (2019), Young al. (2009), Zhang al. (2024), Xu et al. (2025), among many others, typically rely on dataset-specife training, demand substantial domain expertise, and often struggle with knowledge transfer across datasets. Tabular foundation models (Hollmann et al., 2023; den Breejen et al., 2024; Koshil et al., 2024; Feuer et al., 2024; Ma et al., 2024a; Ma al., 2024b; Xu et al., 2024; Zeng et al., 2024; Muller al., 2025: Hollmann et al., 2025; Qu et al., 2025; Garg al., 2025) offer promising alternative. By learning broad, transferable representations tabular data, they enable strong performance in supervised learning tasks with minimal additional training.
In particular, TabPEN (Hollmann et al., 2025) represents a state-of-the-an tabular foundation model, trained millions of diverse synthetic datasets covering wide range of feature types, noise structures, and functional relationships. This diversity allows it to leverage a broad, transferable prior over tabular data distributions. TabPFN enjoys a solid theoretical foundation as it corresponds

---

Published as a conference paper at ICLR 2026
to prior-data fitted network (PFN) (Miller et al., 2022) and can be interpreted as approximating Bayesian prediction under the prior induced by its synthetic training data. TabPFN relies on in- context learning (Brown et al., 2020) (ICL) for generating probabilisticc predictions. At inference time, the pre-trained foundation model employs training features, X and training targets, y"r as the "context" data, whereas the test set features, X'x, play the role of the "query". The output of the query is a sample/prediction y from the posterior predictive distribution of y ".t.s""PP P(y | X X y²",, generated by single forward pass through the model's neural network. TabPFN has been shown to achieve state-of-thee afrt performance in classification and regression tasks in small datasets and, due to its generative nature, can also be directly used to perform synthetic data generation.
Despite its promise, applying TabPEN directly to synthetic data generation reveals an important limitation: the method performs poorly for variables that are only weakly associated with the rest of the dataset. This is expected: when the target data y is uncorrelated with the features X, the context examples X y provide no useful signal for learning how to map X to y. Consequently. when queried with X the model is unable to approximate the distribution of yd While this limitation is less consequential for supervised learning tasks, it poses a significant caveat for synthetic data generation (see Appendix A for details). In principle, exemplar based declarative programming strategies could mitigate this issue, but doing so would likely require fine-tuning, or even retraining the foundation models.
In this paper, we address this problem by showing how to generate high-quality synthetic datasets with the current TabPFN model. Our approach leverages maximal information auxiliary variables (MIAV) for in-context learning. We construct these variables through simple rank-matching of random noise to the real data and establish two key theoretical properties: (i) conditional on its MIAV, a variable X, is independent of all other variables, and (ii) the MIAV of X, retains maximal information about X, in a non- non-parametric, information- thhooretcc sense (see Theorem 1). We further demonstrate that MIAV-based synthetic data generation corresponds to the correct factorization of the posterior predictive distribution conditioned on the original data and MIAVs. Together, these results provide the foundation for more effective synthetic data generation strategies using TabPFN models.
In addition to its theoretical strengths and its ability to generate high-quality synthetic data under weak association settings, our proposed strategy offers several practical advantages. First, unlike the direct synthetic data generation approach of Hollmann et al. (2025), which is sensitive to variable order and therefore requires aggregating results across multiple variable order permutations, our method is invariant to variable order. Second, regarding computational efficiency, TabPFN's 's runtime for fixed sample size is primarily determined by the number of context features, as its complexity scales quadratically with the number of features. Since our approach uses only one feature per variable when generating synthetic data, it attains maximal efficiency and eliminates the need for aggregation across multiple runs
To illustrate the issues around the direct use of TabPFN for synthetic data generation under weak association settings, we describe a couple of direct implementations and compare their performance against the MIAV approach in simulated data experiments (where we are able to control the strength of the statistical associations among the data variables), as well as, on extensive real-world data experiments based on 43 distinct datasets. (For completeness, we also include comparisons against other baseline generators.)
We conduct our evaluations in the setting of privacy-preserving data sharing (Rajotte et al., 2022), where the objective is to produce synthetic copies of real datasets that retain their statistical properties while simultaneously mitigating privacy risks. Accordingly, we assess the performance of the TabPFN-based synthetic data generation strategies using both data fidelity and privacy metrics.
Importantly, our synthetic data generation strategy can be directly applied to other tabular foundation models that approximate Bayesian inference. To demonstrate this, we also implemented our approach using the more scalable TabICL foundation model (Qu et al., 2025), a recently proposed alternative to TabPFN that alleviates some of its data size limitations. (Since the current version of TabICL only supports classification, our implementation and evaluation were restricted to 8 additional real-world categorical datasets.) These additional results highlight the generality of our strategy and suggest that, as PFN-based tabular foundation models continue to evolve, they can be seamlessly integrated into our framework.
2

---

Published as a conference paper at ICLR 2026
In summary, this paper proposes effective, computationally efficient, and generalizable approach that leverages the in-context learning capabilities of modern tabular foundation models to generate synthetic data aiming to facilitate data sharing. It provides a foundation-modd model alternative to traditional synthetic data generators built on the earlier paradigm of bespoke MI. models.
2 NOTATION
Throughout the text, random variables represented in italics, vectors of random variables shown in boldface, e.g., X = (XI,.... ",)' and P(X) is used for probability statements involving random variables. Data matrices and data vectors are represented in uppercase and lower case boldface, respectively. (E.g., if X is an n X p matrix, than the jth column of X is represented by xz.) We use the notation X-j represent the matrix obtained by removing the jthh column from X, and the notation Xej to represent the matrix comprised by the first j columns of X. (Similarly, in the case of a set of random variables, we use the notation Xz, to represent the subset of X containing elements to j - 1.) We adopt the superscripts tr and is to represent the training and test sets, and we let qo (x)* X
x x"") represent TabPFN model (either a regression or a classification model
depending on whether the variable j is numeric or categorical), where x ~ (x()" X X
represents the prediction generated by the model. TabPFN uses both the training features, X",, and training targets, x5r, as examples during the in-context training step, but only the test set features, X during the in-context query step, where the model is asked to generate prediction of the test set targets based on the examples from the training set and the values the test set features.
Because our goal is generate synthetic data copies of given datasets, rather than performing supervised learning tasks, our notation does not explicitly differentiate between feature and targets variables (as the same variable can sometimes play the role of feature and sometimes of target during the synthetic data generation process). Hence, we use the notation (x(x) X X =
go (xx** X X j: x)"") instead of the notation go X X """,, more commonly used in the TabPFN literature.
RELATED WORK
Although the literature on synthetic data generation (SDG) using bespoke machine learning models extensive (see Tond-Taylora al., 2021; Lu et al., 2023, and references therein), SDG based on tabular foundation models remains underexplored. To the best of our knowledge, only two prior studies have addressed this problem. Ma et al. (2023) introduced the TabPFGen algorithm, which relied on an earlier version of TabPFN (Hollmann et al., 2023) that could not handle regression tasks. To produce continuous data, TabPFGen employed an energy-based procedure for generating features no longer necessary with the current version of TabPFN (Hollmann et al., 2025), which supports both classification and regression and can directly generate categorical and numerical data. SDG was also discussed in Hollmann et al. (2025), but only superficially: it was demonstrated on a single dataset without any formal evaluation of synthetic data quality. That work primarily focused on establishing TabPFN's state-of-the-ar performance in supervised learning, with SDG presented merely as secondary capability. In this paper, we (i) highlight the limitations of the direct SDG approach in Hollmann et al. (2025), (ii) propose a more effective alternative strategy, and (iii) conduct extensive evaluations of TabPFN-based SDG methods.
conditional on classification labels. Such procedures
4 DIRECT STRATEGIES FOR SDG BASED ON TABPFN
Here we describe the simple strategy, suggested in Hollmann et al. (2025), for performing synthetic data generation with the TabPFN model, alongside an alternative variation of this direct approach. Their main limitations are discussed and illustrated using simulated datasets (where we can control the strength of the statistical associations between the variables).
Let P(X X represent the posterior predictive distribution (PPD) of the test data conditional on the training data. This conditional joint probability distribution can be fully factorize as,
P
P(x** ts
x")
II j=1
P(X**
X x"),
(1)
3

---

Published as a conference paper at ICLR 2026
X
[b]
X
[C)
X
[d]
X
(e)
Xs
original
250
original HIAV
original
original MIV
]F
ceratty
cersity
150
MIAV
0.4 0.8
onigial
50
50
FC MIAV
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.7
0.8
0.9
100
0.0
0.5
1.0
0.0
0.2
044
0.6
0.8
varlable values
variable values
variablle values
variable values
varable values
Figure 1: Comparison of marginal distributions generated with the JF, FC, and MIAV strategies.
where X represents subset of X containing elements to j 1, and represents the number of features.
Hollmann et al. (2025) suggested generating synthetic data by following the factorization of the joint PPD,
P(X x") = II (x X
(2)
where, due to the in-context learning (ICL) nature of TabPFN, the conditioning on the training set is done on X and x rather than X Note that the approximation in equation 2 represents, in actuality, two different levels of approximation. The first is w.r.t. the approximation of the distribution P(X+* ش ( by the distinct distribution P(X}= X X x which no longer conditions on training variables Xy for j' j. The second is w.r.t. the approximation of P(X)* X X X."" by the transformer model qo (x]] X
x
Furthermore, because the TabPFN model cannot condition on an empty set (as you need provide the model with some input for it to perform ICL) the first product term in equation equation2Zrequires requuirs conditioning in variable Xo. which is not part of the data. Following the suggestion by Hollmann al. (2025), we adopt a random noise feature as our Xo- A detailed description of the implementation of this strategy, denoted "factorization of the joint PPD" (or JF for short), is provided
Algorithms and and 3 in Appendix [ (As pointed by Hollmann et al. (2025), the order of the variables in the joint factorization of the PPD can also affect the results, and Hollmann et al. suggest using a permutation sampling approximation of Janossy pooling to remedy this issue. This requires, however, the generation and aggregation of multiple synthetic datasets generated from random permutations the order of the columns of the real data and is not implemented in experiments.)
Figure iillustrates the application of the JF generation strategy (and other approaches that will be described later) to a simulated dataset containing data draw from highly correlated beta distributions. (See Appendix C for details.) To simulate an uninformative feature, we randomly shuffled the data of variable X2, that it is completely uncorrelated with the other variables. (Figure 5p in Appendix [ shows the correlation matrix for these variables.) The black densities represent the original ("real") data while the orange ones show the synthetic data generated with the JF strategy. Not surprisingly, this example shows that TabPFN provided very poor approximations for the distributions of X1 and X2. In the case of X1, the ICL based on Xo is poor because Xp is a random noise variable which contains no information about Xi- In the case of X2, the ICL again fails because X, does not contain information about X₂ (which is uncorrelated from all other variables).
An alternative approach is to use the full conditional distributions of each variable in the synthetic data generation (denoted as FC, for short). This strategy implemented using the factorization,
P II (x)
X x
where X_, represents a subset matrix obtained by dropping column from X. A detailed description
of our implementation of this strategy is provided in Algorithms and 5 in Appendix B
Namely, Hollmann et al. (2025) generate N distinct synthetic datasets, using different random permutations of the order of the variables during the synthetic data generation process, and average the results across the N synthetic datasets reduce variability and decrease the dependence of the result on the variable order.
4

---

Published as a conference paper at ICLR 2026
[a) p=-0.95
[D] 0.70.75
[C) p=-0.5
[d)
p=-0.25
(e)
p=0
original
original
1o0oo00 0.4 0.8
IIVV
40
ongyial
origiaal
70
FC MIAV
FC
FC MAV
00
0.0
0.5
1.0
0.0
0.5
10
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
varlable values
variable values
variablle values
varlable values
varable values
Figure 2: Degrading performance of JF and FC as association strength decreases. MIAV is unaffected.
Despite the fact that this fully conditional factorization does not correspond to proper factorization of the PPD, this strategy has a few practical advantages. First, eliminates the need for coming up with Xg variable, as the data for X1 is generated from it's full conditional distribution. Second, the generation of each variable leverages information from all other variables. Third, this approach is unaffected by the order the variables. It's main practical disadvantage is that more expensive to compute since increasing numbers of variables lead to increases in compute time. (As pointed by Hollmann al. (2025), the time complexity of TabPFN Onn² + p²), where n and p represent, respectively, the number of rows and columns of the data.)
The blue densities in Figure T illustrate the application of the FC generation strategy. Now, the distribution of X1 is nicely recovered by the FC strategy. (This is easier to visualize in Figure 5 in Appendix [ which reports the same results T a different display.) The approach, however, still fails for X2 because this variable is uncorrelated to all other variables.
In Appendix we describe additional variations of the JF direct data generation strategy.
4.1 PERFORMANCE DEGRADATION IN DATASETS CONTAINING WEAK ASSOCIATIONS
The illustrative examples in Figure T were based in data simulated with very strong correlations. The performance of direct strategies such JF and FC, however, is strongly influenced by the strength of the statistical associations among the variables. Because TabPFN relies on ICL for generating predictions, and weakly associated features provide little information about the target variable, its performance suffers in datasets with weakly associated variables. To illustrate this point, Figure presents the application of the JF and FC strategies (among others) to datasets with decreasing correlation strengths. (The association strength is controlled by the p parameter, as described in Appendix C. Due to space limitations, the figure only reports results for single variable (X4)). The full results are presented in Figures to in the Appendix [ Figure clearly shows that the quality of the synthetic data generated by the JF and FC approaches (orange and blue densities) decreases with decreasing correlation strengths.
CONSTRUCTING MAXIMAL INFORMATION AUXILIARY VARIABLES
The previous section illustrates how direct application of TabPFN for synthetic data generation is problematic when variables lack strong statistical associations with other variables. A simple strategy for improving the synthetic data quality to augment the dataset with auxiliary variables that are highly associated with the real data variables.
To this end, inspired by recent work in non- -orrameettiic and model free data synthesis (Chaibub Neto, 2025), we propose simple approach in which we rank-match arbitrary noise variables to the real data variables. The procedure described detail in Algorithm [ and its output what we denote maximal information auxiliary variable (MIAV).
The basic idea is to induce a monotonic mapping between random noise variable and the real data vector x,. Starting in line 2, the algorithm first draws a sample of size (equal to the length of Xj) from an arbitrary random noise variable and then sorts it from lowest to highest values. (In our implementation sample random noise from uniform distribution in the [0. 1] interval. This choice is, nonetheless, unimportant as the approach is not sensitive to the noise distribution used to construct the MIAV. See Appendix [ for further details.) In lines the algorithm ranks the values of xj. If
5

---

Published as a conference paper at ICLR 2026
Algorithm 1 CGeerceehaiiillliemmmmmoolaaaaar
1: Input: data vector, X;
2: mj f Sort ((Ceeeateeanndoooooiivcccoao
length(xx))) [Generate sorted random noise vector. ]
3: x, is numeric then
4: r Rank(x;) [Compute ranks xy. Ties are broken random..]
5: end if
6: if xj is categorical then
7: r f- NumerreRakkInooonngggCCrteeeeeee Varpaaaaaetottcacaaaalalsesssss (Described in Algorittmn 10 the Appendix.
8: end if'
9: m, f- my(r] [Re-order the entries of m, according the ranks of xy. The result vector my with idcatical ranks as x,.)
Output: the auxiliary variable mj
M,, X, distr.
M, X2 distr.
Mg Xg distr.
Ma. Xg distr.
M,
X, distr.
X1 Mis
X2 Ma
Xg Mg
Xs
(f)
X
X
X
X
04 03
60 P4 Aucupp
Ms
X2
00
044
-0.2
0.8 1.2
0.4 0.8 122
0.0
d.s 1
X5
variable values
veriable values
variable values
variable values
varable values
M, vs X
M, vs X,
M, vs X,
M, vs. X
M, vs X,
assoc. M
()
00
0.0
98
0,4 0a
[k)
M1
(8)
00
0.4
00
H2
N3
M4
0
M5
0.2
0.4
0.6
0.8
0.0
0.4
0.70
0.85
1.00
0.0
0.4
0.8
1.0
2.0
3.0
4.0
X1
X1
Figure 3: MIAVs illustrative example. panels and I. positive correlations are represented in blue and negative correlations are represented in red.
Xj is numeric, the algorithm computes its ranks in the standard way (line 4), breaking ties among identical values using the random assignment approach. If x, is categorical, the algorithm applies numeric rank encoding to the categorical variables, described in Algorithm TO in Appendix F and originally proposed by Chaibub Neto (2025). (In a nutshell, Algorithm T0 counts the number elements of X, in each factor level and distributes numerical ranks ranging from 1 to n randomly
within each level of the categorical variable Xj. Appendix F also includes an illustrative example.) Finally, line 9 the algorithm re-orders the entries m, according to the ranks of x,. The result is vector m, with identical ranks to Xj, that is, the position of the lowest value of m, is the same as the position the lowest value of Xj. the position of the 2nd lowest value of m, is the same as the second lowest value of xj. and distributions of the X, variables in black and their respective MIAVs, Mj. in red. (The MIAVs follow uniform distributions since in implementation
Figure provides some examples. Panels
show the
draw random noise from uniform distributions.) Variable X5, is discrete assuming values 1, 2, 3, and 4. Panels to k show scatterplots of the M, vs X, data, illustrating the monotonic relations. Panels f and show the pairwise associations for the X, and M, variables, respectively, and illustrate that, as expected, the M, variables recapitulate the associations of the X, data.
This procedure has some nice theoretical properties, described in the following result.
Theorem 1. Let M, represent the auxiliary variable generated from X; by Algorithm represent an arbitrary variable other than X, or Mj. Then, ton-parametrically.
7 and Y
1. T(X;;Y M2) 0, i.e., the conditional mutual information of X, and Y given M, is O.
2. H(,, M2) 0, i.e., the conditional entropy of X, given M, is O.
The proof is presented in Appendix G Note that this result holds non- non- parametrically, in the sense that any continuous variable is first discretized into n bins (where n represents sample size), so that can the discrete probability (and non-parametric) definitions of these coormation-thhoretfi

---

Published as a conference paper at ICLR 2026
quantities. This is justifiable because sample of size from a continuous variable always be viewed as the sample of a discrete variable with n distinct levels, each observed with frequency 1 1/n.
This result shows that the auxiliary variable My, generated by Algorithm I has the following two key properties. First, it contains maximal information about Xj. (This holds in the conditional entropy sense, since H (X; M,) 0 implies that X, is completely determined by Mj in non-parametric rank-based sense.) Second, conditional of Mj. X, is statistically independent of any other variables. These two properties are key for the synthetic data generation approach that we propose next.
6 SDG USING MAXIMAL INFORMATION AUXILIARY VARIABLES
Let M = (M y Mp)' represent the set of random variables M, generated by Algorithm Now, consider the augmented posterior predictive distribution of X given X M and M
p
P(X 1s
M .M" x
II P ==1
x X M M
(4)
Now re-writing,
M P(X}* X <j: X", M " P(X)a x "",,,",; Mj., -,,,, Mj),,
(5)
and recalling that X3 is independent from all other variables conditional on Mfs, its follows that,
P( X X M M ,M",, M = P(X | M)
(6)
which, for the same reason, can also be re-expressed as,
p Ml-) P(X)* MV"MM."71
(7)
a format that is better suited for performing in-context learning with a PFN model. Hence, the PPD augmented with the set of maximal information auxiliary variables can formally be expressed as,
P
P(X x M M
II 3=1
P(X) M X
(8)
and readily approximated by a trained TabPFN model as,
P(X La
X M "Mr) v II
go (x m/ my x
(9)
where ICL for each variable X2 is performed by training on my and x and querying on m
This formulation, denoted the "Maximal Information Auxiliary Variables" strategy (or MIAV strategy for short) has several practical advantages. First, it is the most efficient strategy in terms of computa- tion, since ICL of each variable X, is performed using a single variable M, (recall that the complexity of the TabPFN model scales quadratically on the number of columns of the table). Appendix [H reports complexity analyses and compute time benchmark experiments comparing the MIAV, JF. and FC strategies. Second, contrary to all the other direct generation strategies descrihed in Section 4 the MIAV approach based on a proper factorization of the (augmented) PPD (the approximation in cquation[Jis only w.r.t. the transformer model approximation to the predictive distribution). Third, contrary the JF strategy, the MIAV approach is invariant with respect to the order of the dataset columns. Fourth, and most importantly, the MIAV approach handles uninformative features and performs well datasets containing weakly associated features. This last point is illustrated by the red densities in Figures [nd (see also panels to in Figures 9 in Appendix D) where the MIAV approach closely recapitulates the original marginal distributions. Quite importantly, note that while the augmented joint PPD in equation 8 factorizes into separate P(X)* |X MM Mf") com-
ponents, the synthetic data generated by the qe (x xx m m still recapitulates the association structure of X because this association is indirecily induced by the MIAVs (recall that M mimics the
association structure in X, as illustrated in panels f and I of Figure 3 Algorithms 6 7 Appendix
B provide implementation details about the MIAV synthetic data generation approach.
2 Here, is important to clarify that we are able to condition on the test set auxiliary variables. M because we always have unrestricted access to the full X data (which is only split into X" and x for the sake of ICL). Hence, the test set is always available and we generate the corresponding MIAV matrix M (Our goal is to generate synthetic copy of the original data, rather than making predictions about unseen test data.)
7

---

Published as a conference paper at ICLR 2026
7 EXPERIMENTS BASED ON TABPFN MODELS
Experiments. We performed three sets of experiments. The first, used simulated data draw from correlated beta distributions (generated as described in Appendix [ for which the strength of the associations among the variables. The second experiment, used a subset of 36 real-world datasets (Table 4 from the OpenML-CCI8 benchmark suite (Bischl et al., 2021). The third one, used 7 additional datasets (T evaluated in Hansen et al. (2023) and Chaibub Neto (2025). (Appendix L.3 contains further information and describes our rationale for selecting these datasets.)
control
Data splits. In each of the three experiment sets, every dataset was divided into two equal subsets. The first, referred to as the original data, was provided to the synthesizers, while the second, the holdout data, was never accessed by them. (Appendix [.]] provides further details about the data splits, including the description of additional data splits performed in the original data for generating the training and test sets used by the TabPFN models when performing in-context learning.)
Evaluation metries. Synthetic datasets generated from the original data were evaluated using fidelity, utility, and privacy metrics. Fidelity was assessed with the average KS-test statistic (KS), the 1.2 distance between association matrices (L2D), the detection test (DT), the Wasserstein distance (WD), and the energy distance (ED), which measure agreement with marginal distributions, preservation of pairwise statistical associations, distinguishability of real versus synthetic samples, and agreement with respect to joint distributions, respectively. Utility was assessed with machine learning efficiency (MLE) metric, which measures utility with respect performance in downstream prediction tasks by training learners on synthetic data and evaluating their predictive performances on real data (i.e., the holdout set). Privacy was evaluated with the distance closest record (DCR) and the sorted standard deviation interval distance (SSDID), which capture attribute disclosure risks, as well as the sorted distance-based record linkage (SDBRL), which measures re-identification risks. Further details on all evaluation metrics are provided in Appendix L.5
Baselines. In addition to comparing the MIAV-based synthetic data generation strategy with the joint factorization (JF) and full conditional (FC) approaches, experiments and 2 also included the SMOTE generator (Chawla et al., 2002). SMOTE well known for producing high-fidelity synthetic data, although this often comes at the expense of privacy when compared to other baselines (Kotelnikov et al., 2023; Kindji et al., 2024). We selected SMOTE as a baseline because, unlike deep barming-based generators. it is applicable to the small datasets used in our evaluations. (As shown in Table 26 out of the 36 datasets contain fewer than 2,000 samples.) In the third experiment, which extended our comparisons to include additional baseline generators: DDPM (Kotelnikov et al., 2023), ARF (Watson et al., 2023), TVAE (Xu et al., 2019), CTGAN (Xu al., 2019), and Bayesian networks (Young al., 2009), all implemented in Synthcity (Qian et al., 2024). For DDPM, TVAE, and CTGAN, we adopted the hyperparameter values reported by Hansen et al. (2023) and Chaibub Neto (2025), which had been optimized with Optuna (Akiba et al., 2019) using AUROC minimization of an XGBoost classifier. For ARF, we used the values from Chaibub Neto (2025). The corresponding hyperparameters are listed Tables and 7 Relying on these published configurations provided substantial computational savings, since hyperparameter optimization is particularly costly for deep learning- based models, and motivated our restriction of these baseline comparisons to only these 7 datasets.
involves larger datasets,
Experimental details. In all three experiment sets, we also compared against the holdout sets to establish reference values for each evaluation metric under the ideal scenario of generator that samples directly from the same distribution as the original data. To enhance statistical validity, we conducted 10 replications for each real-world dataset, each based on distinct original/holdout splits. For the correlated beta distribution experiments, results were similarly averaged over 10 replications per simulation setting, with variation introduced through different simulation parameters. We con- sidered five simulation settings corresponding to absolute correlations [e| 0, 0.25, 0.5, 0.75, 0.95. Additional experimental details are provided in Appendix [
7.1 RESULTS
Figure presents results (pooled across datasets) for the KS, L2D, DT, DCR, SDBRL, and SSDID metrics.
8

---

Published as a conference paper at ICLR 2026
[al KS (simal))
1D2.20 (nimal)
(C) BT (simul))
[O) (simul)
[C/SDNAL (simul)
[1] SSPID (simu))
104
Tokira 0C dataaor
0.34
1608-
0.22
9.
E
3011
1.00
85-
1.00
KS (CC180
12D (CC18)
DT (CC3)))
PCR (CC10)
[K] SPBRL (CC10)
SSDID (CC
01-
oe aectonnttaae antaae
820
12-
26
(m) KS (base.)
10 (base)
Ibased
DCR (base.)
SDBRL (barel)
SSSID Bbase
redinae maannrochhcoo Bittoue
S g
830-
sonee seee
&
R.
R
@ E
60
MAV
CTC
an1
TWME
SHUTE
SHUTE CTCAN
CTDAM TIME
SHOTE
Figure 4: Pooled experimental results. Top panels show results pooled across the 5 simulated dataset settings. The middle panels show results pooled across the 36 real-world datasets selected from the OpenML-CC18 suite. The bottom panels show results pooled across the real-world datasets used for the baseline generator comparisons. For the DCR metric, higher values indicate better privacy. For all other metrics, lower values indicate either better fidelity or better privacy.
In terms of fidelity, SMOTE generally performed best, with a slight advantage over MIAV (recall that lower KS, L,D,, and DT values indicate higher fidelity). MIAV, however, consistently outperformed IF and FC across all experiments and surpassed all other baseline generators in the third experiment, with the single exception of DDPM, which achieved better scores w.r.t. the DT metric (see panel o) (but still did worse than MIAV w.r.t. the other fidelity scores in panels m and n).
In terms of privacy, MIAV generally outperformed SMOTE the DCR metric (where higher values indicate stronger privacy protection) and showed even larger gains on the SDBRL and SSDID metrics (where lower values indicate better privacy). MIAV also tended surpass the other baseline generators with respect to DCR (panel o), but performed less favorably on SDBRL and SSDID (panels q and r). Across most experiments, JF and FC produced more private data than MIAV, although this came at the cost of substantially lower fidelity.
In general terms, the fidelity-privacy tradeoff patterns observed in the simulated datasets (top panels) closely mirrored those in the real datasets (middle and bottom panels). Overall, these experiments suggest a competitive performance of the MIAV-based synthetic data generator. Additional results broken down by dataset are provided in Figures 14 15 16
18 and 19 in Appendix [.7
Figure 13 in Appendix report the pooled results for the MLE, WD, and ED metrics. Overall, these evaluations show that the MIAV approach again achieves competitive performance with respect to these additional metrics (ranking among the top generators in most experiments). See Appendix [.6] for further details.
Appendix Jintroduces a noisy variant of the MIAV approach that applies controlled amounts of noise before generating synthetic data. This version, referred to as the noisy-MIAV strategy, can potentially enhance privacy protection in sensitive applications, albeit at the cost of reduced data fidelity.
8 EXPERIMENTS BASED ON TABICL MODELS
We also demonstrate the MIAV data generation strategy using the TabICL foundation model (Qu al., 2025) and compare its performance to TabPFN-based generators across 8 categorical datasets from the OpenML-CCII suite. Results, presented in Appendix K show that (i) MIAV-TabICI. and
9

---

Published as a conference paper at ICLR 2026
MIAV-TabPFN achieve very similar performance, and (ii) MIAV-based strategies, whether built on TabICL or TabPFN, tend to outperform the JF and FC baselines. This altemaatve implementation highlights that our synthetic data generation approach is not limited to TabPFN and can be directly applied to other PFN-based foundation models.
FINAL REMARKS
In this paper, we introduce the MIAV strategy, a more effective approach for leveraging TabPFN models in synthetic data generation. MIAV addresses key limitations of direct TabPEN application, offers improved computational efficiency, and can be readily applied with other PFN-based foundation models. We expect it to be especially useful in small-data scenarios, settings that are typically challenging for traditional synthetic data generators but where TabPFN excels. Our experiments on real datasets indicate that MIAV is competitive with established baselines built on bespoke machine learning models. It is worth noting, however, that PFN-based tabular foundation models are still in the early stages of development. As these models continue evolve, it is reasonable to expect that MIAV-based synthetic data generators built on future, more advanced PFN-based tabular foundation models might achieve even better performance.
Our approach inevitably inherits the limitations of the underlying tabular foundation model used for in in-context learning. For the TabPFN model used our experiments (namely, TabPFNv2), these limitations include: (i) a maximum data size of 10,000 rows; (ii) memory usage that grows linearly with dataset size, which can become prohibitive for very large data; and (iii) inference speeds that may lag behind alternative baselines. But, as mentioned above, these are early days in the development of PFN-based tabular foundation models and we expect that future releases will likely continue to relax limitations from the previous versions. For instance, new version of the TabPFN model, denoted TabPFN-2.5, has been recently released which is able to handle datasets with up to 50,000 rows (Grinsztajn et al., 2025). Furthermore, the more scalable TabICL model is already able to handle 500,000 rows but currently supports only classification tasks. Future versions of TabICL that extend to regression could be directly integrated with the MIAV strategy, thereby helping to overcome current model constraints.
In addition to limitations on the maximum number rows they can process, PFN-based tabular foundation models are also constrained by the number of columns they can handle. For example, TabPFNv2 and TabICL support datasets with up to 500 features, while TabPFN-2.5 increases this limit to 2,000. Importantly, however, these constraints do not affect the MIAV approach: because MIAV requires training PFN-based models using only a single feature per variable, it can be applied to datasets containing more columns than the column number limit of the underlying PFN model.
For tabular foundation models that do not approximate Bayesian inference, our approach may still provide a natural strategy for synthetic data generation through in-context learning. Assessing the feasibility of such extensions, however, is left for future work.
Finally, we point out that the MIAV strategy described here is really only intended for synthetic data generation and should not be used for improving predictive performance of supervised learners. As described in Section 6 generating MIAV variables requires unrestricted access to the full dataset X. which is partitioned into X and X and used to construct the corresponding MIAV matrices M and * Because MIAVs must be computed on the test set, the approach is inherently incompatible with supervised learning scenarios where test-set targets are unavailable. But, more importantly, should never be used to enhance supervised learning performance in settings where the full dataset is merely split into training and test subsets for evaluation purposes. In such cases, the generation of the MIAV variable associated with the test target would leak information about the target into the associated MIAV, and inclusion of this MIAV as an input in a supervised model would lead to an artificial boost in predictive performance due data leakage.
R and Python implementations of the MIAV strategy and the code for running the experiments in this paper are available on GitHub: https: / / thub. cww/echa echa wataaaibbbbbaaa.aa
10

---

Published as a conference paper at ICLR 2026
REFERENCES
[1] Akiba, T., Sano, S., Yanase, T., Ohta, T., and Koyama, M. (2019). Optuna: A next-generation hy- perparameter optimization framework, Proceedings of the 25th ACM SIGKDI International Conference on Knowledge Discovery & Data Mining (pp. 2623-2631).
[2] Bond-Taylor, S., Leach, A., Long, Y., and Willcocks, C. G. (2021). Deep generative modelling: comparative review of vaes, gans, normalizing flows, energy-bace based and autoregressive models. IEEE Transactions on Pattern Analysis and Machine Intelligence.
[3] Borisov, V., Sessler, K., Leemann, T., Pawelezyk, M., and Kasneci, G. (2022). Language models are realistic tabular data generators, in: The Eleventh International Conference on Learning Representations (ICLR), 2023.
[4] den Breejen, F., Bae, S., Cha, S., and Yun, S.-Y. (2024). Why In-Context Learning Transformers are Tabular Data Classifiers. S.Ki:22405.13396.
[5] Brown, T. B., Mann, B., Ryder, N., Subbiah, M., and Kaplan, et al. (2020). Language Models are Few-Shot Learners. NeurIPS 2020.
[6] Bischl, B., Casalicchio, G., Feurer, M., Hutter, F,, Lang, M., Mantovani, R. G., van Rijn, J. N., and Vanschoren, (2021). OpenML. Benchmarking Suites. NeurIPS 2021.
[7] Chaibub Neto, E. (2024). Statistical disclosure control for numeric microdata via sequential joint probability preserving data shuffling. Transactions on Data Privacy, 17(3):147-179, 2024.
[8] Chaibub Neto, E. (2025). TabSDS: Lightweight, Fully Non-Parametric, and Model Free Approach for Generating Synthetic Tabular Data. ICML,, 2025.
[9] Chawla, N. V., Bowyer, K. W., Hall, L. O., and Kegelmeyer, W. P. (2002). Smote: synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16, 321-357.
[10] Cresswell, J. C., and Kim, T. (2024). Scaling up diffusion and flow-based XGBoost models. arXiv 2408.16046
[11] Domingo-Fenen Ferrer, J., Muralidhar, K., and Bras-Amoros, M. (2020). General confidential- ity and utility metrics for privacy-preserving data publishing based on the permutation model. IEEE Transactions on Dependable and Secure Computing, 18(5):2506-2517. DOI: 10.1 R0.10099/DSC..22222668222
[12] Domingo-Ferrer, J., and Torra, V. (2001). quantitative comparison of disclosure control methods for microdata. In Confidentiality, disclosure, and data access: theory and practical applications for statistical agencies, edited by P. Doyle, J. Lane, J. Theeuwes, and L. Zayatz. 111-133. Elsevier.
[13] Drechsler, J. (2011). Synthetic Data Sets for Statistical Disclosure Control. Springer, New York.
[14] Feuer, B., Schirrmeister, R. T., Cherepanova, V., Hegde, C., Hutter, F,, Goldblum, M., Cohen, N., and White, C. (2024) TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks, arXiv preprint 13Xiv:2402.11137.
[15] Garg, A., Ali, M., Hollmann, N., Purucker, L., Muller, S., and Hutter, F. (2025). Real-TabPFN: Improving Tabular Foundation Models via Continued Pre-training With Weal-World Data. Proceedings of the 1st ICML Workshop on Foundation Models for Structured Data.
[16] Grinsztajn, L., et al. (2025). TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Models. public/rporttsTaabPFN 2_aibbpFN______11 2_cht report.ddffddae-2255.1111
[17] Hansen, L., Seedat, N., der Schaar, M., and Petrovic, A. (2023). Reimagining synthetic tabular data generation through data-centric AI: A comprehensive benchmark. Advances in Neural Information Processing Systems, 36, 33781-33823.

---

Published as a conference paper at ICLR 2026
[18] Hollmann, N., Muller, S.. Eggensperger, K., and Hutter, F. (2023). TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second. International Conference on Learning Representations (ICLR) 2023.
[19] Hollmann, N., Muller, S., Purucker, L., Krishnakumar, A., Korfer, M., Hoo, S. B., Schirmmistter, R. T., and Hutter, F. (2025). Accurate predictions on small data with a tabular foundation model. Nature, 637, 319-326.
[20]
A., Fatras, K., and Kachman, T. (2024). Generating and imputing tabular data via diffusion and flow-based gradient-boosted trees. AISTATS 2024
[21] Kindji, G. C. N., Rojas-Barahona,
Barahona, L. M., Fromont, E., and Urvoy, T. (2024). Under the hood of tabular data generation models: benchmarks with extensive tuning. arXiv:2406.1294.
[22] Koshil, M., Nagler, T., Feurer, M., and Eggensperger. K. (2024). Towards Localization via Data Embedding for TabPEN. Third Table Representation Learning Workshop at NeurIPS 2024.
[23] Kotelnikov, A., Baranchuk, D., Rubachev, I., and Babenko, A. (2023). Tabddpm: Modelling tabular data with diffusion models. In International Conference Machine Learning (pp. 17564-17579). PMLR.
[24] Lu, Y., Wang, H., and Wei, W. (2023). Machine learning for synthetic data generation: a review. arXiv preprint arXiv:2302.0400 2023.
[25] Mateo-Sanz, J.M., Sebe, F., and Domingo- Ferrer, J. (2004). Outlier protection in continuous microdata masking. International Workshop on Privacy in Statistical Databases. PSD 2004:
Privacy in Statistical Databases pp 201-215.
[26] Ma, J.. Dankar, A., Stein, G., Yu, G., and Caterini, A. L. (2023). TabPFGen Tabular Data Generation with TabPEN. Poster presented at the Table Representation Learning Workshop, NeurlPS 2023
[27] Ma, J., Thomas, V., Hosseinzadeh, R., Kamkari, H., Labach, A., Cresswell, C., Golestan, K., Yu, G., Volkovs, M., and Caterini, A. L. (2024). TabDPT: Scaling Tabular Foundation Models. arXiv:2410.18164.
[28] Ma, J., Thomas, V., Yu, G., and Caterini, A. (2024). In-Context Data Distillation with TabPFN. arXiv preprint arXiv:2402.06971.
[29] Muller, S., Hollmann, N., Pineda, S., Grabocka, J., and Hutter, F. (2022). Transformers can do Bayesian inference. ICLR 2022.
[30] Muller, A., Curino, C., and Ramakrishnan, R. (2025). MotherNet: Fast Training and Inference via Hyper-Network Transformers. ICLR 2025.
[31] Nowok, B., Raab, G. M., and Dibben, C. (2016). synthpop: bespoke creation synthetic data in R. Journal of Statistical Software, 74(11), 1-26. http:///bboorg/1101886/////////
v074.i11 1
[32] Pagliuca, D., and Seri, G. (1999). Some results of individual ranking method the system of enterprise accounts annual survey, Esprit SDC Project, Deliverable MI-3/D2.
[33] Qian, Z., Davis, R., and van der Schaar, M. (2024). Synthcity: a benchmark framework for diverse cases of tabular synthetic data. Advances in Neural Information Processing Systems, 36.
[34] Qu, J., Holzmüller, D., Varoquaux, G., and Le Morvan, M. (2025). TabICL: A Tabular Founda- tion Model for In-Context Learning Large Data. In Proceedings of the 42nd International Conference on Machine Learning (ICML 2025).
[35] Rajotte, J. F., Bergen, R., Buckeridge. D. L., El Emam, K., Ng, R., and Strome, E. (2022). Synthetic data enabler for machine learning applications in medicine. iScience, 25(11), 105331.
[36] Reiter, J. P. (2005). Using CART to generate partially synthetic, public use microdata. Journal of Official Statistics, 44144444
12

---

Published as a conference paper at ICLR 2026
[37] Shi, J., Xu, M., Hua, H., Zhang, H., Ermon, S., and Leskovec. J. (2025). TabDiff: mixed-tyre diffusion model for tabular data generation. ICLR 2025.
[38] Szekely, G. J., and Rizzo, M. L. (2023). The Energy of Data and Distance Correlation. Chapman Hall/CRC, Monographs Statistics and Applied Probability, Vol. 171. 466 pages.
[39] Watson, S., Blesch, K., Kapar, J., and Wright, M. (2023). Adversarial random forests for density estimation and generative modeling. In International Conference on Artificial Intelligence and Statisties (pp. 5357-5375). PMLR.
[40] Xu, L., Skoularidou, M., Cuesta-Infante A., and Vceramachaneni, K. (2019). Modeling tabular data using conditional GAN. Advances in Neural Information Processing Systems, 32, 2019.
[41] Xu, D., Cirit, O., Asadi, R., Sun, Y., and Wang, W. (2024) Mixture of In-Context Prompters for Tabular PFNs. ICLR 2025.
[42] Xu, S., Lee, C., T., Sharma, M., Yousuf, R., B., Muralidhar, N., and Ramakrishnan, N. (2025) Why LLMs are bad synthetic table generation (and what to do about it). arXiv:2406. 14541.
[43] Young, J., Graham, P., and Penny, R. (2009), Using bayesian networks to create synthetic data. Journal of Official Statistics, 25(4):549.
[44] Zeng, Y., Kang, W., and Mueller, A. C. (2024) Tabflex: Scaling tabular learning millions with linear attention. In NeurIPS 2024 Third Table Representation Learning Workshop.
[45] Zhang, H., Zhang, J.. Srinivasan, B., Shen, Z., Qin, X.. Faloutsos, C., Rangwala, H., and
Karypis, G. (2024). Mixed-type tabular data synthesis with score-based diffusion in latent space. in: The Twelfth International Conference on Learning Representations, 2024.
13

---

Published as a conference paper at ICLR 2026
APPENDIX
CONTENTS
Supervised learning versus synthetic data generation in the presence of uninformative features)
15
Algorithms for TabPeN-based synthetic data generation
15
Simulating correlated beta distributions
17
D Supplementary Figures
17
Additional direct synthetic data generation strategies
20
Additional algorithms
21
Proof of Theorem
22
Proof of statement when X, and are continuous
22
G.2
Proof of statement 1, when X, and Y are categorical
24
G.3 Proof of statement the remaining cases
26
G
Proof of statement 2
27
Complexity analysis and runtime experiments
29
H.I
Complexity analysis
29
H.2
Runtime experiments
[
Extended experiments section
30
Overview of data splitting into original, holdout, training, and test sets
30
Simulated data experiments
31
Real-world data experiments
31
1.4
Hyperparameters for the baseline models
33
Evaluation metrics
34
L6
Results for additional evaluation metrics
35
Extended results
37
J The noisy-MIAV strategy
43
K Synthetic data generation based on TabICL PFN models
45
Insensitivity to the choice of the random noise distribution
47
M LLM usage
50
14

---

Published as a conference paper at ICLR 2026
A SUPERVISED LEARNING VERSUS SYNTHETIC DATA GENERATION IN THE PRESENCE OF UNINFORMATIVE FEATURES
TabPFN achieves state-of-the-ar performance in supervised learning tasks 19). However, its direct application to synthetic data generation does not yield comparable results, as weakly associated variables make it challenging for in-context learning methods to capture the true data distribution. This issue is less pronounced in supervised learning, where noninformative features simply fail to contribute to the prediction. Even when the prediction is based solely on such features, TabPFN behaves appropriately. For instance, in binary classification with predictors completely uncorrelated with the target, it produces an AUROC of about 0.5, consistent with random guessing. Although uninteresting, this outcome is exactly what this scenario. By contrast, in the data generation setting, TabPFN's reasonable performance in classification does not carry over, as it fails to approximate the marginal distributions of the data and to capture its statistical association structure. An illustrative example is presented in Figure e Section D where direct application of TabPFN to datasets containing completely uncorrelated data fails to recover the marginal distributions (see panels h, i, and k and n, and q), and fails to recover the correlation structure of the original data (compare panels and against panel a).
would expect from any well-behaved classifier in
B ALGORITHMS FOR TABPEN-BASED SYNTHETIC DATA GENERATION
In Algorithms 2 and 6 the function Sew.raaeeeerrrrooooooooooooooani (.) represents
call to a TabPFN model qe (.).
Algorithm 2 [CC.iitlooimFaaccooiiiittTTTTTT
x(*)
1: Input: training data for ICL, X query data for ICL, X2*
2: Ber NumberOffoos// X
samples x
3: Mts < NumberOfRows X
[Ohtain number of samples of X x!
4: p < NumberOfColumms//
(Obtain number columns x
5: 4 <- [] (Create empty matrix store the synthetic data.)
6: x Generate Uniformly
(Ne)) (Draw sample size ney from uniform distribution.)
7: x & Generate
(is)) (Draw sample of size from uniform distribttion))
Z Z 1] CeneeattPrediitionn Using TabPFN(x;, x .x(" [Predict xf" using x and x as context, and
query. The prediction he from regression classification TabPFN model, depending whether x"" continucus
entegorical.) |
9: for to do X + X
10:
1111::000 [ 1)] [Select the first columns of X*".)
11:
X - (
[Select the first columns of x*x
12:
['i GeneratePrrdiccionnninnnnnnaa Tar is <j. <.j x² [Predict xf" using X and xef"
X
lest, and X as query. The prediction can be from regression or classification TahPEN model, depending on whether x) continuous calegorical.)
13:
end for
14: Output: synthetic data t
Algorithm 3 oonnfeecooiattooo/PPPPeeeeeeeee
1: Input: the original data, X
2: X,, X₂ + DataSplit(X) (Split the original data X into two subsets, X, and X2.) X
3: Zi f- (CCiiittooiiaaaatorriiiiotTTaaal
X2, X
(Generate synthetic data copy X,
using Algorithm
4: Z2 f- IILLittloiiiFAccoooriiiittaaaaaa
X X
(Generate symthetic data copy
using Algorithm 3
5: Z Concatenate Zz, Z2) [Concatenate synthetic Z, and Zz-
6: Z + RoundInteger Variables X, Z) (This function uses X determine which variables have integer and round the
values of the corresponding variables in nearest intoger.)
7: Output: synthetic data Z
15

---

Published as a conference paper at ICLR 2026
Algorithm 4 [CLLittfFlllCooddioon///TTPPPOOC
X
1: Input: training data for ICL. ' query data for ICL, X
2: p f NumberOFColuunns (*) |Obrain number of columns of X
3: 4 <- [:] [Create empty matrix the synthetic data.
4: for j = to p do
5:
_ t X j] Droop j-th columm X
6: _ t X X - (Drop the j-th column x
7:
7'( [b]] Generaterredictioonssiin TabPPN X x [Predict x using x and xe" as con-
X query. The prediction be from regression classification TahPPN model, depending whether xy
comtinuous or categorical.)
8: end for
9: Output: synthetic data 4
Algorithm 5 Tac
1: Input: the original data,
2: X1, X2 DataSplit(X X) (Split original
X1 X2.) 3: Z1 f (CLLiithFulCCCnndtiionllllll TabpNN X2, Xt X1)
(Gienerate a synthetic data copy of X,
using Algorithm
4: Z2 < :CC.iitFilICCCoottoooasaaaPPPPP
= x,, X
[Generate a synthetic data of Xg
using A
5: Z + Concatenate(2,, (2,, Z2) [Concatenate the synthetic datasets 2₁ and Z2
6: Z RoundInteger Variables(X (X. Z)
7: Output: synthetic data Z
Algorithm 6 ICLLwithMLAVTaaPPNN."" X
1: Input: training data for ICL, X query data for ICL, Xts
2: p <- NumberOrColumns/
( OObrain number of columns X
3: 4 < []] [Create empty matrix to store the synthetic data.]
4: for
to p do
5: m f
GearrueMMiiiinlllfooootttoAAllla (ari (Generate the MLAV for xy using Algo-
6: m
[Generate the MIA for sel using Algo-
rithu
7:
[i] + PeneeracPrrciccionnn/ingggo 
my x [Predict x using m and x5" as context,
and m as query. The prediction can be from a regression or classification TabPEN model, depending on whether x5" continuous categorical.)
8: end for
9: Output: synthetic data 7
Algorithm 7 M.LLAVibbPENGeeerttrrX
1: Input: the original data, X
2: X,. X₂ f- DataSplit X) (Spli the original dain X into two subsets, X; and X
3: Z +- (CLwithMIAVTabPFN x X2, X
X1)
(Generate synthetic copy using
4: Z2 +- ICLwinhMIAVTabPPNIXX X X X X2) (Generate synthetic data copy of X2 using Alg
5: Z Concatenate Z1, Ze) [Concatenate synthetic
2, and Z2.)
6: Z + RoundInteger Variables(X X, Z) (This function uses X to determine which variables have integer and round the
the comesponding
integer:)
7: Output: synthetic data Z
Z) uses the original data, X, to determine which variables have integer type and round the values of the corresponding variables in the synthetic data, Z, to the nearest integer. This procepsre step necessary because TabPFN (and most other data synthesizers) return real values for variables that are originally of integer type. In our experiments, apply the same post-processing step the SMOTE baseline.
In Algorithms and the function RoundIntegervarriabl
16

---

Published as a conference paper at ICLR 2026
C
SIMULATING CORRELATED BETA DISTRIBUTIONS
Here, we describe how we simulated correlated beta distributions used in the simulated data experi- ments, as well as, for the illustrative examples provided in Figures I and 3 in the main text, and Figures 5 6
and in Appendix D
The data was simulated follows. First, we simulate data from multivariate normal random variable Y~ (0, Σ). Next, for j ...... P, we compute the correlated uniform variables U, (22)) and the correlated beta random variables X; Gz, (U)), where + and Ga.6 represent, respectively. the cumulative distribution functions of standard normal variable and a beta variable with shape parameters a and
The multivariate Gausssian variable Y is generated with a Toeplitz structured covariance matrix Σ with off-diagonal entries djy = plt-al, and diagonal entries Øj = 1, for E [-1,1], 1]. (Note that under this correlation structure, neighboring variables are more highly associated than more distant variables, and the association decreases the farther apart the variables are. Also, for negative values the direction of the association flips depending on whether the exponent (i ji is even or odd.)
For the illustrations presented in Figures ! 2 and 3 we further randomly shuffle the data of variable X2 order simulate an uninformative feature uncorrelated with all other variables in the dataset. Furthermore, for the illustrations presented in Figure 3 we also discretize variable Xs. into four classes
D SUPPLEMENTARY FIGURES
(a)
orginnll
[b]
JF
(c)
[c]
MlA
(c)
holdout
p=-0.85
(T]
delta coms. F
[a]
(h)
2 IIl
K
ergan
deady
JP
0.
6.7 da
15
1b
84
delta cor:. FC
(m) X;
(n)
(o)
(p)
Argina
n-
0.5
67 d/
st.
3.4
delta cor:. MAV 12
(s)
X,
(U
X
2. Jul
(w)
23
orgas
MUW
0.5
0.3 2/2
da value
st. da 10
0.4
Varible
vakes
X
(22) Xa
X;
(24) X
(25) X5
oegnal
aighna
oriiial
Vel vilue
Aarabi vik
1ii values
scrabo wikes
Figure 5: Comparison of the JF, FC, and MIAV synthetic data generation strategies for correlated beta variables generated with p -0.95 Panel a shows the Pearson correlation matrices for the original data, while panels b to show the correlation matrices for the JF. FC, MIAV synthetic datasets and the holdout set (positive correlations are represented blue and negative correlations are represented in red). Panels f, I, r, and show the difference between the original data correlation matrix and the respective synthetic datasets and holdout set. The remaining panels show the marginal distributions generated by the distinct synthetic data generation approaches and for the holdout set.
17

---

Published as a conference paper at ICLR 2026
(a)
original
[b]
FC
R R
MIAV SREE
x A
holdout
0.75
delta com.
[0] X
Xx
Xs
Xa
Xs
orgeai
ap
orgeal
orginal
ergral
100
deey 20
awey
of
24
03
20
detta
X3
X2
X3
X4
X3
90
ereeal
ergiaal
of
24
05
3.0
25
varlade
(r)
X
X,
X2
Xa
Xx
Xs
oriinal
orignil
origial
F 1
33
60
02
60
67
to dl 10
ortible
values
12)
[2]] X
(z2) X+
[23] X
(zd) Xx
(z5) Xs Son
Po
crgeal
Tebel
Feldes
20
33
holoee 20
02
valasie
watibee wkees
Wertte vakes
write vikee
Figure 6: Analogous comparisons in Figure 5 but for data simulated with -0.75.
original
[b]
(c)
FC
MIAV
R 2 2
holdout
00=55
X4
delta com..
X
R
(_(h)
X
All
X
orgedt
orignal
wan x
erginal
2-
ergral
100
190 20
33
84
02
2/
delta
(m) X,
X
X;
Xa
Xs
premel
ergral
any
33
24
03
3b
0.9
(r)
dalta
2
X
X3
X2
X3
X
Xs
cra
oriiaal
cripel
orggial
33
9.6
el 30
2/a
Vartacie
wartadee
(2)
X
X
(z2) X)
(23) X;
(z4) X
(z5) Xs
cro
cripul
orginll
orggnal
20
33
10
09
05
3.5 oalirs
2/4
valuee
Figure 7: Analogous comparisons as in Figure 5 but for data simulated with p -0.5.
18

---

Published as a conference paper at ICLR 2026
(a)
original
[b]
12
(c)
PC
(d)
MIAV
holdout
0.25
X4
delta com.
Q Q
[0] X
g_
Xx
X3
Q
X1
X
orgeai
orginal
24 2p.
orginal
ergral
deec
dare
%
24
03
0.5
3.
24
1.
to
0.
10
detta
(m]
X3
X2
X3
X4
X3
ereeal
ergiaal
carott
33
24
05
24
0.1
2%
varladie
warlable
Varlabla Xa
(r)
X
X,
X2
Xx
Xs
orihnal HIW/
origial BISS
origial
33
69
02 0.
24
05
0.
10
to dl 10
0.4
ontible
(2)
[2]] X
(z2) X+
[23] X
(zd). Xx
(z5) Xs
crgeal
orgent Feldect
Tebelt
20
33
02
24 aatille wkess
25
04 write wikes
valasie ses
wrrttee vakes
katelle waees
Figure 8: Analogous comparisons in Figure 5 but for data simulated with p 00.55.
original
[b]
(c)
FC
(d)
MIAV
holdout R 3 2
=0
X04
delta comr.
2
X
X
X
orgeai
oripial
crgiaal
orgral
100
29 40
83
310
1p0
03
2.0
04
06
09
1/4
JI)
detta corr. Q
(m) X,
320
X2
X;
Xx
X
og
prginal
ergral
220 300
darey 2u
uney
drey
00
35
20 64
6.8
06
6a
25
2.
Variaute bas
Ir)
dalta
2
X
X3
X2
X3
X
Xs
cra
oriiaal MiW
cripiet
orggral
33
Nik
de
0.8
el 30
Varaacie
(2)
X
X
(z2) X)
(23) X;
(z4) X
(z5) Xs
crg
crepal
orginal
orginal
20
33
50
09
05
04
2/4
veratle values
Valuen
Figure 9: Analogous comparisons in Figure 5 but for data simulated with p p= 0.
19

---

Published as a conference paper at ICLR 2026
E ADDITIONAL DIRECT SYNTHETIC DATA GENERATION STRATEGIES
As described in Algorithms 2 and our implementations of the JF and FC strategies use the real test data when performing the in-context queries. In the case of the JF strategy, alternative approach would be to use the synthetic data generated in the previous steps when we query the model in the current generation step. That is, instead of generating the data according equation generate according to,
P (xx)
X x"))
(10)
where X represents the predictions (i.e., the synthetic data) generated in the previous j generation steps. We denote this strategy as the "updated joint factorization" (or UJF for short), and implement it Algorithms [ and 9 Note that Algorithm g implements two versions of the UJF approach. The first, simply takes bootstrap sample from the original X data as the "synthetic version" of X1. The second, uses the random noise variable X, to generate X₁ (similarly to the JF strategy). Intuitively, any drifts in the distribution of the synthetic data generated in the previous steps would propagate to the later ones.
would expect UJF under-perfone in comparison with the JF strategy, since
Algorithm 8 1
X version 1)
1: Input: training data for ICL., X query data for ICI., X!s; version of the UJF strategy
2: Mtr <- NumberOfRows! X
[Obtain number of samples of x
3: Nta NumberOfRRos X
[Ohtain number of samples x!*.)
4: < NumberOrCollunss( (X**) [Obtain number of columns of X
5: 4 < []] (Create empty matrix store the synthetic data.)
6: if version = then
7: Z'((, 1] - BootstrapSample(xf))
8: end if
9: if version = then
10:
sample new uniform distribution. ) f- Dirrrr/uununommmiiiiiitrtbtlllle (Taa) (Draw sample of size new from uniform distribution) [(1,1]
xfi
GenerateUniformly
11:
X
12:
CinnraacePrccciiitosnnnnnaaaaaao [xf" x xx²) (Prodict xt" using xf" and x²" context.
and x as query. The prediction can be from regression or classification TabPEN model, depending on whether x is continuous
categurical.)
13:
end if
14:
for
p do X + X [:::0 1)] [Select the tirst columns X X!".)
15:
16:
Zhe t ld (1 (Select the first columns of "2
17:
['] +- Generratercdictioonusnngg
<j X x [Predict x using X xe" as
texL and 2 query. The prediction can be from regression or classification TabPEN model, depending en whether xy is continuous categorical.)
18:
end for
Output: synthetic data t
Algorithm 9 Uppiec/ODnffecoozzzztooooPPooerr
Input: the original data,
2: X X2 f- DataSplit(XX) X) (Split original data subsets, X, and X2.!
3: Z1 f- CCw.wittupaaaeddooooaaoooooaaaoo
-2,, X
[Generate synthetic data
cupy using Algeritim]]
4: Zg f
_1,, X X
(Generate synthetic data
enpy of Xf using Algorithm
5: Z f- Concatenate Zz, Zz)
(Concatenate the synthetic datasets Zj and Zz..)
6: Output: synthetic data Z
This intuition is confirmed in Figure I0 which shows the application of version of the UJF strategy in purple (panels f to j) and of version (panels p) in cyan. For comparison, the plot also include
20

---

Published as a conference paper at ICLR 2026
original
UJ
F1 &
F2
JF
(a)
X
(c)
St
(d)
X1
X1
0=-0.75
(e)
delta UJ
X1
[g]
X
X3
X
X5
2.0
orisau
original
original
criginal
6
UjF
X2
Qurckty
deesty
10
dertt)
gind
a.6
0.4
0.2
ds 1.0
0.
valies
values
(k)
delta car. Uj F2
(I)
X1
[m)
X2
[n]
[o]
X
[p)
X5
120
orighnal
original
origial
criginal U]
U2
150
UJF
UJF2
30 40
Genstty
sp
dembtyy
dimeddy
X3
X4
20
0.2
6.6
0.0
0/2 6. 1.0
0.4
varadde solucs
vorlable values
var adle values
variable votues
variable values
(q)
delta cor, J
(r]
[s]
X2
[t]
(u)
X
[v]
120
orighial
original
original
criginal ]
dersity
80
cereity
dertity
derraty
04 99
X4
giaal
X5
0.2
0.6
0.0
0.4
0.8
0.2
ds
1.0
0.0
0.1
Var abla valucs
Varlable values
able values
varbable values
varibble values
Figure 10: Comparison of JF and the UJF approaches.
the results from the JF approach (panels to v). Even in version 1, where the algorithm samples the first variable from the correct distribution, the distributions of the synthetic data still ends drifting away from the original data distributions at later steps of the data generation approach. (Note that UJFI performs considerably worse than the JF approach for variables Xa. X4, and Xs.) version (panels to the performance is even worse because the quality of the X data generated in the first step already very poor.
F ADDITIONAL ALGORITHMM
Algorithm 10 Numnrrekkkammaoodddnngoooaocnate Variallssx))
1: Input: categorical variable data Xj
2: levels + ExtractLevels(xx) (Obtain levels of variable xy-]
3: tb +- Table Xx il) (Obtain level counts variable x
4: cumcounts + CumulativeSum((t) (Compute cumulative counts from the table counts.) ]
5: numlevels <- Length
[Obtain number levels.]
6: r < [] (Create empty vector to store the numeric rank encodings.)
7: for k to numleeels do
8: idr < Which(x; = levels [k))
(Obtain the indexes of the records for which equals level k.)
9: lowerbound t cumcounts [k]
[Compute the lower bound for the numerical encoding of level & of x;-)
10: upperbound +- cuncounts ik - 1]
[Compute the upper bound for the numerical encoding of level & of xy-]
nrseq <- Sequence(louerroonne upperbound) [Create sequence mumeric rank values starting at lower-
bound and ending upper- bound.]
rlidz] + (kandomPermutaaionnnrresr [Assign randomly shuffled numeric rank values the positions of x;
corresponding level
13: end for
14: Output: numerical rank encoding
21

---

Published as a conference paper at ICLR 2026
As example. suppose x; =AAAAABBBBBCBCCCCCCC.C. variable has four levels A, B. C' and D with counts AA = 4. nB = 3, nc = and n.D = 3. To generate the numerical rank encoding presented in Table the algorithm transforms the categorical values according to the mapping, mapr, in Table T Note that this mapping corresponds to an arbitrary ranking of the categorical levels according to the arbitrary order A < B < C < D. That is, starting with class A, the mapping assigns ranks 1, 2, 3, and 4 (in random order) to the four tied elements A in Xj. (Note that by assigning the ranks in random order the mapping effectively using the "at random" method to break ties among identical values of the categorical variable.) For class B, the mapping assigns ranks 5, 6, and 7 (in random order) to the three tied elements B in x,. For class assigns ranks 8 to 12 (in random order) to the five tied elements C. Finally, for class D, the mapping assigns ranks 13, 1 4, and 15 (in random order) to the three tied elements D in X,.
This categorical
Table Toy example illustrating the application of the numeric rank encoding procedure for categori- cal variables implemented by Algorithm 10 to x, = (A, A, A, A, B, B, B, C, C, D)"
mapr
A
(1,2,3,
3, 4},
B
(5,6,7).
7),
C
(8, 9, 10,
12,
D (13,14,15)
X
A
A
A
A
B
B
B
C
C
C
C
C
D
D
D
4
2
7
5
6
9
8
11
12
10
14
15
13
G PROOF OF THEOREM
Consider variables X,. Y, and M,, where Y is a placeholder notation for any variable other than Xj or Mj (Y could, for example, be any other X,, or My for j' * j). Variables X, and Y might be continuous or categorical. Variable M, is always continuous by construction. To prove the result in non-parametric setting, we treat any continuous variable levels (where n represents the number of samples in the data). We also assume the independent samples setting, where the rows of dataset X are independent.
categorical variable with distinct
G.1 PROOF OF STATEMENT 1, WHEN X, AND Y ARE CONTINUOUS
We consider first the case where X3 and Y are originally continuous variables. In this case, all variables are treated as categorical variables with n levels, where i = l.....,,,, 1,..., n, and . n indexes the categorical levels of variables X,. Y, and Mj. respectively.
Proof.
k = 1,....77
The conditional mutual information (CMJ) of X, and Y given Mj defined as,
7 T(X;; MM,)) a
P(X, = --=Y = M, =
P(X, -,Y
i =,M, = k)) log
P(X, = M, k) k) P(Y = 1IM, = k)
P((, -, kk) P(M
P(M =k)
-Σ :x,=i,Y = 1, N.MM-K. = =b)
P(X, = i, i, = P(Y = L, = k)
(11)
where the joint probabilities are defined from the joint frequency counts in a finite population. For instance, P(X, = Y = L, M, k) = f(x, = i, Y = M, = k)/n where f(X,- = i, I, M, k) corresponds to the number of instances in population of size n for which X, and Y == and M, = k. Note that because all three variables have n distinct levels, these frequencies will be either 0 depending
whether the combination of values is observed not in the dataset.
To avoid division by O, we consider the smoothed version of the CMI where an arbitrarily small constant e is added to the frequency counts of each cell in the contingency tables defining these probabilities. Namely,
, - 1, = =) = 1,M, =k) +
(12)
22

---

Published as a conference paper at ICLR 2026
and Pr (X, = i, L, k) is obtained by normalization as,
,,
P*(X; = i,Y = M, =k) Σ i=1 -
f(X, = i, = -1,M,
E 1= Σk=1 = Y ,1,, E Di=1 Ek=1
f(x, i, Y = 1, M2 = +c
+
(13)
n n
Similarly, we have that.
= i, f(X, = i, M, = k) +E
p =
(14)
n + 22
1, = k)) +e
PE(Y 1, =
(15)
n + 12E
f(M, = =k)
1+c n n11 + c) n
Pe (MM =
(16)
Now, because the mank-matching procedure used in the construction of M, implies perfect monotonic relation between the values of X, and M, we have that the discretized versions of X, and Mj will be identical (see Table for illustrative toy example). This correspondence between X, and M, implies that these variables cannot concomitantly assume different values at the same time (i.e., the joint frequency f(X; M, = k) will always be if of k). It also implies that the joint counts f(x, = i, M, = ke) will always be equal to the marginal counts f(M; k) when i = k. Hence, we have that,
f(M, k)) =1
if = k
s(X; -, i, = k)
(17)
0
if k
and we can re-express equation
14 as.
f(M, = k) +
1 1
if i=
P' = M, = =
n1 + n²E
+ n
(18)
i
+ n²
Similarly, we have that,
f(Y = L,,M, = =),
ifi=
= - =
(19)
0, if
and we can re-express equation 13 as.
= = 1,M, kk) +
ifi=k
(' = Y L,M, k)
n + a
(20)
ifi
n + n2
23

---

Published as a conference paper at ICLR 2026
Re-expressing the conditional mutual information in terms of the smoothed probabilities and separat- ing the summation over the cases where from the cases where it o we have.
(XXxY(M)) =
(21)
1,,, = = (M) =1) -Y,, -,,, = P) =
- -
log
- L,M, k) Pn PM, = k))
(X, = i, M, = k) log
p i, - P L, M, = k)
1=1 KHi
J(Y = 1, M2 = =1) =1)+c +
[(M, =1) +
J(Y = 1, M₁ =) - log
n + n
n + ne
n + n3c
f(M, = i) +
f(Y = M, = i) +E
H
n1 n2E
n n2e
f(M, kk)
n
n n c
n + me
log
n + n
S(Y = 1, M2 = k:) +
i=1
logfi
n + n²E +22
+ n²E
=) = 1, ii) +e log
nn (n+ n²e) (n + =nte)
+
Tt + niee
(n (n + nE)
(n n2+ + n² (MM, =k) +c)
log
n + n3e lighi
nn (n + (f( = M, = k) +c)
Taking the limit as e + we have that [X;;; (M,,) converges to,
n
n J(Y M, 1)
n n
[(; =k) F(Y = M, = k))
n log (1)+ + Σ Σ
log
=0
(22)
Observe that the term Olog(f(M, k)Yf(Y L, k)) in the above equation equals 0 because it can be rewritten as 0 flo = k) 0log f( = M, = k) and Olog fM =
k) = Olog(1/n) /n) = 0 and log [(Y = M, k) equals 0log 0 i i or 0 0 0 0 if I 1 k. (Note that this last case, we are adopting the convention that 0 log 0 = 0, as customary in information theory.)
G.2 PROOF OF STATEMENT 1, WHEN X, AND Y ARE CATEGORICAL
Proof. We now consider the case where X, and Y are categorical variables. Let b, and by represent the number of levels of X, and Y, and let i = 1,..., bz. I= I,...,b,, by. and k = I,... represent the indexes of the levels of variables Xy. Y. and M,. respectively.
Now, the CMI of X, and Y conditional of M, is given by,
((x;Y|MM) =
6y a
-1=Y I =k)
i - = 1:))) log
P(X, = il M, = P(Y =l|M, = R:)
P((x -i, = P(Z,
P(2 =k)
1,M, =b)log
= My k
(23)
24

---

Published as a conference paper at ICLR 2026
To avoid division by O, we again consider the smoothed version of the CMI with smoothed probabili- ties given by,
(X = =i =:a==ii--Li 1, = kb) = 2, 1, M, = +
(24)
Tu + bΦ by nE
f(X, M, k) +e
P=(X, = i, i, k)
(25)
n + b T1
P((Y = M, = k)) f(Y 1, M, k)) +
(26)
n1 + by TE
Pe (M, k)) f(M, = n ne
1 1 n(1+ + E) 7
(27)
By construction, when Xj is categorical, have that the following relations hold between X, and the discretized My.
f(x, = i, 1-) { f(M, k),
if
(28)
0,
if $ 1;
f(X, = i, = M, = k) { f(Y = 1, M, = =k), if I
(29)
0,
if 1
where J, represents the set of indexes of the discretized M, variable for which X, = and 1 (k: E 1.) represents the indicator function assuming value 1 when l belongs to I,, and otherwise. (Table provides illustrative example explaining the above relations.)
Using equations
28 and 29
we can re-express equations
(25) and as,
S(M, = k) +) +
if I if $ 1,
P5(X, = i, Mr =k)
n. + by ne
(30)
+ br ne
and,
f(Y= 1,M, = +)+E
if E T k
P'(X, = y 1,M, b)
b ne
(31)
11 + b b
25

---

Published as a conference paper at ICLR 2026
Re-expressing the conditional mutual information in terms of the smoothed probabilities and separat- ing the summation over the cases where E 1e from the cases where & I3 we have,
Ist (IX)) bx by
(32)
-ii== - M, L,,, -)
- = 1)) log
+
-i p P = -Y,,, = =k)
1=1 KEL
by
P* = =i,Y = 1, M2 = k) PS (M, k)
P = -, i, 1, = 1:) log
PX2
p = =, = kk Pc(Y = 1, 1,M, = k))
f(Y - L, = k)) +
f(M, k) + n + TE
J(Y = 1, M₁ = k)) te log
n + by by NE
+ by by ne
f(M, = k) +E
f(Y = M, = k) +
KEL.
by ne
+ by
f(M, = +c
by
ni + by by n
n + ne
log
11 - by by ne
C 12 + n
f(Y = 1, M, = 16) +
1=1 Ko
n by NE
by
f(Y L, M₁ -k)+ +e
( (n (n+ ne)(n + by ne)
by
n + by by ne
log
(n+ by by n ) + nt))
+
kEl.
by
(n b2 by n (n by n (f(, = +
n by by ne log
(n by by n e) (
=1, =,, = 1b) +c)
Taking the limit as + 0 we have that (xx5; M1M))
converges to.
by
bz by
f(Y 1, M₁ =
n log (1)
ΣΣ>>>>> Olog
f(Mj k) J(Y = I, k))
=0. (33)
1=1 kdT
G.3 PROOF OF STATEMENT 1, IN THE REMAINING CASES
Proof. In the case where X, is continuous and is categorical, a similar argument to the proof in the case that both variables are continuous shows that,
T =
(34)
f(Y = M, = 1) +e log n + by n²E
(n + n² c) (n + nE)
(n+by n² E) (n + nt).
by
= kc) (n In+n² (n + by (e)
log
(n + by n2 e)(n ne) -1,M, =) -k, +e)
by n²
which again converges to 0 - 0.
In the case that Xj is categorical and Y is continuous, a similar argument to the proof in the case that both variables are categorical shows that,
1 ['X,; ((M,)
(35)
b.
f(Y = M, = k) +E log
( (n (n+ (n) + n² c)
+
n + by 1i=E
(n+ by n2 c)(n + ne)
1= kel:
(n + by ne) n² ()( = k) + +c)
log
n + br n²
(n + by n² e) + ( =l, = k) +
k
26

---

Published as a conference paper at ICLR 2026
which again converges to O.
G.4 PROOF OF STATEMENT 2
Proof. By definition, the conditional entropy of X, given Mj is given by,
P(X; Mj = =k] P(M, k)
H(X, M2) ΣΣ P(X, = i, k) log
PX2 = M, P) k) log P(X; Mj k))
(36)
In the case where X, is continuous (and discretized into classes) have that equation 36 reduces to,
H(X, M2) Σ
P(X, = Mj i),(M log P(X; il M, = i)
i=1
+ Σ
p(x,= = | = k) = k)) log P(X, = Mj = k)
(37)
1=1 kz
after we split the summation over into the cases where and k + i. Since, by construction, have that,
1,
if
P(X; = Mj =
(38)
0 if
if follows that,
n Σ P(M, = i) log + Σ Σ
H(X; M2)
OP(M, k:) log
(39)
Aigh
where we take 0 log 0 to be defined as 0 (as its is customary in information theory).
Similarly, in the case where X, is categorical (with classes) equation (36) reduces to,
by
H(X, M2) =
P(X, = M2 = P)M(M = kb) P(X,, il M2 = k)
1=1 KE
Σ
Σ
k)P(M
p(x; = Mj = =k),
(40)
i1 kggg
after split the summation over k into the cases where k E I; and & $ I,. Since, by construction, we also have that in the categorical case,
P(X, = il i M2 k)) = {
: if
1. i 1
(41)
follows again that,
6
b
H(X, M2) = Σ Σ
P(M, = 1) log 1 +
Σ
OP(M, = log 0 = 0.
(42)
i=1 kel
Ag
27

---

Published as a conference paper at ICLR 2026
Table 2: Illustrative toy example showing the generation of M, using Algorithm T when X, is
continuous. The sample x, = (0.92. 1.29, 0.34, -0.93, 0.71, 0.39, 0.65, 0.85, 0.84. I.38, -0.32)* (with n 11) represents observations from X,. Rows (i) and (ii) display the original values of X, and their corresponding ranks. Row (iii) shows the generated variable Mj. which is uniformly distributed and rank-matched to X, (i.e., the ordering of M, identical to that of X,). Row (iv)
confirms this by showing that the ranks of M, match those of row (ii). Because of this perfect monotonic correspondence, the discretized versions of Xj and M, are identical for any chosen number of bins. Rows (v) and (vi) illustrate this fact by presenting the discretizations of X, and Mj based on n bins
():
ORIGINAL X,
0.92
-1.29
0.34
-0.93
0.71
0.39
0.65
0.85
0.84
1.38
-0,32
(I):
RANKS X,
0IC:
ORIGINAL M2
0.69
0.12
0.41
0.20
0.61
0.44
0.
0.67
0.62
0.74
0.39
(VV):
RANKS OF M
10
1
2
7
5
8
11
(V):
X, (DISCRETIZED)
10
1
2
7
5
8
11
3
(VI)
M, (DISCRETIZED)
Table 3: Toy illustrative example of the generation of My using Algorithms I and 10 the case where X, is categorical. Let x, = (A, A, A, A, B, B, B,C,C. C, C*2² represent sample of size n = 11 from the categorical variable X, which has distinct levels, (A, B, C). Let 1,2, 3 represent the indexes of the levels of X,. Row (i) shows the data in its original format, while row (ii) shows the same data in terms of the level indexes representation, where level
indexed as I, level as 2, and level C' as 3. Row (iii) shows the output, r, of Algorithm TO which generates a numeric rank encoding of the categorical variable Xy. (It adopts the arbitrary order A < B < to the categorical levels of X, and starting with class A, it assigns ranks 1, 2, 3, and 4 (in random order) to the four tied elements in Xj. For class B, it assigns ranks 5, 6, and 7 (in random order) to the three tied elements B in Xj. Finally, for class C it assigns ranks 8 to 11 (in random order) to the four tied elements C.) Row (iv) shows the output of Algorithm]] [ It first sample and sort 11 values from uniform distribution in the [0, range (line 2) given, in this example, by (0.12 0.20, 0.39, 0.41, 0.44, 0.50, 0.61, 0.62, 0.67, 0.69, 0.74) and then rank match this sorted random noise vector to the numeric rank encodings show in line (iii). The result is a random noise vector with identical ranks as r. Row (v) shows Mj after discretization into = 11 levels (in terms of level indexes representation). Clearly, when we discretize Mj into n levels we recover the numerical rank encoding in row (iii). Now, let 7 represent the indexes of the M, values for which X; i. In this example 7, 2, 4). 12 (5, 6, 7). and /3 = {8, 9, 10, 11]. Note that, by construction, whenever M, cquals 2, 3, or 4, the value of X, will be 1. This implies that f(X;
1, M, = k) f(M; k) if 21, 3, 4}. and
f(x; 1, M, M = k) = 0 if & & {1, 21,2,3 3, 4)4 Similarly, f(X, 2, M, k) f(M, = k)
k (5, 6, 7), and f(x; 2, M, k)) if k & (5. 6, 7), and so on for other values of i. Clearly,
this result holds in general, as stated in equation 28 A similar argument justifies equation 29
())
ORIGINAL X, X, LEVEL INDEXES
A
B
C
C
(I):
1
1
1
2
2
3
3
3
(II)
NUMERIC RANK ENCODING
3
7
8
(IV):
M, (CONTINUOUS)
0.39
0.12
0.41
0.20
0.61
0.44
0.50
0.67
0.62
0.74
0.69
(V):
M, (DISCRETIZED)
3
2
7
5
9
8
11
10
(VI):
I. MAPPING
I
(1,2,3,4 4).
12
(5.6, 6.7).
Ja
(8,9,
11)
28

---

Published as a conference paper at ICLR 2026
H COMPLEXITY ANALYSIS AND RUNTIME EXPERIMENTS
H.1 COMPLEXITY ANALYSIS
As pointed in Hollmann et al. (2025) the complexity of the TabPFN algorithm scales quadratically with the number of samples (n) and the number of features (p), i.e., On + p²))
The generation of synthetic data with the MIAV strategy (Algorithm 7 involves calling Algorithm two times (see lines 3 and 4). Algorithm(6 6 by its turn, involves calls to TabPFN models trained with a single feature (see the for loop in lines 4 to 8 of Algorithm 6). Since the complexity of a
TabPNF model trained with single feature is O(n² + 1) O(n²), it follows that the complexity of Algorithm is O(p ) Consequently, the complexity of the MIAV strategy (Algorithm) 7 is p(2 O(p
The FC approach (Algorithm 5). on the other hand, has complexity O(pn² + +p"), since Algorithm involves p calls TabPFN models with p 1 features, that,
Opp(n² + (p- = p²)
(43)
Similarly, the JF approach (Algorithm also has complexity Opp p²), since Algorithm involves p calls to TabPFN models with number of features increasing from to p - whose total complexity is given by,
+0 (p- 1)p(pp (12-1))(20 1)
O(n² + = -0
Σ n2 End
+0
Σ
12 h22
=
k=1
= O(pn²) + 0(p') 0ppn²+p')
(44)
These analyses show that, for fixed n, the MIAV strategy scales linearly with increasing number of variables, whereas the FC and JF approaches scale cubically with increasing number of variables.
H.2 RUNTIME EXPERIMENTS
We performed 7 runtime experiments comparing the MIAV, JF, and FC synthetic data generation strategies. The experiments were based on simulated datasets containing 1000, 2000, 3000, 4000, 5000, 10000, or 20000 samples (rows) with number of variables (columns) varying from 10 to 100, in increments of 10. (The data was simulated from correlated beta distributions as described in Appendix
Uniform(O, 10), and = 0.5.) In each experiment, the runtime was measured with the perf_ _eff__oouter_rr () function from the t time Python module. Results are reported in seconds and based on replications of each experiment. All experiments were performed on an AWS EC2 g5 xlarge instance with NVIDIA A10G GPU (24 GiB), 4 vCPUs, 16 GiB RAM, 250 GB NVMe SSD, and up to 10 Gbps network bandwidth.
C using a ~ Uniform(O, 10),
Figure reports the runtime results for number of rows (n) increasing from 1000 to 5000, while Figure 2 reports results for n equal to 5000, 10000, and 20000. As expected, both figures show that, as the number of features increases, we observe considerably longer runtimes for FC (blue curves) and JF (orange curves) when compared to the MIAV (red curves). The figures also illustrate that the runtimes of the FC and JF strategies scale cubically with increasing number of features, whereas the runtime of the MIAV approach scales linearly. Panel f in Figure [1 and panel d in Figure 12] report the results for the MIAV approach for the distinct sample sizes side by side. (The red lines represent the median values across the 5 replications.)
"Note that the complexity of the other computations in Algorithm aare dominated by the TabPEN compu- tation. Explicitly, the MIAV generation (line 5 and line 6 of Algorithm 6b has complexity O(n log n.) since, as shown in Algorithm T it: (i) involves sorting of random noise vector (line 2 of Algorithm I. which has complexity (n log n.) (since random noise generation of vector of length n has complexity (n) and it's Algorithm I. which has complexity (n log n): and (iii) involves a call to Algorithm (line 7 of Algorithm I which has complexity (n) (as both the CummulativeSum operation (line 4 of Algorithm 10) has complexity (nz) and the repeated applications of the RandomPermutation operation (line 12 of Algorithm T0) inside the for loop also have total complexity ((n)).
sorting has complexity On log ni)): (ii) involves the ranking a vector of length n (line
29

---

Published as a conference paper at ICLR 2026
1300
(a)
=1,000
3300
(b)
=20000
1500
[c]
3,000
FC
FC
Fundrra Sac3l
1000
rundima (ee sec4)
1000
MIAV
rundrra s0633
1000
500
10
30
50
70
90
10
30
50
70
90
10
30
50
70
90
(d)
= 4,000
(e)
n = 5,000
MIAV
1500
1500
FC
FC
170
-550000
IF
m4000
nundine MMCS)
3000
MIAV
nundinwe (in sacs)
2000
MIAV
==0000
3,000
rundenne fin
60
300
40
70
10
50
70
10
50
70
90
60
80
100
features
fealures
Figure 11: Runtime experiments. Panels b, and report runtime benchmark results comparing the FC, JF, and MIAV synthetic data generation strategies for datasets containing 1,000, 2,000, 3. 3,000, 4,000 and 5,000 samples, respectively. Panel f compares the MIAV results across the different sample sizes.
n = 5,000
[b)
10,000
[c]
20,000
MIAV
8000
FC
8000
FC
8000
FC
20,000
400
10.000
nurtine in sec3)
5000
nurtine (In secc
5000
nurlimm (In vec
6000
wumm lin ware (ooa
5,000
4000
092
2000
2000
2000
100
10
50
90
10
50
90
10
50
90
80
100
number of footures
of footures
of footures
number of features
Figure 12: Runtime experiments. Panels a, b, and report runtime benchmark results comparing the FC, JF, and MIAV synthetic data generation strategies for datasets containing 5,000, 10,000, and 20,000 samples, respectively. Panel d compares the MIAV results across the different sample sizes.
Note that the computation of the JF approach we use a single variable order for the factorization of the joint posterior predictive distribution in all experiments in this paper. Hollmann et al. (2025) suggest using a permutation sampling approximation Janossy pooling for combining results across different permutations of the order of the variables. (This is done account for the fact that the variable order influences the quality of the synthesized data.) Implementation of this strategy would, however, increase the computation time by factor of k where represents the number of variable order permutations used by the approximation. (Hollmann et al. adopted & = 24 in their experiments.)
I EXTENDED EXPERIMENTS SECTION
1.1 OVERVIEW OF DATA SPLITTING INTO ORIGINAL, HOLDOUT, TRAINING, AND TEST SETS
Our experiments are performed as follows. Each dataset D is first split into two datasets X and Xh denoted as the "original" and the "holdout" datasets, respectively. In our evaluations, the original dataset, X, plays the role of the "real" data, from which we generate a synthetic data copy. (We denote it the "original" data because D might be real-world dataset simulated dataset in our experiments.) The holdout dataset, X is used to "estimate" the performance of an ideal data generator capable of generating data from the same distribution the original data. In
30

---

Published as a conference paper at ICLR 2026
experiments, in addition to evaluating the synthetic datasets using the metrics described in Section L.3 below, we also compute the metric values for the holdout set in order to get a sense about the range of values we would expect to see for the metric in the ideal case of generator truly able to draw independent samples from the same distribution as the original data.
For the generation of synthetic data, based on ICL using the TabPFN- RabPeN-based strategies described in the main test, namely, JF, FC, and MIAV, we further split the original data, X, into two subsets, X, and
X2. As described in Algorithms 3 and 7 in Section B the synthetic data generation is performed as follows. First, the algorithms generate a synthetic copy of X1, using X2 as the training set and X as the test set. Second, the algorithms generate synthetic copy of X2, by using X1 as the training set and X2 as the test set. Third, the algorithms concatenate the datasets generated in the previous steps to obtain the full synthetic dataset copy of X.
For the generation of synthetic data using SMOTE (and other baselines), the original data X is directly fed into the synthesizer with no need for further data splits.
1.2 SIMULATED DATA EXPERIMENTS
For the simulated data experiments we evaluate the synthetic data generators over 5 distinct settings spanning different correlation structures among the variables. In all settings, we generate datasets with 5 variables from correlated beta distributions described in Section C The correlation structure (Toeplitz) is controlled by single parameter p. and in our experiments we adopt correlation strengths, (p]. in the range |pl {0,0.25, 0.25, 0.5, 0.75, 0.95).
For each experimental setting (i.e., |pl value) we simulate 10 distinct datasets, D, of size 800, using Toeplitz correlation parameter randomly set to either P or -p with equal probability, and adopting different shape parameters, a and b, for the beta distributions. For each variable X, ~ Beta(a, b), the and 6 parameters are randomly sampled from uniform distributions follows:
U00.1, (0.11, for X1.
a U(1, for X2.
a
for X3.
a U(5, (5), 1- (0,.bb~~5511111 15), for X4. a U(1 10), b (5, 15), for X5.
Each dataset D is split into original (X) and holdout (X*) datasets of size 400. Only the X datasets are used by the synthetic data generators.
1.3 REAL-WORLD DATA EXPERIMENTS
For the real-worke data experiments selected subset of the datasets from the OpenML-CC18 benchmark suite (Bischl et al., 2021), which were analyzed in Hollmann et al. (2023, 2025). The OpenML-CC18 suite contains 72 datasets but, similarly Hollmann et al. (2025), we selected datasets with at most 10,000 rows (cxamples), at most 500 columns (features), and containing categorical variables with at most 10 classes. Furthermore, because most of the evaluation metrics used in this paper are tailored to numeric data, we applied the additional filter that the datasets needed to contain more numerical variables than categorical ones. After applying these filters to the 72 datasets in OpenML-CC18 we were left with the 36 datasets listed in Table
We also compared the proposed synthetic data generation strategies against popular synthetic data generators using a subset of the datasets evaluated in Hansen et al. (2023) and Chaibub Neto (2025) listed on Table [5] Note that while Table 5 contains datasets with more than 10,000 samples, we still able to fit TabPFN models because in our evaluations we split the datasets into original and holdout sets, and the synthetic data generation is only applied to the original set (which, by its turn, is further split in subsets during the ICL learning step). We allowed for larger datasets in these baseline comparison experiments because the deep-learnigg based baselines benefit from larger datasets. (We constrained the maximum number of samples to be less than 20,000, however, because the computation of the evaluation metrics became too slow for larger datasets.)
Similarly Chaibub Neto (2025), the Abalone data fetched using the commands:
31

---

Published as a conference paper at ICLR 2026
Table 4: Datasets used in the real-world data evaluations. These include all 36 datasets in the OpenMI.- CC18 benchmark suite with at most 10,000 samples, 500 variables, 10 classes per categorical variables, and larger number of numeric variables than categorical ones. the first column assign simplified dataset names (D1 to D36) to the original dataset names. The number of categorical variables is abbreviated as #cat, and the number of classes of the categorical variable with most classes is abbreviated as #class.
NAME
ORIGINAL DATASET ASAMPLES
ACOLUMNS
#CAT
ACLASS
OPENML
D1
D2
MFEAT FACTORS
2000
217
10
12
D4
BREAST-W
699
10
15
D5
MPEAT- KARHUNEN
2000
10
D6
2000
10
D7
MUEAT-ZERNIKE
2000
48
10
22
D9
DIABETES
768
37
DII
VEHICLE
846
19
53
D12
SATIMAGE
6430
37
2074
DI3
ANALCANDATA-AUTHORSSIII
841
3549
D14
PC4
1458
38
3902
D16
KC2
522
22
3913
2109
22
DI8
PC1
1109
22
3918
D19
WDBC
569
9946
D20
5404
D21
OSAR-BIODEG
1055
42
9957
NOVOT-NAVIGATION
D23
SEMEION
1593
257
10
9964
ILPD
11
9971
D25
OZONE-LEVEL-SIEE
2534
73
9978
D26
FIRS- ORDER THEOREM PROVING
6118
52
9935
D27
CORNOOE--UTTEETICR AUTHENTICATION
1372
D28
BLOOD TRANSFUSION SERVICE CENTER
748
10101
GESTURREPAARSSSEMNNATTTTOOO
D30
MICEPROTEIN
1080
78
146800
D31
STEEL-PLATES-PAUS CLIMATE HAMATE-MODEL--TEE MODEL-SIMULATTIN CRASHES
1941
146817
D32
540
19
146819
D33
WILT
4839
146820
D34
2310
17
146822
D35
MFEAT PIXEL
2000
241
10
146824
CHURN
Table Datasets used in the baseline comparison experiments. See Table for the column descrip- tions.
ABALONE
4177
9
2
BM
BANK MARKETING
10578
8
44126
CREDIT
16714
44089
EM
EYE MOVEMENTS
7608
21
44130 2
HOUSE
13488
44123
MT
MAGIC TELESCOPE
13376
2
44125
10082
44122
from celearn.dattsett import fetch petch_openmi
fetch_openml "abaloe,,,,,"" version-1, as_frame= True)
The other datasets were fetched using:
openml www.daaaaae...mmmm wtwpseseetssee__ttttsseeeeeeeeee (openmlid)
32

---

Published as a conference paper at ICLR 2026
1.4 HYPERPARAMETERS FOR THE BASELINE MODELS
Table Hyperparameters used for the generative models trained on the Abalone (AB) dataset.
Model
RLJoer: 7615
DDPM
Ir: (102999978122701118
batch size: 970
timestege:
1s. classification: Fubse
50
max cnrly stpp: False min rode sine: a_ Ir. 0.001
TYAE
decoder_n Jayyerr_____eess hidden:
weight_de.op 0.0001
batch_size: 128
L.uniis_embeddingg 200
decoder naiss,_____aaee hudden: hudden 150
decoder_naalin: tanb
deander dropont 11994111881888166 encoder_n Jtyyerrs___yeess________ hidden:
encoder_n_miis__ee hidden: 100
encoder
encodor dropout: 0.060025531122244
Mer: 700
CTGAN
www.er__a___aa____ee hidden:
pencrator hidden: 100
generator, ponlin: elc
generaloe_drupun 0.1 1336612599477665
diseriminater
hidden:
discriminate_ wuwiiss,______ii_oo__oa_e hiddes:
discriminatie, monlin: lanh
diserimnntee n. Ser
discriminacr_drpppe 0.22366569996555777
deipbr_decay: 0,0001
batch, size: 200
encoder, clussers: 8
Table Hyperparameters used for the generative models trained Bank marketing (BM), Credit (CR), Eye movements (EM), House 16H (HO), Magic telescope (MT), and Pol (PO) datasets.
Hirr: 1051
DOPM
8-80099778885422886677
mum timesteps: 998 classification: Trme
max Iten:
_troy Trae
NLjeer:
TVAE
In: 0.0002
decoder_jt Baaeeerr___lles__sae_a hiddes:
weight decay:
butch size:
n_umiis, embedding: 200
decoder uniis hidden: 300 nonlin:
decoder, dropout: 0.19122119911226
envoder n. layers hidden: I
hidden: 450
Laxoder_nonlin: kaak
envoder, dropout: 00423853700004718
iter 1000
CTGAN
hidden
genersice umite hädde: 50 generatoe nonlin: tach generator dropout 0.0575
discriminstor Joyers hidden: discriminstor
discriminato, noolin: relu
discriminato iten
discsiminttr drepout: 0.1
weight decay: 0.001
hatch, size: 200
33

---

Published as a conference paper at ICLR 2026
1.5 EVALUATION METRICS
We evaluated the quality of the synthetic data in terms of data utility, data fidelity, and data privacy.
Data utility was evaluated using the machine learning efficiency (MLE) metric for measuring utility in downstream predictive tasks. MLE was computed using a random forest learner trained in the synthetic data and evaluated in the holdout set (which corresponds real data). (To measure the ground truth performance, we train the random forest learner on the original data and evaluate its performance on the holdout set.) The predictive performance was measured using AUROC in classification tasks and R² in regression tasks. Hence, larger values of MLE imply better utility of the synthetic data in downstream ML tasks.
Data fidelity was evaluated with respect to the quality of the marginal distributions, quality of the pairwise statistical associations, overall data quality, and quality of the joint distribution according to the following metrics:
Average KS-statistic (KS). This metric is used to evaluate the quality of the synthetic data marginal distributions. It is based on the Kolmogorov- test (KS-test) for the equality of distributions. For each variable it computes the KS-test statistic between the synthetic and original data, and the metric corresponds to the average KSSstatistic across all variables. Lower values of this metric indicate better agreement between the synthetic and original marginal distributions.
two-sample statistical
L2 distance between association matrices (L22D.. This metric is used to evaluate how well the synthetic data recovers the pairwise statistical associations observed in the original data. The L2 distance is computed as the average of the squared difference between the elements of the synthetic and original data association matrices. Lower values of this metric indicate better agreement between the synthetic and original data pairwise statistical associations. Since the datasets might include both numerical and categorical variables, we assess pairwise associations as follows: for numerical pairs we use Pearson correlation; for categorical pairs we use the Cramer's 's V statistic; for numeric/categorical pairs we regress the numeric variable on the categorical one and use the square root of the It2 statistic as our association measure (this reduces to the absolute correlation coefficient when both variables are numerical).
Wasserstein distance (WD). The WD metric, also known as the Earth Mover's Distance, quantifies the dissimilarity between two probability distributions by measuring the minimum "cost" of transporting probability mass from one distribution to the other. It is used to evaluate how well the joint probability distribution of the synthetic data approximates the joint distribution of the original data. Lower values of this metric indicate better agreement between the synthetic and original joint probability distributions. Because we adopt the squared Euclidean distance cost matrix for the computation of the transport cost, this metric is sensitive to scale (i.e., variables with larger numerical ranges dominate the distance). Hence, we first re-scale all variables before computing the WD.
Energy distance (ED). This metric is also used to evaluate how well the joint probability distribution of the synthetic data approximates the joint distribution of the original data. Again, lower values of this metric indicate better agreement between the synthetic and origi- nal joint probability distributions. (Energy distance (Szekely and Rizzo, 2023) represents special case of the maximum mean discrepancy statistic.) Since ED uses Fuclidean norms to compare observations, it is also sensitive to scale (i.c., variables with larger numerical ranges dominate the distance). Hence, we first re-scale all variables before computing the ED.
Detection test (DT), This metrie measures the overall quality of the synthetic data by evaluating the performance of classifier trained to discriminate between synthetic and original data examples. When the synthetic data is indistinguishable from the original data the classifier should achieve a random guess performance level, otherwise the classifier performance should be better than random. In our experiments, we use a random forest classifier and evaluate classification performance using AUROC. Values closer to AUROC= 0.5 indicate better agreement between the synthetic and original data.
Data privacy was evaluated according to the following metrics:
34

---

Published as a conference paper at ICLR 2026
Distance to closest record (DCR). This represents record-level privacy metric that mea- sures how similar a synthetie record is to the closest record in the original dataset. It is computed by measuring, for each synthetic record, the minimum distance to any record in the original dataset using Fuclidean distance; values close to zero indicate higher dis- closure risk because synthetic records nearly indistinguishable from the original data ones, whereas larger values suggest safer privacy protection by ensuring greater separation between synthetic and real data (this might be achieved the cost of lower data fidelity, though). Since Euclidean distance is sensitive to scale (i.e., variables with larger numerical ranges dominate the distance), we first re-scale all variables before computing the DCR.
Sorted distance-based record linkage (SDBRL). This metric represents variant of the Sastace--Bssed Record Linkage (DBRL) metric. The DBRL metric (Pagliuca and Seri, 1999; Domingo-Ferrer and Torra, 2001) is widely used method for evaluating re-identification risk of perturbation methods within the Statistical Disclosure Control field (Drechsler, 2011 1). It operates by computing the Euclidean distance between each record in the perturbed dataset and all records in the original dataset, designating perturbed record as 'linked' when its nearest neighbor corresponds to its true original record. The DBRL, value is then given by the proportion of perturbed records successfully linked back to their original counterparts. Strictly speaking. the DBRL metric is only intended for evaluating data perturbation methods, since it assumes the existence of direct mapping between the original and perturbed values (for example, when perturbed data are obtained by adding noise to the original data). In the case of synthetic data, where such mapping is absent, approximate correspondence can still be established following the approach of Domingo-Feri Ferrer et al. (2020) and Chaibub Neto (2024, 2025). The idea is sort the rows of both the original and synthetic datasets by the values of chosen attribute (column) and then compute the metric on these sorted al.
datasets. (A rationale for this procedure given in section 3 of Domingo-Ferrer (2020).) This adapted version of the DBRL metric is referred to as the "sorted DBRL" metric, or SDBRL for short.
Sorted standard deviation interval distance (SSDID). This metric represents a variant of the Standard Deviation Interval Distance (SDID) metric. The SDID metric Mateo-San -anz et al. (2004) is commonly used method for evaluating attribute disclosure risk of perturbation methods in the Statistical Disclosure Control literature. It corresponds to the proportion of original records inside standard deviation interval whose center is the corresponding perturbed record (where the interval width is computed in terms of a percentage p of the standard deviation of the variable). A record in the original dataset considered to be inside the standard deviation interval of the perturbed record if, for all variables j. it is inside the respective standard deviation interval. Similarly to DBRL,, the SDID metric also assumes the existence of a mapping between the original and perturbed values, and we adopted the sorted version of this metric (SSDID) proposed by Chaibub Neto (2024, 2025) in our synthetic data evaluations.
In our evaluations, we only compute the KS, WD, ED, DCR, SDBRL, and SSDID metrics for numeric variables. (Note that simulated datasets contain only numeric variables and, as shown in Table 4 the real-world datasets contained mostly numeric variables as well.)
1.6 RESULTS FOR ADDITIONAL EVALLATION METRICS
In the main text, we present evaluation results for three fidelity metries (namely, KS, L2D, and DT) and three privacy metrics (namely, DCR, SDBRL, and SSDID). Here, we present additional evaluation results for the: machine learning efficiency (MLE) metric, which measures data utility in downstream predictive tasks; and the Wasserstein distance and the energy distance metrics, both of which measure data fidelity by evaluating how well the joint probability distribution of the synthetic data approximates the joint distribution of the original data.
Figure 13 report the results (which were computed in the exact same data splits and synthetic datasets as the previous evaluation metrics). Panels a, b. and show the results from the experiments based on simulated data draw from correlated beta distributions. Panels d, e, and f report the results for the 36 real-world datasets from the OpenML-CC18 benchmark suite described in Table Panels g, h, and i show the results for the 7 additional datasets evaluated in Hansen et al. (2023) and Chaibub Neto (2025) described in Table 5
35

---

Published as a conference paper at ICLR 2026
88 MLE (simul.)
[b] WD (simul.)
[C) ED (simul.)
044
0.4
F
0.3
0.2
01-
0.0
0.--
0.0
Telturn
[0]
(CC18)
(CC18)
0.7
0.6
05
1..6
04
Erangr
0..
ML
MLE (base.)
WD (base.)
i
ED (base.)
1.0
12
0.8
ME (3/RCC
0.8
0.6
0.0
0.4
FC Ik
0.
ao
FC
DD OEGAN TIWE
my-
MAV
SMOTe
CTENN
TWNE
MAy
FC
SHOTE BDeM
"WMe
Alt
Bnu-
Figure 13: Pooled experimental results. Top panels show results pooled across the 5 simulated dataset settings. The middle panels show results pooled across the 36 real-world datasets selected from the OpenML-CC18 suite. The bottom panels show results pooled across the real-world datasets used for the baseline generator comparisons. For the MLE metric, higher values indicate better data utility (MLE is measured by AUROC in classification tasks and R in regression tasks). For the WD and ED metrics, lower values indicate better fidelity.
In terms of MLE (panels a, d, and g), where larger values indicate better utility, the MIAV approach (red boxplots) performs consistently well across all experiments, closely tracking the ground truth performance, reported by the holdout set results (green boxplots). Overall, most baselines tended to show comparable performances, with the exception of the JF and the bayesian-network (BN) generators in panel (and the FC generator in panel a), which tended to underperform when compared to the other generators.
In terms of the WD and ED fidelity metrics, where lower values indicate better fidelity, the SMOTE and MIAV approaches tended to outperform the JF and FC methods in all experiments. Interestingly, the comparisons of WD and ED across the other baselines (panels h and i) show some surprising results. particular, the bayesian- network (BN) generator, which achieved the worse data utility performance measured by MLE (panel g) as well the worse data fidelity performances in terms of the 1.2D and DT metrics (see panels and o in Figure 4 in the main text), tended to produce surprisingly strong results in terms of WD and ED. The DDPM baseline, weak performance in terms of WD and ED despite ranking among the strongest baselines in all other data utility and data fidelity metries. These observations corroborate similar surprising results reported by Hansen et al. (2023), which also observed better fidelity scores for eeyesian-netwone than for the DDPM model in their experiments. These findings provide yet another example of how distinct performance metrics do not always generate consistent conclusions, and underscore the need for evaluations based on multiple metries.
the other hand, achieved
36

---

Published as a conference paper at ICLR 2026
1.7 EXTENDED RESULTS
Due to space limitations, in the main text we only present experimental results pooled across all datasets. Here provide more detailed results. Figure 14 presents the results from the simulated data experiments separated by simulation setting. Figures [5] 16 17 and [8 present separated results for each of the OpenML-CCII real-world datasets, Figure presents separated results for each of the real-world datasets in Table 5
in 11.0.55
120, p-104
P
R Id11
SD3RI, P
b1)
[ell
117-
804
L20,
508RL, p.w0.7
0.22 0221
3012
308
120. Rmax
DT. 12-03
DCR, p p-033
SEBAL 2-15
25013, p-23
0.35-00
(:3)
[-3]
[9)
[24]
(4)
(04)
[641
64]
308
2026
$115
2000
vo
LID.
SCORL
5-010,
[051
1051
802-
21-
800
Figure 14: Simulated data experiments separated by simulation setting. Each boxplot displays the results from 10 separate replications based different simulation parameters.
37

---

Published as a conference paper at ICLR 2026
50831.0
2.1
616
8006
L,0.022
DCB.
5083L.02
$501b.0
00011
1155
(4)
[44
[44]
0.0m
0.34
[a51
[051
E51
305-
2404
3000
802
[16]
(6)
(081
+
805-
(07)
6771
CK-8o7)
H 11
2000
COBBS,
SSDID
6.35 1a81
A081
6.7
16)
6.3
COAL
6.31
(391
(00)
(20)
Td91
69)
110)
828
F
2000
2-10-
2006
000
Figure 15: Experiment results for datasets D1 to D9 (see Table for the original dataset names).
4
Each boxplot displays the results from distinct original/holdout data splits.
38

---

Published as a conference paper at ICLR 2026
M0CH
800
D11
5088L
55D10,
6415
8311
[44
8:5-
144
200
2.13-
305
0.00
1051
614-115
612
III
[86]
(66
1681
[6]
1111
300e-
(o7)
677
[+7]
(4-)))
[[[
4
S08R1,
E5DID
(A81
(080
(8)
Y081
[39]
615
(00)
220)
[60
19)
2017
Bas
H
Figure 16: Experiment results for datasets D10 to D18 (see Table for the original dataset names).
Each boxplot displays the results from distinct original/holdout data splits.
39

---

Published as a conference paper at ICLR 2026
D12
6000
Q
2045
C
D20
02.028
55D10,
6021
2645 (12
0323
Sett
63)
825-
0.0
006-
644
0.10
0323
g002
6305
T851
6.8
[061
(06)
(08
811-666
511
2000
[A71
(0))
(671
[o7]
2.05
BCB
S08R1,
S50ID
(681
(08)
3411-
Box
0.0
(39]
(00)
[20)
[60
60)
111
Figure 17: Experiment results for datasets D19 to D27 (see Table for the original dataset names).
Each boxplot displays the results from distinct original/holdout data splits.
40

---

Published as a conference paper at ICLR 2026
DCK,02
R0.55
K5,
02.02
5088L,00
55D1D,
621
T
800
g014-
[34)
17-144]
0.10
6-15-
004-
[851
204
011
800
1061
(081
306
c6]
20.33
(07)
(671
[07]
(7)
5.15
b35
BCB.
E5DIb
(681
(60
6.6 681
$50t0
[39]
(00)
[60]
(0)
1013
308
F 1
3033
2415
Figure 18: Experiment results for datasets D28 to D36 (see Table for the original dataset names).
Each boxplot displays the results from distinct original/holdout data splits.
41

---

Published as a conference paper at ICLR 2026
K5,A8
DT.A8
DCR,
SOONL,
SIDID,AS
amrJall
[b1)
Id11
[ell
0-
"Tap
SDBRL, BH
EspiD BM
[22)
(2)
308-
302
300
Mon
moae
Ic
S0BAL CR
S2D10
A33-4031
631
111-
+
6.00
moas
1-
ourtt
SDBRL, EM
SSpiD, BM
[94]
104
017-444)
5 1
3000
+1
2000
SOBRI,
[05]
200
S08AL,MT
190
0314
0317
12
F
Saitc
bumt
Clea
SSDID
8.5
12
CIDAY
DEPE
DeP
Figure 19: Experiment results for datasets in Table distinct original/holdout data splits.
S
Each boxplot displays the results from 10
42

---

Published as a conference paper at ICLR 2026
J THE NOISY-MIAV STRATEGY
simple way to improve the privacy of the MIAV synthetic data generation approach is to add controlled amounts of noise to the MIAV before generating the synthetic data with the TabPFN model. The details are presented in A and As shown in lines 7 and 9 of Algorithm [1 we generate noisy versions of the MIAV by adding Gaussian noise with mean 0 and standard deviance equal to a fixed percentage of the standard deviation of the respective MIAV. By increasing the percent of noise parameter we generate increasingly noisier versions of the MIAV.
Algorithm 11 (CLwithNooisMIIATAAPPPAAA
X percent)
1: Input: training data for ICL, X query data for ICL, X
2: NumberOfRows X (Ohiain number nows of xx*.)
3: p < OfCllrOfsolummss (X*)) [Ohtain number of culumms of (.)
4: Z < [] (Create empty matrix to store the synthetic data.)
5: for j to do
6: m
[Generate MIAV for xe"" using Algo-
rithm
7:
m t
my +
n,, mean 0, sd
percent sd(m))) (Add
Gaussian noise the MIAV aecording specified percent of the MTAV's standard deviation.]
8:
CeneratMMxiimalnnfomttiooonnnni variiiiraeeiiillllaaaaallleee Variable(x") (x)) [Generate the MIAV for x3" using Algo-
9:
m f m + GenerateNormmllaaribbal (size = n,, mean = 0, sd = percent * sd(m)))) (Add
Gaussian noise the MIAV according specified percent of the MIAV's standard devintion.)
10:
['] t GeneratePrediction CaarrrtoonnnnnnnnaaPNaeiii
m xx(") (Predict x using mn and xff as context,
mf as query. prediction can be from a regression classification TabPFN model, depending on whether x5" continuuus or categorical.)
11: end for
12: Output: synthetic data *
Algorithm 12 NoiiyMMLAVT.oPPrGeeeaaotto percent)
1: Input: the original data, X
2: X X, < DataSplit(X X) [Split the original data X into subsets, X2 and X2.)
3: Z1 + MELwitNNoisMMAAVabbPNN
X2, X its X percent) (Generate a synthetic data copy of
using Als
4: Ze + ICLWiINNGGSMIIVVAAPPNNXXX
X1, its X2, percent) (Generate a synthetic data copy of
X3 using A
5: Z f Concatenate (Z,, Zg) (Concatenate the synthetic datasets Z, and Zz.-]
V X, Z) Thhss function uses X to determine which variables have integer type round the 6: Z RoundInteger Variables
values of the corresponding variables in Z the nearest integer:)
7: Output: synthetic data Z
Figure 20 shows an illustrative example (using simulated dataset with correlated beta distributed data). Panels al to show a scatterplots of the MIAVs. My versus the original variable, X,. Panel a2 to e2 show the respective scatterplots for the noisy MIAVs obtained by adding Gaussian noise with standard deviation given by 0.1 sd (M,) (where sd(M) represents the standard deviation of the MIAV Mj). Similarly, panels a3 to e3 and panels a4 to e4 show the scatterplots for the noisy MIAVs obtained with standard deviations 0.2 x sd( M,),, ) and 0.3 x sd(M,). respectively.
Figure 21 compares the densities of synthetic data generated with the MIAV approach (panels al to el) against synthetic data generated with the noisy-MIAV approach using increasing amounts of noise (namely, 0.1 sd(M,) for panels a2 to c2, 0.2 x sd(M)) for panels a3 to e3, and 0.3 x sd( (M,) for panels a4 to e4.)
Finally, in Figure 22 we report the results from synthetic and real-world data experiments evaluating the noisy-MIAV approach w.r.t. the same fidelity and privacy metrics using the same 43 datasets based on 10 distinct random splits of the data
evaluated in the main paper. As before, the results into original and holdout datasets and the figure report results pooled across all datasets. As expected, the noisy-MIAY approach trades increase in data privacy by a decrease in data fidelity.
43

---

Published as a conference paper at ICLR 2026
0.7
66
82
16
0.80
193
66 37
0.
0.2
3.8
80
81
01
18
18
635
0.00
2.00
1.00
60 32
04
300
04
[030
0.80
5
15
Ny
164
T041
101)
6
Figure 20: MIAV and noisy-MIAV versus original data scatterplots. Panels al-el, a2-e2, a3-e3, and a4-e4 show scatterplots of original data versus MIAV and original data versus noisy-MIAVs generated with increasing amounts of noise.
zighal
02
0.9
04
35
(a2)
X1
(b2)
X
[c21
X
(d2)
X
(e2)
X
md
(n:3
X1
(b:3)
X
[=3]
X3
(d3)
X
(o:)
X5
AN,
00 92
mol
Vig ml
warddeeviiee velues
Laradde-siee X
VAP85 viluet
(a4)
X1
(bA)
X
[c4]
(d4)
X
(o4)
vickt
srola
NMMV 03
32
05 81
06 02 04 0.5
02 3.9
60
25
0.6 3.5
Figure 21: Marginal distributions generated with the noiv strategy. Panels al to el report the densities based on the standard MIAV. Panel a2-e2, a3-e3, and a4-e4 show densities based on the MIAV (NMIAV) generated with noise percent set to 0. 0.2, and 0.3, respectively.
44

---

Published as a conference paper at ICLR 2026
[a] KS (simel)
1 (nimal)
(C) BT (simul)
pC (simul)
[e/SDNAL (simul.
[1] SSPID (simul)
0.87
rec
c2005
2012-
20.55
0.44
0.0000
1 000
AM00 02-
A DI
13000 D00
015
AM00 025
D1N
0200002
[gl KS.KKSCCC (CC18)
12) (CC18)
(
DT
(
DCR (CC18)
SDBRL (CC18)
(I) SSDID (CC18)
medan oeanelooo ocn ao drttu
0.35
&
2.000
1S00 21-
Mow 0.1
wHo 02-
www.da
www. a
wow,a2
unav 278-
AD0
VIDO 215-
www,01
www 0-1-
www.0
0.89
Siaa c
www. a
wwwcaa
F0000
(m) KS (basa.)
n)) L2D (baca)
(b)
DT [base.)
(p)
DCR [base.]
g/SDBRL (base:)
(r) 55D1D (aas.))
&
1.000
Nora 072
UA 003
EX A.11
hot 00, 0.22
Har
URWA 00
bo 0.15
hot 00, 0.2-
MREAL 43
022
011-
Figure 22: Experiment results for the noisy-MIAV strategy with noise percent increasing from 0.05 to 0.3 (in 0.05 increments). Increasing amounts of noise lead to decreasing data fidelity (larger values for the KS, L2D, and DT metrics), but improved data privacy (larger DCR values and lower SDBRL and SSDID values). Each boxplot displays the results from 10 distinct origina//holdout data splits. Top panels show the pooled results for the synthetic data experiments. Middle panels show the pooled results for the OpenML-CC18 datasets in Table Bottom panels show pooled results for the datasets in Table 5
(The boxplots omit outliers to improve visualization.)
K SYNTHETIC DATA GENERATION BASED ON TABICL PFN MODELS
To illustrate that our proposed synthetic data generation strategy is not restricted to TabPFN models, here we illustrate its application in conjunction with the TabICL model (Qu et al., 2025), which corresponds to an alternative PaNed tabular foundation model. TabICL provides a much more scalable alternative to TabPFN, being able handle datasets with up to 500,000 examples using affordable compute resources. However, similarly to TabPFN, it is still constrained to datasets with most 500 features. (Observe, however, that because the MIAV strategy only requires training of TabPFN (or TabICL) models with single feature per variable, this constraint has no impact on the MIAV approach, which can potentially be used to generate synthetic data versions of datasets containing more than 500 variables.)
Because TabICL currently can only handle classification tasks, our illustrations are restricted to datasets containing only categorical variables. Since we are also comparing the TabICL-bbased synthetic data generation against the strategies based on TabPFN we restrict our comparisons to the datasets in the OpenML-CC18 benchmark suite with at most 10,000 samples, 500 features, containing categorical variables with at most 10 level classes, and which contain more categorical variables than numeric ones. (We dropped the numerical variables from the few datasets that contained both numeric and categorical variables.) Table 8 shows the selected datasets for these comparisons.
As before, consider data generation strategies, namely, the MIAV-TabICL, the JF-TabICL, and the FC-TabICL and compare it against the corresponding TabPFN based MIAV. JF. and FC strategies (denoted as MIAV-TabPFN, the JF-TabPFN, and the FC-TabPFN in the plots below).
45

---

Published as a conference paper at ICLR 2026
Table 8: Datasets used for the TabICL illustrations and comparisons with TabPFN. These include all 8 datasets in the OpenML-CC18 benchmark suite with at most 10,000 samples, 500 variables, 10 classes per categorical variables, and larger number of categorical variables than numeric ones. In the first column we assign simplified dataset names (C1 to C8) to the original dataset names. The number of categorical variables is abbreviated as #cat, and the number of classes of the categorical variable with most classes is abbreviated as #class. In datasets C2 and C3 we remove the numerical variables before running the analyses.
MАМE C2
NAME
KR-VS-KP
3196
37
37
CMC
1473
10
23
CREDIT-G
1000
21
14
10
SPLICE
3190
61
61
45
C5
TIC-TAC-TOE
958
10
10
3
49
ANALCATDATA--IE DMFT
C7
CAR
1728
146821
DNA
3186
181
181
167140
The implementation of the TabiCL-based strategies is analogous to the TabPFN ones (we just need to switch the iontertttteeeiiiiioooooonnnnnnao corresponding Generaaterredtiionnsssisnggaaaa
() function in Algorithms and by the () function).
The experiments were run as before, where each dataset was first randomly split into approximately equal sized original and holdout sets, and report the results from 10 data splits. Since we only consider categorical datasets, we adopt: (i) the average KL-divergence metric to measure the quality of the synthetic data marginal distributions (where compute separate KL-divergence scores for each column of the dataset and take the average across all columns as the final metric); and (ii) the 1.21) metric for measuring how well the synthetic data captures the pairwise statistical associations observed in the original data (where the pairwise associations of the categorical variables are measured with the Cramer-V V statistic.). For both of these metrics, lower values indicate better data fidelity.
Figures 23 to 24 report the results for the average KL-divergence and L2D metrics, respectively, For comparison, the figures also show the results for the holdout datasets. For all datasets, the results for MIAV-TabICL and MIAV-TabPFN were very close. As before, the MIAV-based strategies tended to outperform the JF and FC ones.
(b
C3
C4
0.4
3.0
03
06
15-
15
1.0
10
05-
HOM SUPP
Hww, TORCLE
bouboou
FC. TeBCL
boatouu
Mioy TAPPA
IF TAPPA-
Jr. Iretwec--
FC, TeaCL
SUPP
May, RECCL
PC -U
FC THRICL
FC
(g)
C7
(h)
C8
20
overagg Beerggene
everrgg X0
00
FC -Clee-----
HWw Tourre
Jr TACC.
TMAC
MAAL MOALTTOTE TUITH
JF. Thter-
Jr. nacc
FC -Llouer-
FC, naIC.
M000 te-lbuu
Now TMCCL
IF Talers
PC R--eer
FC DACCL
Figure 23: KL-divergence comparison for categorical datasets C1 C8 (see Table @ for the original
dataset names). Each boxplot displays the results from 10 distinct original/holdout data splits.
46

---

Published as a conference paper at ICLR 2026
(b)
C3
C4
10.00
6.12
20:00
29.04
5.10
408
50.06
0.00
0.02
go:2-
400
20.00
20.00
HWV TooKL
Jr. TT.TheLL
PC, CUeP-
houloou
NM02 SAPPr
Ir. Tr.tcc
FC, bIpP
FC Teatt
IF TMPPN-
Ir. TT.TToCL
PC_ "Cuoo
FC TeaCs
Mav Jooe
PC, PC-P-
FC. TeaC.
FC.
Ir.
C6
C2
C8
2:00 between MEOO mentoss
332
6.35-
300
1.00
0.10
313
5.110
..00
2000
20.00
Jr. T.bcc
PC, TOOPPN-
MUN T00PP--
Jr. T....
PC, PC.OPPP
HIAL TeoPr
JE TePP Jr. T..necc
FC T...eN
PC. TabCL
MIsu Incabbuu
Mak. TabCL
PC, teeer
FC. T...A
IF Te00
MAy.
Figure 24: L2D comparison for categorical datasets C1 to C8 (see Table the original dataset
names). Each boxplot displays the results from 10 distinct original/holdout data splits.
L INSENSITIVITY TO THE CHOICE OF THE RANDOM NOISE DISTRIBUTION
In all experiments reported this paper, we construct MIAV variables with random noise sampled from uniform distribution. A natural question is whether the MIAV approach is sensitive the choice of the distribution used the generate the MIAV variable. Here, we clarify that the MIAV approach based on the TabPFN model is insensitive internally, TabPFN pre-pros pre-process all features to approximately standard normal distributions before running them through the transformer. (As described in the Methods section Hollmann al. (2025), the neural network of TabPFN expects approximately normal features after all pre- pre-processing steps. To this end, for each input, TabPFN employs Yeo-Johnson power transformation stabilize variance and make the distributions approximately normal, followed by z-transformation center the inputs at and scale their variance to 1.) As a consequence, the choice of noise distribution does not have an impact on the performance of the MIAV approach since the MIAV input variable is internally transformed to approximate a standard normal distribution.
the choice of noise distribution because,
To illustrate this point compare the performance of the MIAV approach implemented with different random noise distributions including uniform (U' (0, 1)). gaussian (NV 1)), and exponential (Exp(1)) noise. We again simulate data from correlated beta distributions and evaluate qualitatively the quality of the MIAV-based synthetic data generated with different random noise distributions. (In all these illustrations we use the same original data. Only the synthetic datasets generated with the MIAV approach differ.). Figure 25 reports the marginal distributions of the original data (black densities) and their respective MIAVs (red densities) (panels to e), alongside seatter-plots of the original and MIAV variables (panels f to j), for synthetic data generated with MIAVs following uniform distribution. Figures(26) and 27 report analogous comparisons for synthetic data generated with MIAVs following gaussian and exponential distributions.
Figure 28 reports the marginal densities of the original data (black) and the synthetic data (red) generated with uniformly distributed MIAVs. The figure shows the results from 3 separate replications based on different random seeds, where the top, middle, and bottom panels report the results generated with distinct random seeds. Figures 29 report the analogous results for synthetic data generated with gaussian and exponential random noise distributions, respectively. Comparison of Figures 28 29 and 30 shows that the quality of the synthetic data remains unchanged with the different choices of random noise distributions.
47

---

Published as a conference paper at ICLR 2026
M
X1
distr.
Mg.
X distr.
Ma X3 distr.
Ma. Xa distr.
Ms Xs distr.
3.0
a)
X1
X2
00
(C) X3
0, 03 1.2
(d)
3.0
[e)
X5
M
M
Ms
2.0
density
density
density
2.0
1.
1.0
M
0.0
0.0
0.2
0.4
0.8
1.2
-0.2
0.4
0.8
1.2
-0.2
0.4
0.8
1.2
0.0
0.5
1.0
-0.2
0.4
0.8
1.2
variable values
variable values
variable values
variable values
variable values
Mj VS X
M X
M vs X3
M4 vs X4
Ms
Xs
ff)
()
0.8
0.8
0.8
163
M2
0.4
0.4
00
0.4
0.0
0.0
00
0.2
0.4
0.6
0.8
0.0
0.4
0.70
0.85
1.00
0.0
0.4
0.8
0.0
0.2
0.4
0.6
X1
X2
X
X4
X
Figure 25: Illustrative example with MIAVs generated with uniform random noise.
M 1 distr:
M2
distr.
M3. X3 distr.
Ma. X4 distr:
Ms.
X5
distr.
30
(b)
X2
1.2
X
3,0
Xs M
M1
M
M
Ma
GRTS
2.0
density
demsitty
Resuao
00
2.
Seise
1.0
10
1.
2
00
00
00
-2
-2
-2
variable values
variable
variable values
variable values
variable values
M VS X
M, vs X
M VS X
M, vs X
M vs X,
M2
7
17
T
0.4
0.8
0.0
0.4
0.70
0.85
1.00
0.0
0.4
0.8
0.0
0.4
X1
X3
X
Figure 26: Illustrative example with MIAVs generated with gaussian random noise.
M X1 distr.
M2 X2 distr.
M3 X3 distr.
Ma Xa distr.
M
Xs distr.
3.0
X1
X
80
X
X
3.0
Xs
Mi
M
Ma
Ma
Ms
2.0
density
0. 08
2.0
1.0
1.0
0.0
00
variable va luees
variable values
variable values
variable values
variable values
M, Vs. X
Ma
X
M vs.
M, vs X,
Ms vs Xs
f)
g)
(h)
M1
M2
M4
2
0.2
0.4
0.6
0.8
0.0
0.4
0.70
0.85
1.00
0.0
0.4
0.8
0.0
0.2
0.4
0.6
X1
X
X
X
X
Figure 27: Illustrative example with MIAVs generated with exponential random noise.
48

---

Published as a conference paper at ICLR 2026
(a) X (repl. 1)
Xb (repl. 1)
(C) ( 1)
x 1)
(e) Xs (repl. 1)
3.0
original
original
0. 08 12
3,0
original
M LAV
MIAV
MIAV
2.0
density
density
6
07 O'
1.0
original MIAV
original MIAV
0.0
00
00
0.6
0.0
0.5
1.0
0.0
variable values
varible values
variable values
variable values
variable values
X, (repl. 2)
(g) 2 (repl.
(h) X, (repl. 2)
X4 (repl. 2)
X (repl.
00 12
3.0
original LAV
original MIAV
30
original M
densRy
20
density
6
2 T1 Senie
10
40
original MIAV
original MIAV
0.0
00
00
0.2
0.6
0.0
0.4
0.8
0.7
0.9
0.0 0.5
1.0
0.0
0.4
variable values
variable values
variable values
variable values
variable values
(k) X, (repl. 3)
X (repl. 3)
(m)x (repl. 3)
(n) X (repl. 3)
(o) X, (repl. 3)
3.0
original
original MIAV
21
original MAV
LAV
density
2.0
Gersity
dersity
a 6
Arsuao
80
0.0 1.0
original MIAV
original MIAV
00
0.2
0.6
0.0
0.7
0.9
0.0
0.5
0.0
variable values
variable values
variable values
variable values
variable values
Figure 28: Marginal distributions of the original data (black densities) and the synthetic data (red densities) generated with uniformly distributed MIAVs. The top, middle, and bottom panels report results generated with different random seeds.
(g) x (repl. 1)
(h) x (repl. 1)
(i) X (repl. 1)
Xa (repl. 1)
(k) X5 (repl. 1)
3.0
original
8
original MIAV
1.2
original MIA
dersity
20
dersity
MAV
dereity
5
Aesuss oa
1.0
original MIAV
2
01
original MIÁV
0.0
00
0.2
0.6
0.0
0.9
0.0 0.5 1.0
0.0
variable values
variable values
variable values
variable values
variable values
(f) X, (repl. 2)
X (repl.
(h)x, X (repl. 2)
(i) X (repl. 2)
X (repl.
8 10
original
original MIAV
1.2
0 3.0
original M
LAY
densty
density
6
denehy
0. 08
4
original MIAV
2
original MIAV
0.0
00
0.0 o's 1.a
0.2
0.6
0.0
0.4
0.8
0.7
0.9
0.0
variable values
variable values
variable values
variable values
variable values
(k) X, (repl. 3)
(I) X, (repl. 3)
(m)x, (repl. 3)
(n) X (repl. 3)
(0) X (repl. 3)
3.0
original
original
original MAV
denstty
20
density
M LAV
density
5
MIAV
Arsuso
80 04
2
1.0
original MIAV
2
original MIÁV
0.0
00
0.6
0.0
0.9 variable values
D variable values
0.0
variable values
variable values
variable values
Figure 29: Marginal distributions of the original data (black densities) and the synthetic data (red densities) generated with normally distributed MIAVs. The top, middle, and bottom panels report results generated with different random seeds.
49

---

Published as a conference paper at ICLR 2026
(a) X (repl. 1)
Xb) (repl. 1)
(C) X (repl. 1)
x 1)
(e) (repl. 1)
3.0
original
original
original MIAV
density
2.0
dersity
M IAV
6
MIAV
density
0.0 04 0.8
1.0
original MIAV
original MIAV
0.0
0.2
0.6
0.0
0.7
0.9
0.5
0.0
variable values
vanable values
variable values
variable values
variable values
(f) X (repl. 2)
(g) 2 (repl. 2)
Xh) (repl. 2)
X, (repl. 2)
X (repl.
3.0
original
original
1.2
original MAV
8
MIAV
dersity
2.
dersity
6
dersity
08
1.0
04
origina MIAV
original MIAV
0.0
0.0
0.2
0.6
0.0
0.4
0.8
0.7
0.9
0.0
0.s
1.0
0.0
0.4
variable values
variable values
variable values
variable values
variable values
(k) x (repl. 3)
(I) X (repl. 3)
(m)x, (repl. 3)
(n) (repl. 3)
(o) Xs (repl. 3)
12
3.0
original
original
02 0 TO 00 Aqsuap
original MIAV
density
2.0
density
M IAV
dunsity
6
MIAV
Jonm 0. 08
1.0
original
2
original MIAV
0.0
0.0
0.5
0.2
0.6
0.0
0.7
0.9
0.0
0.0
variable values
variable values
variable values
variable values
variable values
Figure 30: Marginal distributions of the original data (black densities) and the synthetic data (red densities) generated with exponentially distributed MIAVs. The top, middle, and bottom panels report results generated with different random seeds.
M LLM USAGE
ChatGPT-5 was used to refine the grammar and clarity of some paragraphs of the paper and to translate code into Python code. We reviewed and verified all Al-generated content for accuracy and take full responsibility for the paper's final content.
50


Reviews:
[{"review_id": "Y1MFZephLa", "reviewer": "Reviewer_4vKk", "summary": "This paper focuses on tabular foundation models (like TabPFN) and explores the use of these models for synthetic data generation (as opposed to tabular classification). The authors claim that when such models are used in an ICL (in context learning) mode weak associations between columns can cause poor performance. As a result they propose a so-called maximal information auxiliary variable (MIAV) strategy, which enriches the model’s context using auxiliary variables that they derive through a rank-matching process.", "questions": "- Please address the above questions from the Weaknesses section.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- Focuses on synthetic data generation which appears to be understudied in the tabular foundation model space\n- Generalizes across multiple tabular foundation models\n- The proposed MIAV appears to be theoretically grounded", "weaknesses": "-  The proposed framework seems highly reminiscent of Bayesian structure discovery algorithms which also need to learn an ordering of variables to support better sampling (e.g., for MCMC methods). Can you comment on the relationships (or lack thereof) to that literature?\n- The idea of creating auxiliary variables is good but it seems orthogonal to seeing whether a different ordering of variables will work. For instance, see this work: https://arxiv.org/abs/2406.14541. Can you comment on when we will need a different ordering vs when your approach is needed?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aiarZ0l9BP", "reviewer": "Reviewer_zUsD", "summary": "The paper proposes an approach to improve tabular data generation models like TabPFN on the synthetic data generation task. Specifically, the authors develop a maximal information auxiliary variable (MIAV) based framework capable of producing accurate synthetic data even in the context of variables with weak associations, overcoming the scalability issues of TabPFN. Experiments performed on numerous synthetic and real-world datasets demonstrate that the proposed MIAV conditioned TabPFN model is comparable to or better than state-of-the-art synthetic data generation approaches. Overall, the paper develops a novel method that is consistent with the claims made in the paper and will contribute to the development of efficient, high-quality synthetic data generation for tabular data especially with weakly associated and small datasets.", "questions": "1. How does the proposed model perform in the context of out-of-distribution synthetic-data generation task compared to TabPFN, especially owing to its ability to perform well in the context of variables with weak associations? \n\n2. Scalability is an important aspect of such models and this is somewhat of a concern for me at the moment. Why have synthetic datasets only been generated with <=3000 instances in the scenario evaluating runtime analysis? Is it difficult to scale the model over these to truly large scale data (e.g., 10000 rows or greater)?  If scaling is difficult, what are the bottlenecks? \n\n3. How does the current proposed approach scale with the number of columns in the datasets?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well written and easy to follow. Further, the problem of synthetic data generation of tabular data comprising continuous and categorical variables, that is being tackled, is of prime importance.\n\n2. The proposed MIAV method is simple yet effective in accounting for weak associations along with improving generation scalability. The method is capable of being applied to both continuous and categorical variables. \n\n3. The paper also develops theory to support its claims of improved scalability and demonstrates state-of-the-art results on numerous synthetic and real-world data compared with multiple baselines.", "weaknesses": "1. Out-of-Distribution generalization is alluded to the the introduction but it is unclear if it has been demonstrated in the paper. \n\n2. Run-time experiments have been run on small synthetic datasets (e.g., <= 3000 samples).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2Q7IuGaWrV", "reviewer": "Reviewer_DDxA", "summary": "The paper identifies a problem with using tabular foundation models like TabPFN to generate tabular data, viz., that their performance degrades significantly when variables have weak statistical associations. The paper then proposes a novel method called the Maximal Information Auxiliary Variable (MIAV) strategy to deal with this problem. The MIAV strategy creates an auxiliary variable for every variable through a rank-matching procedure between the real data and a random noise vector. The core insight is that this auxiliary variable is perfect (in the rank-based sense) predictor of the corresponding real variable. A concise theoretical justification for MIAV is provided and its practical benefits, including computational efficiency (reducing complexity from cubic to linear in the number of features) and invariance to column order are demonstrated through extensive empirical evaluations.", "questions": "1. A uniform distribution is used as the random noise generator for the rank-matching. What happens if the distribution were changed, say to a Gaussian? How sensitive is the method to this choice of distribution?\n2. SMOTE seems to be performing well on  fidelity metrics alone, although MIAV is often better when fidelity-privacy trade-off is considered. Can you elaborate on when a practitioner would prefer MIAV over the much simpler SMOTE?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "+ The paper clearly identifies the \"weak associations\" problem.  \n+ The MIAV strategy is simple and effective.\n+ The strategy reduces computational complexity, which is a practically significant contribution.\n+ The Theorem 1 provides strong theoretical grounding for the MIAV strategy. It is also well supported by the extensive evaluations.", "weaknesses": "- The MIAV strategy is an \"add on\" in the sense that its performance of the overall generator  will  always be limited by the underlying model (TabPFN, TabICL etc.). This could be more explictly stated as a limitation\n- The method requires generating MIAV for the entire dataset, including the test set. This could cause information leakage if someone were to apply this method for prediction. Although, footnote 1 on page 7 clarifies that this meant for generation not classification task, it should probably be stated even more emphatically in the paper, to prevent misapplication of this method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3pcP24VTdg", "reviewer": "Reviewer_PSBh", "summary": "The paper highlights a limitation of current tabular foundations models (and in particular of TabPFN): namely, it does not perform in features that are poorly correlated with the others. \n\nTo overcome this limitation they propose to leverage maximal information auxiliary variables (MIAV). These variables are build from random noise and retain the information about how the datapoints are ordered (from lowest to highest in case of continuous features) with respect to the value of a feature. The authors show that following this construction, each variable $X_j$ conditional on its MIAV is independent of all other variables and the MIAV retains maximal information about $X_j$ in a non-conditional way.", "questions": "See weaknesses", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "The paper identifies a limitation of a well established model. \n\nThe authors propose a sensible solution that seems to work.", "weaknesses": "1) The paper needs a lot of rewriting to make it clearer and more structured. I give some pointers to some improvements below: \n   - The section Notation and Related Work should be two sections: it is not clear why this is a single one\n   - Calling vectors of random variables in slanted boldface and matrices as simple boldface makes it difficult to distinguish between the two of them (the space of the matrices of size $mxn$ is isomorphic wrt to the space of vectors of length $mn$, why not treating everything as vectors for example? it might help wrt the notation_\n   - In the notation the symbol $X_{-j}$ is used at line 119 but only introduced at line 122\n   - In $q_\\theta(x_j^{ts} \\mid X_{-j}^ts, X^tr)$ it would be clearer if you just define it as $q_\\theta(x_j^{ts} \\mid X_{-j}, X^tr)$ as you are removing the feature $j$ from both the training at testing\n   - $p$ in equation (1) is not defined (I guess it is the number of features)\n   - equation (2), the small $x$ might be missing the $\\hat$\n   - page 4 the authors talk about the permutation sampling approximation of Janossy pooling: explaining what this means takes 2 lines and would make the paper more accessible \n   - the experimental analysis is just a big wall of text right now and could use some structuring: the authors could consider adding paragraphs titled \"Metrics\", \"Experimental Setup\",\"Datasets\" etc. \n\n2) It is not clear to me why the authors are sorting in line 2 of the algorithm if then the noise needs anyway to be sorted on the ground of the values of the features\n\n3) in both the algorithm and the theorem the a variable $Z$ and vector $z$ appear, which I think are used as $M$ and $m$ but they are not defined anywhere\n\n4) it is not clear how to read the colours in the charts in the last column of Figure 3 \n\n5) At page 6 the authors write that $X_j$ is *completely determined* by $M_j$. I am not sure how that is possible if $M_j$ is essentially just modelling the ordering \n\n6) In page 6 Equation (4) I do not understand why the authors suddenly use $X_{<j}$ - I thought they were just considering $X_{-j}$.\n\n7) In the experimental analysis  is missing  common metrics used in tabular data generation like utility and sample generation time. Also, in order to measure the statistical fidelity why didn't the authors use the Wasserstein distance for continuous features? That ia very commonly used. \n\n8) The authors did not consider more recent baselines like GReaT which are transformer based.\n\nOverall, I believe the paper presents an interesting solution but it needs more work before publication.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)