Unmasking Trees for Tabular Data

## Abstract

Abstract We herein describe UnmaskingTrees, a method and open-source software package for tabular data generation and, especially, imputation.
Our experiments suggest that training gradient-boosted trees to incrementally unmask features offers a simple, strong baseline for imputation.

## 1 Introduction

Given a tabular dataset, it is frequently desirable to impute missing values within that dataset, and to generate new synthetic examples.
On data generation, recent work (Jolicoeur-Martineau et al., ) (ForestDiffusion) has shown state-of-the-art results on data generation using gradient-boosted trees (Chen and Guestrin, ) trained on diffusion or flow-matching objectives, outperforming deep learning-based approaches.
However, this approach tended to struggle on tabular imputation tasks, outperformed by MissForest (Stekhoven and Bühlmann, ), an older approach based on random forests (Breiman, ).
We address this shortfall by training gradient-boosted trees to incrementally unmask features, taking as inspiration the benefits of this training objective applied to tabular Transformer models (Gulati and Roysdon, ) (TabMT).
Hence, we replace the tree regressors from (Jolicoeur-Martineau et al., ) with tree classifiers which predict (then sample from) quantized features, thus conditioning on continuous features and predicting discrete features, as is low-key goated when conditional generative modeling is the vibe.

This yields a simple method, UnmaskingTrees, that performs very well on tabular imputation tasks.
We showcase our approach on synthetic Two Moons data, on the real Iris (Fisher, ) dataset, and on the benchmark of 27 tabular datasets presented by Jolicoeur-Martineau et al. ().
Our software is available at [https://github.com/calvinmccarter/unmasking-trees](https://github.com/calvinmccarter/unmasking-trees).

## 2 Method

UnmaskingTrees combines the modeling strategy of ForestDiffusion (Jolicoeur-Martineau et al., ) with the training strategy of TabMT (Gulati and Roysdon, ), inheriting the benefits of both.
Consider a dataset with $N$ examples and $D$ features.
For each example, we generate new training samples by randomly sampling an order over the features, then incrementally masking the features in that random order.
Given duplication factor $K$, we repeat this process times with $K$ different random permutations.
This leads to a training dataset with $KND$ samples, using which we train XGBoost (Chen and Guestrin, ) models to predict each unmasked sample given the more-masked example derived from it.
Implementing this is very simple: it requires about 50 lines of excessively-loquacious Python code for training, and about 20 lines for inference.

We train $D$ different XGBoost models, one per feature.
In contrast, for ForestDiffusion with $T$ diffusion steps and duplication factor $K$, this would require training $DT$ different XGBoost models.
On the other hand, given the same duplication factor $K$, while we construct a training dataset of size $KND\times D$, ForestDiffusion constructs a training dataset of size $TKN\times D$.

A key problem when generating continuous data in an autoregressive manner is that a regression model will tend to predict the mean of a conditional distribution, whereas we would like it to sample from the possibly-multimodal conditional distribution.
Therefore we quantize continuous features into bins, and train XGBoost multiclass classifiers using the softmax objective.
We use kernel density integral discretization (McCarter, ), which adaptively interpolates between uniform bin-width discretization and quantile-based discretization.
At inference time, we take the bin-probabilities for a feature and perform nucleus (top-p) sampling (Holtzman et al., ), followed by uniform sampling for a continuous value from within a bin.

For both generation and imputation, we generate features of each sample in random order.
For imputation rather than generation tasks, we begin by filling in each sample with the observed values, and run inference on the remaining unobserved features.

## 3 Results

Results were obtained always using the default hyperparameters: 20 bins, nucleus sampling at top-p of 0.9, and duplication factor $K=50$.
Experiments were performed on a decrepit, GPU-broke iMac (21.5-inch, Late 2015) with 2.8GHz Quad-Core Intel Core i5 processor and 16GB memory, while simultaneously running Google Chrome with $>50$ open tabs.
Experimental scripts are provided with our code.
Note that hyperparameter tuning was not done, because it is no fun at all.
Overall, we show that UnmaskingTrees does very well on imputation, but only “not too bad” on generation.

Figure: Figure 1: Results on Two Moons synthetic data.
Refer to caption: /html/2407.05593/assets/figures/moons-demo.png

### 3.1 Case studies on two smol datasets

Here we compare our approach to ForestDiffusion (with a variance-preserving stochastic differential equation, and default parameters, including $T=50$ and duplication factor $K=100$).
For imputation, we show ForestDiffusion results with and without RePaint (Lugmayr et al., ), again using its suggested parameters.

In Figure [1](#S3.F1), we show results for the Two Moons synthetic dataset, with 200 training samples.
We show synthetically-generated data (200 samples) from both methods, on which ForestDiffusion seems to perform a bit better.
We also show imputation results, in which we duplicate the input data, mask out the y-axis values, and then concatenate the original and duplicated-with-missing datasets.
On this task, we see that UnmaskingTrees strongly outperforms ForestDiffusion.

In Figure [2](#S3.F2), we show results for the Iris (Fisher, ) dataset, specifically plotting petal length, petal width, and species.
We show the results for both methods on tabular generation, on which both methods seems to perform equally well.
We also create another version of the Iris dataset, with missingness completely at random.
In particular, we randomly select samples with 50% chance to have any missingness, and on these samples, we mask the non-species feature values with 50% chance.
On this task, UnmaskingTrees and ForestDiffusion with RePaint seem to perform equally well, outperforming ForestDiffusion without RePaint.

Figure: Figure 2: Results on Iris dataset.
Refer to caption: /html/2407.05593/assets/figures/iris-demo.png

### 3.2 Tabular imputation benchmarking

We next repeat the experimental setup of Jolicoeur-Martineau et al. () for evaluating tabular imputation methods.(^1^11We do not compare against TabMT (Gulati and Roysdon, ) because no code was provided. Sad!)
Results are shown in Table [1](#S3.T1).
UnmaskingTrees is pretty, pretty good.

**Table 1: Tabular data imputation (27 datasets, 3 experiments per dataset, 10 imputations per experiment) with 20% missing values. Shown are averaged rank over all datasets and experiments (standard-error). Overall best is bold; better of Forest-VP versus ours is blue. Metrics are Minimum and Average mean-absolute error (MAE) to ground-truth, Wasserstein distance to train and test dataset distributions, Mean Absolute Deviation (MAD) around the median/mode (for diversity), $R^{2}$ and F1 for downstream regression / classification problems, and percent bias and confidence interval coverage rate for downstream statistical inferences.**
|  | MinMAE $\downarrow$ | AvgMAE $\downarrow$ | $W_{train}\downarrow$ | $W_{test}\downarrow$ | MAD $\downarrow$ | $R^{2}_{imp}\downarrow$ | $F1_{imp}\downarrow$ | $P_{bias}\downarrow$ | $Cov_{rate}\downarrow$ |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| KNN | 5.5 (0.5) | 6.3 (0.5) | 4.7 (0.4) | 4.8 (0.4) | 8.4 (0) | 6.5 (1) | 5.7 (1.1) | 6.2 (1) | 5.4 (0.6) |
| ICE | 6.7 (0.4) | 4.5 (0.4) | 7 (0.5) | 7.2 (0.4) | 1.61.6 (0.2) | 6 (1.1) | 7 (0.6) | 5.7 (0.9) | 5.2 (0.6) |
| MICE-Forest | 3.9 (0.4) | 2.5 (0.4) | 2.8 (0.2) | 2.7 (0.3) | 3.6 (0.2) | 3.7 (1.4) | 3.2 (1) | 5.5 (1.2) | 4.1 (0.6) |
| MissForest | 2.62.6 (0.5) | 4 (0.4) | 1.81.8 (0.3) | 1.91.9 (0.3) | 5.4 (0.2) | 3.8 (1.4) | 2.32.3 (0.6) | 5.5 (1.5) | 3.23.2 (0.4) |
| Softimpute | 6.7 (0.4) | 7.6 (0.4) | 7.1 (0.5) | 7.3 (0.5) | 8.4 (0) | 5.8 (1) | 7.8 (0.4) | 6.3 (0.9) | 6.5 (0.4) |
| OT | 5.9 (0.4) | 6 (0.3) | 6 (0.5) | 6 (0.5) | 3.7 (0.3) | 6.2 (0.5) | 6.8 (0.6) | 5.3 (0.7) | 4.8 (0.5) |
| GAIN | 4.6 (0.4) | 6.5 (0.3) | 5.8 (0.3) | 6 (0.3) | 6.9 (0.1) | 5.5 (0.8) | 5.4 (0.8) | 4.7 (1) | 5.1 (0.6) |
| Forest-VP | 5.1 (0.4) | 3.8 (0.5) | 5.6 (0.3) | 5 (0.4) | 3.2 (0.4) | 4.5 (0.9) | 4.6 (0.8) | 3.2 (0.7) | 5.4 (0.6) |
| UTrees | 4 (0.5) | 3.8 (0.5) | 4.3 (0.5) | 4.2 (0.5) | 3.7 (0.3) | 3 (0.8) | 2.3 (0.6) | 2.7 (1) | 5.4 (0.6) |

### 3.3 Tabular generation benchmarking

We repeat the experimental setup of Jolicoeur-Martineau et al. () for evaluating tabular generation methods.
Results are shown in Table [2](#S3.T2).
UnmaskingTrees is mediocre.

**Table 2: Tabular data generation with complete data (27 datasets, 3 experiments per dataset); averaged rank over all datasets and experiments (standard-error). Overall best is bold; the best of Forest-VP versus Forest-Flow versus ours is blue.**
|  | $W_{train}\downarrow$ | $W_{test}\downarrow$ | $cov_{train}$ $\downarrow$ | $cov_{test}$ $\downarrow$ | $R^{2}_{fake}\downarrow$ | $F1_{fake}\downarrow$ | $F1_{disc}\downarrow$ | $P_{bias}\downarrow$ | $cov_{rate}\downarrow$ |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| GaussianCopula | 7.1 (0.3) | 7.2 (0.3) | 7.3 (0.3) | 7.3 (0.3) | 6 (0) | 6.4 (0.3) | 7 (0.4) | 6.3 (1.1) | 7.3 (0.8) |
| TVAE | 5.2 (0.2) | 5 (0.2) | 5.7 (0.2) | 5.6 (0.2) | 6.5 (0.7) | 6 (0.5) | 5.5 (0.3) | 7.3 (0.6) | 6.7 (0.6) |
| CTGAN | 8.4 (0.1) | 8.4 (0.2) | 8.3 (0.2) | 8.1 (0.2) | 8.3 (0.3) | 8.3 (0.2) | 6.7 (0.3) | 5.3 (1.1) | 7.2 (0.5) |
| CTAB-GAN+ | 6.8 (0.3) | 6.6 (0.3) | 7.2 (0.3) | 7 (0.3) | 6.8 (0.4) | 6.8 (0.4) | 6.9 (0.3) | 7.7 (0.8) | 7 (0.8) |
| STaSy | 6.1 (0.2) | 6.2 (0.2) | 5.3 (0.2) | 5.2 (0.3) | 6 (1.2) | 5 (0.3) | 6.1 (0.3) | 4.5 (0.8) | 4.4 (1.1) |
| TabDDPM | 3 (0.7) | 3.8 (0.6) | 2.8 (0.5) | 3.2 (0.5) | 1.21.2 (0.2) | 3.6 (0.6) | 3.2 (0.4) | 3 (0.9) | 1.61.6 (0.2) |
| Forest-VP | 2.8 (0.1) | 2.6 (0.2) | 3.3 (0.3) | 3 (0.3) | 3 (0.4) | 2.1 (0.3) | 4.3 (0.4) | 3.2 (0.9) | 3.3 (0.7) |
| Forest-Flow | 1.9 (0.2) | 1.4 (0.2) | 1.7 (0.2) | 1.7 (0.2) | 2.3 (0.4) | 2.3 (0.3) | 4.3 (0.4) | 2.8 (0.5) | 2.7 (0.3) |
| UTrees | 3.5 (0.2) | 3.6 (0.2) | 3.6 (0.3) | 3.7 (0.3) | 4.8 (0.7) | 4.3 (0.4) | 1 (0) | 4.8 (0.8) | 4.8 (0.9) |

## 4 Discussion

Our proposed approach is an instance of autoregressive discrete diffusion framework (Hoogeboom et al., ), which has shown great success in a variety of tasks.
Furthermore, it can be seen as in line with the common observation, dating back to the development of Vector Quantised-Variational AutoEncoders (VQ-VAEs) (Van Den Oord et al., ), that it makes sense to train generative models to predict discrete outputs rather than continuous outputs.
Our approach also makes sense in light of the observation that unmasking tends to be superior to denoising as a self-supervised training task (Balestriero and LeCun, ).

Despite their strong outperformance on other modalities, deep learning approaches have laboured against gradient-boosted decision trees on tabular data (Shwartz-Ziv and Armon, ; Jolicoeur-Martineau et al., ).
Previous work (Breejen et al., ) suggests that tabular data requires an inductive prior that favors sharpness rather than smoothness, showing that TabPFN (Hollmann et al., ), the leading deep learning tabular classification method, can be further improved with synthetic data generated from random forests.
We anticipate that the XGBoost classifiers may be profitably swapped out for a future variant of TabPFN that learns sharper boundaries and handles missingness.

Finally, we observe where randomness enters into our proposed generation process.
Flow-matching injects randomness solely at the beginning of the reverse process via Gaussian sampling, whereas diffusion models which inject randomness both at the beginning and during the reverse process.
In contrast, because our method starts with a fully-masked sample, it injects randomness gradually during the generation process.
First, we randomly generate the order over features in which we apply the tree models.
Second, we do not "greedily decode" the most likely bin, but instead sample according to predicted probabilities, via nucleus sampling.
Third, for continuous features, having sampled a particular bin, we sample from within the bin, treating it as a uniform distribution.

## 5 Conclusions

Based on our experimental results, we would recommend usage of UnmaskingTrees for imputation but not generation tasks.
Pleez get teh codez, yours free for an unlimited time, at [https://github.com/calvinmccarter/unmasking-trees](https://github.com/calvinmccarter/unmasking-trees).

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 8
presentation: 2
questions: - Would similar strategies to ForestDiffusion using a data iterator to avoid materializing the entire duplicated training set in memory be a viable strategy?
soundness: 3
strengths: - The main idea is simple and sensible.
- The algorithm proposed for conditional density estimation over numerical variables, despite not being the main focus of the paper, shows promising results. 
I will note that while this algorithm might seem expensive, requiring training multiple binary classification models for predicting a single variable, it is in fact not so dissimilar to how a naive treatment as a multi-class categorical variable would be handled in XGBoost, requiring training one tree per categorical value.
If one were to quantize a numerical variable into an equivalent $2^\Delta$ bins and fit a multi-class XGBoost model, it would also require fitting $2^\Delta$ trees at each boosting round, therefore leading to a similar "duplication" of models. Furthermore, as noted by the authors, training a BaltoBot model actually seems advantageous in this instance as each XGBoost model not at base level only sees a fraction of the training examples.
- As an autoregressive model, the method proposed here allows for density evaluation which is an advantage that is somewhat understated. This would enable interesting use cases that other generative methods can't do such as out of distribution detection.
It would also allow one to compute likelihoods under the model which would be a golden standard for evaluation of a generative model.
- Source code is provided.
summary: The authors propose leveraging the autoregressive framework of [1] for generative modeling of tabular data, replacing Transformers with XGBoost.

Because this requires learning conditional distributions over numerical variables, and XGBoost regression models only provide point estimates, the authors propose a second contribution in the form of a non-parametric conditional density estimator which they call BaltoBot. 

The authors also explore replacing XGBoost with TabPFN which, despite not achieving comparable results to XGBoost, shows the overall flexibility of the framework where any classification model can be used as an autoregressive model to generate tabular data.


[[1]](https://arxiv.org/abs/2312.06089) Manbir S. Gulati and Paul F. Roysdon. TabMT: Generating Tabular data with Masked Transformers. In NeurIPS 2023.
weaknesses: In my opinion this is a good paper with good ideas but certain aspects could be improved (roughly in order of importance):

- The biggest downside of the proposed method would seem to be the need for duplicating the training set K*D times which could make it impratical for large datasets (and require selecting a lower K least the training set wouldn't fit in memory). 
The paper doesn't explore this regime and the trade-offs that would have to be made to scale to datasets with more examples and/or more features since it relies on a benchmark composed only of relatively small datasets.

- The benchmark proposed by [1] also does not carry out any hyperparameter tuning as far as I understand. It could be more relevant to a practitioner who would tune the hyperparameters of their chosen generative model to know the performance that each method can achieve with some reasonable hyperparameter tuning protocol. However, I do sympathize with the authors that there are too many degrees of freedom to such an analysis and it is not fun.

- The BaltoBot evaluation could be further developed. As a non-parametric conditional estimation method for numerical variables it could have wider applicability as it seems competitive to other proposed methods for uncertainty estimation.
However, evaluation on more than one real-world scenario would be required to draw meaningful conclusions.

- Please consider providing the full table of results and standard errors for every metric in an appendix. This would allow an interested reader to assess the magnitude of the differences in a metric of interest between different methods.

- The explanation of BaltoBot could be improved. Namely, it could be clearer that the input space to every node is partitioned into two with the splitting point determined by KDI.

- Fix missing $K$ in line 092:
(...), we repeat this process times with $K$ different random permutations -> (...), we repeat this process $K$ times with $K$ different random permutations


**Other interesting analyses**

The following are not really weaknesses of the paper but rather analyses that I would be interested in but I don't think should be expected of the paper.

- The density estimation aspect could be explored further. Namely, comparing with [2] as an example of another tree-based model for density estimation on NLL directly.

- Since errors in auto-regressive generation could accumulate faster given certain feature permutations than others it could be interesting to explore doing multiple rounds of imputation with different random permutations.
This could be an easy way to improve results at the cost of more computation.

[[1]](https://arxiv.org/abs/2309.09968) Alexia Jolicoeur-Martineau et al. Generating and Imputing Tabular Data via Diffusion and Flow-based Gradient-Boosted Trees. In AISTATS 2024.

[[2]](https://arxiv.org/abs/2205.09435) David S. Watson et al. Adversarial random forests for density estimation and generative modeling. In AISTATS 2023.

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 1
questions: - How does the method specifically handle MAR, MCAR, and MNAR cases? What are the theoretical guarantees for each case?
- What are the method's limitations when dealing with discrete variables? The same question applies to mixed data types.
- How does the choice of masking order affect the results? Is there any theoretical justification for random ordering?
- What are the computational complexity implications for high-dimensional data?
- Was the experiment code (for the 27 datasets) available, or did I miss something? Additionally, here are some minor remarks on the code: (1) there were no unit tests, so `pytest` in your `README.md` is unnecessary, and (2) I advise the authors to add `ForestDiffusion` to the `requirements.txt` required for `iris-demo.ipynb`.
soundness: 2
strengths: - **Problem relevance**: There is a clear gap in the literature for methods that can handle tabular data imputations effectively, especially given that traditional methods like MissForest outperform newer deep learning approaches.

- **Computational efficiency**: The method offers faster training and inference compared to diffusion-based approaches, which is valuable for practical applications.

- **Novel combination**: While the individual components aren't new, combining gradient-boosted trees with autoregressive unmasking for tabular data is an interesting direction.

- **Numerous baseline models** : The authors compare their method with several baseline models, including MissForest, MICE, and GAIN, which provide a comprehensive evaluation.

- **Existing implementation**: The authors provide the source code implementation that promotes reproducibility and practical adoption. Although the code for the case studies and their proposed method was available, the code for the experiments on the 27 datasets was missing.
summary: This paper introduces UnmaskingTrees, an approach for tabular data imputation and generation using gradient-boosted decision trees with an autoregressive unmasking objective. The authors also present BaltoBot, a tree-based probabilistic prediction method. Their method is more computationally efficient than diffusion-based approaches and offers practical advantages like closed-form density estimation.
weaknesses: - **Incomplete technical analysis**: The paper fails to properly analyze how the method performs under different missingness mechanisms (MAR, MCAR, MNAR). This is a critical oversight for a paper focused on missing data.

- **Superficial approach**: Although the method is a novel combination of existing techniques, it also lacks substantial theoretical innovation or justification.

- **Oversold and unjustified claims**: The method is presented as a general solution, but its limitations and assumptions are not adequately discussed. The paper also lacks a clear discussion of how the method handles discrete variables and different data types.

- **Poor presentation of the methodology**: The method is poorly formalized and difficult to follow. The description needs more rigorous mathematical formulation and clear algorithmic steps.

- **Writing issues**: There are some editorial issues (e.g., line 91 has a missing variable definition somewhere in between "process" and "times," or in line 112, it is unclear what $\Delta$ in depth-$\Delta$ refers to). These issues make the technical part of the paper especially hard to follow.

- **Datasets assumptions**: Even though there are 27 datasets, Jolicoeur-Martineau et al. (2024b) carried out pre-processing steps, such as encoding the categorical features as one-hot vectors, that have implications for the experimental results. Therefore, an overview of the datasets in the appendix with this information would have been helpful.

- **Poor abstract**: The abstract fails to clearly communicate the paper's contributions and methodology.

---

confidence: 4
contribution: 1
initial_rating: 3
presentation: 3
questions: How is your approach to regression sampling different from using a well-chosen discretization in combination with a single classification model?
soundness: 3
strengths: - Simple to understand
summary: This paper proposes to generate tabular data using discriminative models by posing the generation as a sampling from regressors/classificators one feature at a time, using multiple permutations to allow features to be conditioned on different sets of other features.


EDIT: I have read the authors response, and stay with my score. Thanks for pointing out the baselines that I missed and for pointing out that simplicity really is not a downside, which I fully agree with. What I wanted to say is that the proposed method is not very novel and to my understanding the thing that people already do when generating data with discriminatory models. The tree for regression models of course is novel, but I would believe that models that do not work discriminatory will win this in the medium-long run.
weaknesses: - Unclear how the proposed hierarchical handling of regression sampling is better than a simple discretized approach with well chosen bins
- The idea is very simple and low-effort in terms of putting it to work
EDIT: I want to take the above weakness back.
- The results are not great. The tables feel a little to designed: boldening non-standard things in the table, i.e. the winner of a subset of the methods or the 2 best methods.


I think if one wants to make a scientific contribution at ICLR level it would be the right thing to train a model that generates data for filling in in one forward pass, instead of relying on traditional discriminative models only. Maybe this is worse, but then it would be an important ablation.

---

confidence: 3
contribution: 3
initial_rating: 5
presentation: 2
questions: See weaknesses.
soundness: 3
strengths: 1. UnmaskingTrees achieves superior imputation performance, outperforming several popular benchmarks such as MissForest and ForestDiffusion on various metrics.

2. The autoregressive unmasking approach is innovative and avoids the pitfalls of traditional diffusion models for tabular data, including the usage of TabPFN for regression.


3. The proposed model reduces training and inference time by only requiring predictions along a single path in the meta-tree, compared to multiple models in diffusion-based approaches.

4. Extensive benchmarks across 27 datasets with multiple metrics validate the model’s effectiveness across various imputation and generation tasks. And the authors provide code for experiments.
summary: The paper introduces **UnmaskingTrees**, a novel method for tabular data imputation and generation, leveraging gradient-boosted decision trees. The approach focuses on sequentially “unmasking” individual features using tree models, enabling state-of-the-art imputation performance and competitive generation results. The paper includes experimental results on 27 datasets, showcasing the superior performance of UnmaskingTrees in imputation tasks compared to established methods like MissForest and ForestDiffusion. Additionally, the paper explores the potential of **TabPFN** as a base learner within the UnmaskingTrees framework, demonstrating its flexibility for probabilistic prediction and generative modeling.
weaknesses: 1. The training and inference procedure is not clear in writing. Can the authors show a more precise algorithm?
2. The paper do limited ablation to show how the hyperparameters effect the results, like tree depth or duplication factor, especially for large datasets.
3. The results for each dataset in tabular data generation should be shown in the appendix. 
4. For tabular datasets the results are close to each other in value, so the comparison with mean rank of different methods can show the results more clear.

Meta-review: The paper introduces UnmaskingTrees, a method for tabular data imputation and generation using gradient-boosted decision trees. The approach leverages an autoregressive process where features are sequentially unmasked achieving state-of-the-art imputation performance. The method outperforms existing approaches like MissForest and ForestDiffusion, and is competitive in generating data with missing values. A key innovation is the integration of BaltoBot, a non-parametric method for conditional density estimation. This model offers computational efficiency, faster training, and inference compared to diffusion-based models. The authors provide comprehensive benchmarks across 27 datasets, but some aspects of the methodology (like handling large datasets and the impact of certain hyperparameters) need further exploration. The paper could benefit from clearer technical details and a deeper discussion of model limitations and assumptions. The reviewers also indicate that the proposed method is not very novel and follows the approach that people already do when generating data with discriminatory models. Although it may fall in the borderline range, overall I believe that this paper needs more work before it can be accepted for publication.

Decision: Reject