Benchmarking Missing Data Imputation Methods in Socioeconomic Surveys
Authors: Siyi Sun
David Antony Selby
Yunchuan Huang
Ayush Patnaik
Sebastian Josef Vollmer
Seth Flaxman
Anisoara Calinescu
Venue: Accepted by TMLR ()

Abstract: Missing data imputation is a core challenge in socioeconomic surveys, where data is often longitudinal, hierarchical, high-dimensional, not independent and identically distributed, and missing under complex mechanisms. Socioeconomic datasets like the Consumer Pyramids Household Survey (CPHS)-the largest continuous household survey in India since 2014, covering 174,000 households-highlight the importance of robust imputation, which can reduce survey costs, preserve statistical power, and enable timely policy analysis. This paper systematically evaluates these methods under three missingness mechanisms: missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR), across five missingness ratios ranging from 10% to 50%. We evaluate imputation performance on both continuous and categorical variables, assess the impact on downstream tasks, and compare the computational efficiency of each method. Our results indicate that classical machine learning methods such as MissForest and HyperImpute remain strong baselines with favorable trade-offs between accuracy and efficiency, while deep learning methods perform better under complex missingness patterns and higher missingness ratios, but face scalability challenges. We ran experiments on CPHS and multiple synthetic survey datasets, and found consistent patterns across them. Our framework aims to provide a reliable benchmark for structured socioeconomic surveys, and addresses the critical gap in reproducible, domain-specific evaluation of imputation methods. The open-source code is provided.

Published in Transactions Machine Learning Research (02/2026)
Benchmarking Missing Data Imputation Methods in Socioeconomic Surveys
Siyi Sun
styi. Siy.ssun@ss.oo oz. ub
Department of Computer Science
Universily of Oxford
David Antony Selby German Research Genter for Artificial Intelligncce
david antoogg.ellyy@dffi..d
Yunchuan Huang
Sunchuan.huanggeene uk
Department of Enginecring Science
Universily of Ouford
Ayush Patnaik XKDR Forum
wywshpatmaikkgmmmii..om
Sebastian Vollmer
sebustion.
German Research Center for Artificial Intellignnce
Seth Flaxman
Department of Computer Science University of Orford
Anisoara Calinescu
ari. www.ccalinesccuucs.coo oz.
Deparbment of Compuler Science University of Orford
Reviewed on OpenReview: https: // opwrrevee.. net/ forum? id= HLh 190hBu6
Abstract
Missing data imputation is core challenge in socioeconomic surveys, where data is often longitudinal, hierarchical, high-dimensional, not independent and identically distributed, and missing under complex mechanisms. Socioeconomic datasets like the Consumer Pyramids Household Survey (CPHS) the largest continuous household survey in India since 2014, covering 174,000 households-highhe the importance of robust imputation, which reduce survey costs, preserve statistical power, and enable timely policy analysis. This paper systematically evaluates these methods under three missingness mechanisms: missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR), across five missingness ralios ranging from 10% to 50%. We evaluate imputation performance on both continuous and categorical variables, assess the impact on downstream tasks, and compare the computational efficiency of each method. Our results indicate that

---

Published in Transactions Machine Learning Research (02/2026)
classical machine learning methods such as MissForest and Hyperlmpute remain strong baselines with favorable tradc-offs between accuracy and efficiency, while deep learning methods perform better under complex missingness patterns and higher missingness ratios, but face scalability challenges. We ran experiments on CPHS and multiple synthetic survey datasets, and found consistent patterns across them. Our framework aims to provide a reliable benchmark for structured socioeconomic surveys, and addresses the critical gap in reproducible, domain-specific evaluation of imputation methods. The open-source code is provided in Appendix A.2
1 Introduction
Missing data is a pervasive challenge in data science and machine learning, especially in real-world
socioeconomic survey datasets (Silva-aamirez et al. 2015 Wang et al. 2021 Data is often
incomplete due to nonresponse or privacy concorns (Rubin
2004h. Imputation mitigates nonresponse
bias and supports policy evaluation (Chen & Shao 2000
Little & Rubin 2019 Yang et al. 2024
Abdelnaby al. 2024
Despite the proliferation of imputation methods, there is conspicuous lack of benchmarks to evaluate them on publicly available, large-scale, realistic datasets that capture the complexity of real-world socioeconomic survey data while allowing controlled introduction of missingness. Most empirical studies on missing data rely on relatively small datasets such as the UC1 machine
lcarning repository (Zhang et al. 2025 Du et al. 2024 Miao et al. 2023 Bertsimas et al., 2018b) or limited clinical datasets (Zheng & Charoenphakder 2022)-11 or on synthetically generated data with simplistic assumptions (e.g., features drawn from a standard normal distribution) (Sun al. 2023 Missingness is often simulated by randomly masking data under MCAR MAR assumptions, which fail to reflect complex real-world patterns. In practice, missingness often follows the more challenging MNAR mechanism, where whether value is missing depends directly on its unobserved value. Furthermore, numerous current benchmarks emphasize exclusively the accuracy of imputation, specifically evaluating the proximity of the imputed values to the actual values, while neglecting the consequences on downstream tasks (Zhang et al. 2025 Jarrett et al., 2022 Hastie et al. 2015 Bicssmann et al.) 2019). In practical applications, the goal of imputation is usually to enable reliable analysis or predictive modeling; thus, evaluating how different imputation methods affect the performance of subsequent tasks is crucial.
Although benchmarks exist for imputation on sociocconomic survey data Wang et al. 2021 Li et al. 2024 Kalton & Kasprzyk 1982), they typically suffer from several fimitations, such as exel excluding MNAR scenarios, relying on a small set of missingness ratios (Bertsimas et al. 2018b), and lacking a systematic evaluation framework. To address these gaps, our work bridges the divide between restricted real-world data and reproducible experimentation by introducing a comprehensive and open benchmark for missing data imputation in socioeconomic surveys. To the best of our knowledge, we are the first to provide large-scale, publicly shareable benchmark that integrates real, synthetic, and open socioeconomic datasets under diverse missingness scenarios and systematic evaluation metrics. Our contributions include the following:
Evaluations on Real, Synthetic, and Public Datasets: We benchmark imputation
methods on three datascts: the real-world CPHS (Pais & Rawal 2021), its high-fidelity
2

---

Published in Transactions Machine Learning Research (02/2026)
synthetic counterpart SynthCPHS, and the publicly shareable SubSDIC derived from the World Bank's SDIC.
Comprehensive Missingness Scenarios: We evaluate 14 imputation methods under three missingness mechanisms (MCAR, MAR, MNAR) and five missingness ratios, offering broad and realistic spectrum of evaluation conditions.
Multi-metric Analysis Downstream Task Evaluation: In addition to the imputation accuracy on both continuous and categorical variables, we assess performance on downstream classification and regression tasks using multiple models to ensure robustness. We also systematically compare the computational elficiency of each method.
The remainder of this paper is organized as follows. Section reviews related work on benchmark datasets, imputation methodologies, and syuthetic data. Section introduces the datasets used in study. Section 4 defines the problem and missingness mechanisms. Section 5 describes experimental setup and evaluation protocols. Section 6 presents results and analysis. Finally, Section 7 concludes the paper.
2
Related Work
2.1 Benchmark Datasets for Tabular Imputation
Research on imputation for tabular data often uses small, flat datasets like those from UCI machine lcarning repository (Kelly et al. 2025). As shown in Table 1 these datasets usually contain a few thousand to 100k samples with dozens features, lacking clear hierarchical or temporal dependencies between variables. Details of these datasets are Appendix A.4 Even more limiting, most studies simulate missing data, focusing on simplified MCAR or MAR scenarios with a single missingness
level, which limits generalizability, since real-world data have more complex patterns.
A notable exception is the work of Jäger et al. 2002)), who conducted large-scale benchmark of imputation methods across 69 heterogeneous datasets from OpenML. Although their study offers valuable insight into general-purpose imputation performance, the datasets used are not drawn from the socioeconomic survey domain and lack the hierarchical and longitudinal structures typically present in national houschold surveys.
Recently, synthetic data benchmarks (Sun et al. 2023) have gained traction to test imputation algorithms under controlled conditions (e.g., varying missing rates mechanisms); but most synthetic setups do not capture the complexity of real-world structured data. In particular, few, if any, existing benchmarks replicate the large-scale, multi-level characteristics of national socioeconomic surveys, which span millions of entries with state or regional hierarchies and repeated observations over time.
2.2 Imputation Methodologies
Approaches to imputing missing values can be grouped by their modeling philosophy. Statistical and itcrative methods use repeated cstimation cycles, including mean or mode filling, and classic algorithms like MICE Van Buuren
2011] and MissForest (Stekhoven Bühlmann 2012) that iteratively train predictors for each feature, as well matrix-complltion methods like SoftImpute (Hastic et al. 2015). These methods assume lincar or low-rank structures
Soothhui--udshoone
3

---

Published in Transactions Machine Learning Research (02/2026)
Table 1: Comparison of our proposed socioeconomic benchmarks against existing datasets commonly
used in imputation literature (cited in 2 3 studies). Columns Definition: Samples: Total number of rows; Features: Number of continuous (Noon) and categorical (Neat) variables; Evaluated Scenarios: The number of different missingness ratios tested under MCAR, MAR, and MNAR mechanisms; Capabilities: Whether the benchmark supports Train Test split evaluation, Downstream task ovaluation, and contains hierarchical or longitudinal structures. Unlike standard UCI datasets (lop), our proposed datasets (boltom) caplure the complex, structured nature of real-world survey data.
Dataset:
Samples
Fratures
Evaluated Socnarios (Nratios)
Benehmark Capahilitles
(N) Naw
Nes
MCAR
MAR
MNAR
Train/
Dowestroum
Hierarchiea Langitadinal
Bristing Generul- Generul-Parpore Purpose Benchmarkk (UCI)
Culifornia Housing
2ilk
11
Letler Recogailion
2llk
1
1
Covlit Card Clients
3ilk
1
Oulice News
4ilk
1
1
V
Concrete Strongth
4
Wine Quality
5k
4
4
Disbetee Hoalth
2ük
14
4
SpanBaze
4k
4
Propused Sacioeconumiic Benchmarrke
CPHS (Real)
1.4
5
5
SyuthCPHS (Synthetic)
1M
5
SubSDIC (Public)
500k
0
12
and often struggle with complex nonlinear feature interactions. Recent methods such as MIRACLE Kyono et al. (02T) introduce causally aware regularization that models the missingness mechanism jointly with the data, encouraging imputations consistent with the underlying causal structure. methods align observed and imputed distributions: MOT (Missing- Optimal Transport) (Muzellec et al. 2020 formulates imputation finding the allocation of missing values that minimizes the optimal transport distance between batches of incomplete data, while its successor TDM (Transformed Distribution Matching) (Zhao et al., 2023) learns a nonlinear mapping before applying optimal transport to better caplure the data's 's intrinsic geometry. These methods achieve saate-of-the-an accuracy on many benchmark tasks and are particularly effective for in-sample imputation but generalize poorly to new records since they treat missing entries as learned model parameters.
Distribution-matchinn
Deep generative models (VAE (Mattei & Frellsen 2019), GAN Yoon al. 2018 diffusion Zheng & Charoenphakdes 2022 capture joint distributions of observed and missing data. While these models can capture complex nonlinear dependencies, they often face challenges in estimating distributions from incomplete data and in performing conditional inference, especially under high missingness. To overcome these issues, recent methods combine generation with iterative refinement. For example, DiffPuter (Zhang al., 2025) integrates diffusion models into an EM franework, using iterative E- and M-steps to improve imputation quality.
Hybrid deep learning methods blends machine learning pipelines with automated model selection specialized architectures. HyperImpute (Jarrett et al. 2022) employs an AutoML-style pipeline that selects the best model for each variable and updates imputations iteratively. Other architectures leverage advanced designs: DSAN (Lee & Kim 2023) applics self-attention to learn feature and sample dependencies via masked reconst ruction, while ReMasker (Du al. 20244) extends masked autoencoding by re-masking observed entries during training, promoting robustness across different missingness patterns. Deep learning models have demonstrated robust performance challenging
4

---

Published in Transactions Machine Learning Research (02/2026)
imputation tasks, especially when missing rates are high or feature types are heterogeneous (Zhang
ct al. 2025
Some recent. studies have empirically evaluated the "impute-then-predie" pipeline. For example, Bertsimas et al. (2018a) and Poulos & Valle (2018) proposed frameworks that highlight the importance
of integrating imputation with supervised learning tasks. More recent work by Paterakis et al. (2024) questions whether explicit imputation always necessary in predictive pipelines, particularly within the context of AutoML. However, our work places equal emphasis on two complementary objectives: restoring incomplete datasets to reduce the need for costly follow-up surveys, and ensuring robust performance downstream tasks.
2.3 Synthetic Data for Imputation
When real-world sociocconomic data private or lacks ground truth, synthetic data provides a practical alternative for benchmarking imputation methods. Prior work has used synthetic simulations to evaluate methods under controlled conditions (Kyono et al. 2021 Sun et al. 2023 , but most rely on simple i.i.d. data or low-dimensional toy scttings (Muzellec et al. 2020 Bertsimas et al.
2024 lacking the structural and statistical complexity of real surveys. National household surveys like CPHS feature multi-level hierarchies, repeated measurements, and non-random missingness, but their proprietary nature limits open evaluation. To address this, we introduce a synthetic benchmark dataset, SynthCPHS, which we design replicate the structure and distribution of CPHS. We validate the similarity using the Caammogoro--Smmnng (KS) test and Censen-Shammnn (JS) divergence, and the synthetic construction enables GPU-supported evaluation beyond CPHS's secure CPU-only server environment. We also present SubSDIC, public subset derived from World Bank data (World Bank 2023), which supports systematic and reproducible benchmarking the socioeconomic domain.
3 Datasets
3.1 CPHS & The SynthCPHS Dataset
3.1.1
CPHS
The Consumer Pyramids Household Survey (CPHS) by CMIE the largest continuous household survey in India, running since 2014 and covering a panel of over 174,000 sample houses (about 111,000 rural and 63,400 urban) spread across most states in India surveyed thrice yearly (Pais & Rawal 2021 Somanchi 2021 It captures a broad array of household attributes including labor supply, income, consumption (expenditure on various needs), borrowing, and asset ownership (Pais & Rawal 2021). This breadth makes it highly valuable for socioeconomic analysis (Chatterjee & Dev 2023 Kathuria & Dev 2024 Jagannarayan & Prasunal 2024). In this paper, we select 25 socioeconomic features, as shown in Figure]) 1]. Because some variables were missing in earlier waves, we restrict the analysis to complete cases from waves 18 30 (each wave is survey round), preserving the longitudinal and hicrarchical structure and yiclding 1,341,651 records.
5

---

Published in Transactions Machine Learning Research (02/2020
Survey
Demographics
Honschold
WAYE Number
Age
Total income
Gender
Monthly income of member from all
Personal Assets
Whether member has bask
Monthly household from sources
member
mobille phone or not
House Demographics
Health
Activity Time
STATE
Statos of member
Work
hospiralizod bast days
Homogencous region
sports
Statos of member heing
Region type
Travel
Overview of selected features in the CPHS and SynthCPHS datasets. Household-leve (orange) and individuua-level (blue) features form hierarchical structure. The target variable (green) indicates individual health status for downstream classification. The wave number indexes each survey round, capturing longitudinal structure.
Figure 1:
3.1.2 SynthCPHS
As CPHS is proprietary, available only by subscription, and restricted to secure CPU-only server, benchmarking is severely constrained, especially for GPU-based imputation. To address this, we construct SynthCPHS, a synthetic dataset that mirrors the statistical properties and structure of CPHS but can be used on external GPU systems, enabling full-scale evaluation imputation accuracy and efficiency. SynthCPHS includes 1, 1,000,000 records and shares the same feature sets with the CPHS dataset as shown in Figure ! The dataset was generated using the synthpop package in R (Nowok et al. 2016), widely used tool to produce artificial microdata that preserve the statistical properties of the original survey while protecting individual confidentiality (Nowok al. 2017 To support its validity, we compared the marginal and joint distributions of key features in SynthCPHS and CPHS using the Kolmogorov Smirnov (KS) test and Jensen Shannon (JS) divergence, finding no significant distributional differences.
3.2 SubSDIC
To ensure reproducibility, we introduce SubSDIC, public subset dataset derived from the World Bank's s Syuthetic Data for an Imaginary Country (SDIC)-a fully synthetic census dataset representing an imaginary middle-income country. SDIC was generated using REaLTabFormer (Solatorio & Dupriez, 2023 deep generative model trained on global household survey data, including IPUMS International, DHS, and the World Bank Global Consumption Database. SDIC consists of two flat tables for household- and individual-level attributes. We join these tables via household ID, select 19 mixed-type variables spanning both levels, and randomly sample 500k records from the full 10 million to construct SubSDIC. We designate the individual's highest oducational attainment (cat (cat_educ__at aeduc_attain) as the targel. variable for rownstream classification and years of schooling _on_yrs_school) the target variable for downstream regression. SubSDIC preserves realistic sociocconomic frameworks, including hicrarchical relationships among houscholds, individuals,
6

---

Published in Transactions Machine Learning Research (02/2026)
provinces, and districts, allowing for regulated evaluation of imputation accuracy, efficiency, and downstream performance across different missing data scenarios. Kolmogorov-Smirno tests and Jensen- divergence analysis confirm that SubSDIC closely mimics the marginal distributions of the original SDTC dataset. Detailed feature descriptions are provided in Appendix A.5
4 Problem Definition and Missing Mechanism
4.1 Problem Definition
Let X denote the matrix n x d that contains the complete data values in the variables d for all n units in the sample. Define the mask variable M as an n x d 0-1 matrix indicating whether data point of X observed (1) or missing (0). The elements of X and M are denoted by Diy and my, respectively, where i I...., and j = l,..,. d. We further define the partially observed data matrix X, and its elements Tij; such that
{ Fijs if Mij
Tay
0, if myj
Here 0 represents an unobserved value. In the missing data imputation problem, the task is imputing data matrix X from the observed data matrix X and make it as similar as possible to the complete data matrix X.
4.2 Missingness Mechanism
Little & Rubin (2019) find it helpful to differentiate between the missingness mechanisms, which refers to the relationship between the occurrence of missing data and the values of the variables the data matrix. The missing mechanism indicates whether the occurrence of missingness connected to the underlying values of the variables the dataset. The importance of missingness mechanisms lies in the fact that the effectiveness of data imputation methods is highly influenced by the specific dependencies present in these mechanisms. Therefore, we introduce the three missingness mechanisms defined by Rubin (1976) here. Let X and M be defined in Section 4.1 Assume the rows (x; m.) are i.i.d. across i. The missingness mechanism is specified by the conditional distribution PM x (m; X;, 0), where 0 denotes unknown parameters.
If the missingness is independent of the data values, either missing or observed, this means for all and any distinct values x, in the sample space of X, the conditional distributions are equal:
MCAR:
PMIX (m; x2, 0) = PM (m; 0)
where x serves as a placcholder for distinct, hypothetical value that could take the place of x; in the sample space of X.
MAR: Lot X(1); represent the observed components of x; and X represent the missing components of x,. A less restrictive assumption than MCAR is that the missingness depends on x; only through the observed components X(1)-- This implies that for any distinct values (x(o),, Koye of the missing components within the sample space of X(ojs the probability of
7

---

Published in Transactions Machine Learning Research (02/2026)
missingness remains the same. In mathematical terms, the conditional distribution of the missingness mechanism can be expressed as:
PM) X(o)s, (1)ss 0) = PMX (m; X(0)a> ( 0)
(1)
Unlike the MAR mechanism, where missingness is related to the observed values, if missingness is dependent on the unobserved values, the mechanism is classified as MNAR. The distribution of m, depends on the missing components X,, which means that equation [
MNAR:
is not valid for some units and some values (X(o)e: x(o)) of the missing components.
4.3 Missingness Generation
In this study, missing values are synthetically introduced into complete datasets to establish ground- truth benchmarks. We adapt procedures from the t-miss-tastic platform (Mayer et al.) 2019 to generate missingness under MCAR, MAR, and MNAR mechanisms. Crucially, the generation process is fully independent for every experimental configuration; a new missingness mask is generated for each sample dataset.
Missing Completely at Random (MCAR) Missingness is independent of observed and unobserved data. For each feature and sample i, the missingness indicator maj follows a Bernoulli distribution P(mig = = where is the target proportion.
Missing at Random (MAR) Missingness in feature Xj depends on other observed features but not on Xj itself. We employ feature-wise logistic model outlined in Algorithm 1 To strictly adhere to the MAR assumption, the weight matrix W is constructed with a zero diagonal (W,, 0). The influencing covariates are assigned uniform weights 83 = 1/(d- 1) for all & + in a d-dimensional dataset, ensuring bias-free dependency on the remaining features.
Missing Not at Random (MNAR) MNAR allows missingness to depend on the feature's own value. As shown in Algorithm 1 this is achieved by initializing W as an all-ones matrix (retaining Wjj = 1) and normalizing the weights row-wise to Aa = 1/d, thereby allowing the missingness of a variable to depend on all features, including itself.
Distributional Bias and Extrapolation. The logistic missingness generation mechanism (Algo- rithm ! inherently introduces distributional bias by assigning higher missingness probabilities to extreme values. This simulates tail censoring, aligning with empirical findings in survey methodol- ogy where high- and low cow-income households are significantly less likely to disclose earnings (Riphahn
& Serfling 2005 Meyer al. 2015).
5
Experimental Setup and Evaluation
5.1 Dataset Distribution Comparison
We evaluate how closely the synthetic dataset (SynthCPHS) reproduces the marginal distributions of the real CPHS across continuous and categorical variables.
8

---

Published in Transactions Machine Learning Research (02/2026)
Algorithm 1: Generation of Missing Values under MAR/MNAR Mechanism
Input : :Complete Matrix X (n d), Missingness Ratio a, Mechanism "MAhanism='MAR" "MNAR'
Output: Partially Observed Data Matrix X // Initialize d xd matrix with ones
// Initialization
X X; Wdxd 11
// Step 1: Define Dependency Structure
2 Mechanism== "MARR then
3 W W Wa
// Remove self self-dependency // ensure Σn W j.k
W Normalize( W, axis row)
// Step 2: Generate Missingness
S XWT;
// Compute weighted scores (n d)
Z <- Normalize(S, axis col): ;
// Standardize scores feature-wise
7 Determine vector y Rd such that mc + y), axis col) R a;
P < (Z +7);
// Compute probability matrix (n d)
Generate M from Bernoulli distribution with parameter P // Mask matrix (1 Missing)
// Step 3: Post post-psocessin Safety Check
10 Find indices I + (i د M.,j = =d};
// Find rows that are fully missing
11 if I is not emply then
Sample indices k E {I....,,d) dل uniformly;
13
M[I,K] + 0;
// Ensure at least one observed value per row // Apply final mask to data
14 [ < NA;
15 return X
Method for continuous variables: two-sample KS test. Let {a.]2,, and {yz h be two i.i.d. samples with empirical CDFs F, and Gm. The two-sample Kolmogorov Smirnov statistic is
Dn,m
Fup Gm(())/
2
zER
Under the mull hypothesis that both samples from the same continuous distribution, mmm
m Dn.m converges in distribution to the Kolmogorov distribution, which yields exact or asymptotic p-values Massey Jr 1951). In all our results we report the unscaled statistic Da.m in Eq. (2), denoted as "KS" in the figures, together with its two-sided p-value.
Method for categorical variables: JS divergence. For two discrete distributions and Q over the same support X, the JS divergence the symmetrised, smoothed version the Kullback-Leibler (KI.) divergence:
JS(PIIQ)= KL(P)|M) + KL(O) M = (P+
(3)
P(a) where KL(PIIQ) = Erex P(x) log q(a)" JS is always finite, bounded between 0 (identical distributions) and log (base-e), and admits a metric square root (Lin 2002 Endres & Schindelin 2003). We report values in [0, by dividing by log 2.
=
9

---

Published in Transactions Machine Learning Research (02/2026)
5.2
Benchmark Experimental Settings
5.2.1
Missingness Mechanism and Ratio
The effectiveness of missing data imputation methods is strongly influenced by factors such the missingness mechanism and ratio. To rigorously evaluate imputation methods, introduce missing values under 3 missingness mechanisms: MCAR, MAR, and MNAR. The missingness implementation details are provided in Section 4.3 We create versions of the datasct with missingness ratios of 10,, 20%, 30%, 40%, and 50%. The missingness ratio is calculated as the fraction of all entries that are masked, and each feature gets roughly the same fraction of its values missing, though in MAR/MNAR /MAR this can vary slightly due to the conditioning. Each missing scenario is generated with five samples and then fixed, so all methods are evaluated on the exact same missing data patterns for fairness.
5.2.2 Imputation Methods
We provide a comprehensive benchmark of 14 widely used imputation methods across four categories: (1) statistical bascline Mcan/Mode Mode imputation; (2) distribution-matching methods such as MOT
M uzellec et al., 2020), which uses optimal transport to align observed and imputed distributions
3 iterative machine learning methods including MICE [Van Buuren & Groothuis Oudshoorn
2011p, MIRACLE (Kyono et al.) 2021 b. Soft (Hastie et al., [2015), and MissForest
(Stekhoven & Biihlmann 2012); and (4) deep generative models - MIWAE (VAE) (Mattei &
Frellsen 2019). GAIN (GAN) (Yoon et al. 2018 DSAN (self-attention) Lee & Kim 2023), and
TabCSDI (diffusion) [Zhng & Charoenphakdes 2022 p. We also include three recent state-of-the-art
approaches: ReMasker (Du et al.J (2024), Imputer (Jarrett et al.] 2022), and DiffP utter
Zhang et al. 2025). along with DSAN variant (DSN) without attention assess its contribution.
Implementation details and hyperparameters are in Appendix A.8
5.2.3 Imputation Performance
For each dataset with missingness, 80% of samples are used for training and 20% for testing. All methods are trained on the training set and then used to impute both in-sample and out-of- sample data. Imputation performance is measured using RMSS for continuous and F1 score for categorical variables to provide balanced evaluation given class imbalance. The RMSE is computed on standardized inputs (zero mean, unit variance) based training-sel statistics, and accuracy for categorical variables is also reported as a supplementary metric.
5.2.4 Downstream Task Performance
To robustly assess the downstream impact of imputation, we test two task types: classification and regression. For classification, all three datasets are evaluated with Logistic Regression, Random Forest (RF), XGBoost and LightGBM models; for regression, only SubSDIC is used, with Linear Regression, Random Forest (RF), XGBoost and LightGBM models. Using multiple models reduces model-specific bias. Models are trained on complete training data and evaluated on imputed test sets. For classification, we report the ROC-AUC degradation the drop ROC-AUC from the fully observed test set the main metric, while accuracy is provided in the supplement. For regression, we report the RMSE increase, the percentage risc in RMSE relative to the fully
10

---

Published in Transactions Machine Learning Research (02/2026)
observed test Smaller ROC-AUC degradation or RMSE increase indicates better imputation quality and stronger prescrvation of predictive signal.
5.2.5 Runtime and Efficiency
To provide practical insight for real-world deployment under resource constraints, we report the total wall-clock time for each method, including both training and imputation. For iterative algorithms such as MICE and MissForest, time covers all iterations; for deep learning models, includes all training epochs. Appendix [A.3] details the experimental setup. Each experiment is repeated five times, and we report mean values and standard deviations as final metries.
5.3 Ranking Consistency
5.3.1
Across Datasets
We assess cross-datasel performance consistency using Kendall's coefficient of concordance W as described by Abdi 2007). computed over the mothods common all datasets. Although the full benchmark includes 14 methods, DiffPuter could not be executed on the proprietary CPHS dataset hosted CPU-only server and is therefore excluded from the consistency analysis. For l datasets and N methods, let. Rı be the rank of method on dataset, j, RS = Σ5= Rij the aggregate rank, and R = k(N+1) Then
N
13
W 62 N)
R
Rj
(4)
W E 1] (0- no agreement; perfect concordance). For A > 2, k(N - 1)W is asymptotically
X under independence (Abdi 2007). In our study, 1 = datascts and N = 13 methods. We use common thresholds: strong IW > 0.70), moderate (0.50 S W S 0.70), and weak (W < 0.50)
agreement (De Maere et al. 2022).
5.3.2 Across Downstream Task Models
To quantify the ranking consistency of imputation methods' rankings across different downstream models, we employ Kendall's coefficient of concordance W, defined in Eq. g This metric enables us to assess whether the relative ordering of imputation methods remains stable across downstream lasks and model families, independent of the absolute performance values.
6 Results and Analysis
This section begins with distribution comparison results. We then present multi-metric benchmark of 14 imputation methods, covering imputation performance, downstream task performance, and computational efficiency. In all figures, the numbers after method names in the legend indicate the average rank of that metric across missingness ratios (shown for the top nine methods only). Full hyperparamoter scttings and complete results for all datasets and motrics are provided in A Appendix A.8 and in the supplementary material. Finally, we quantify the performance consistency between datasets using the Kendall coefficient of concordance computed on the 13 methods available all datasets.
11

---

Published in Transactions Machine Learning Research (02/2026)
6.1
Distribution Comparison Results
con con_WAVE_NG
con BAe_AGE__V YRS
14
Cunuletive Probabilly
36
04
04
04
KS = 0.0004 0.9999
KS 0.0007 p 0.9362
True Data
True Data
Syrthetic Data
Synthetic Data
Value
Value
con T8, ON WORK FOR EMPLOYER
con TB ON TRAVEL
True Data
Synthetic Data Data
Cumutetive Probabiltty
KS = 00.0005
KS 0.0005
p
True Data
Synthetic Data
Value
Value
Figure 2: Empirical CDF comparison between CPHS (red) and SynthCPHS (blue) for four represen- tative continuous variables. The double-headed arrow indicates the KS gap Da,m-
Continuous variables (KS). For each variable we compute the two-sample KS statistic Da,m in Eq. (2, comparing CPHS and SynthCPHS. The KS values displayed Fig. 2 are exactly this maximum CDF gap Da,m- In most cases Dn,m < 10-3 and the correspond- ing two-sided p-values are close 1, providing no evidence against the mull of identical distributions.
Categorical variables (JS). For each categorical variable compute the normalized JS(P11() where JS(PIIQ) is defined in Eq.(3) and we simply rescale it to 1]. Across all examined variables, JS (to mumerical precision), indicating identical empirical distributions between CPHS and SynthCPHS; therefore, we omit plots.
JS divergence, defined as JS Tog?
Detailed per-variable values for both continuous and categorical features are provided in the supplementary material.
6.2 Imputation Performance
Figures and compare in-sample and out-of-sample performance across methods for continuous (RMSE) and categorical (F1 score) variables, respectively. We observe four key findings:
12

---

Published in Transactions Machine Learning Research (02/2026)
MCAR
MAR
MNAR
Average
In-sample RMSE
1.1
1.4
1.0
1.4
1.2
0.9
1.2
0.8
1.2
1.0
0.6-
Out- Out-of-sample RMSE
1.4
1.2
1.0
1.2
1.0
0.8
1.0-
1.0
0.8
0.1
0.2
0.3
0.4
0.5
0.1
0.2
0.3
0.4
0.5
0.1
0.2
0.3
0.4
0.5
0.3
0.4
0.5
Missing ratios
Missing ratios
Missing ratios
Missing ratios
Mean 8/ 8/@
MissForestt/@3)
DSAND/2 DSN
Hyperlmpute@//
DiffPuterS/S
MIWAE
MICE$/0
GAIN
Softimpute
TabCSDi0/2
Remaskera/@
Figure 3: Contimuous variable imputation performance. Top row: in-sample; bottom out-of- sample. Lower RMSE is better. MIRACLE and MOT are omitted due to excessively high RMSE. Legend entries show in-sanple/ouut-o--aaae sample ranks (first/second number).
MAR
Average
Out-of-sample F1 Score In-sample F1 Score
0.5
0.5
0.4
0.4
0.4
0.4
0.3-
0.2
0.2
0.2
0.1
0.5
0.5
0.4
0.3-
0.3
0.2
0.1
0.2
0.1
0.0
0.0
0.0
0.3
0.2
0.3
0.5
0.2
0.3
Missing ratios
Missing ratios
Missing ratios
Missing ratios
Mode MICE
MissForestow/D
DSANX/0 DSNS/RR
Hypertmputess//s
MDT
DiffPuter///g
MIWAE Remasker2/22
GAINS/S
MIRACLE
Softimputc
TabCSDI
Figure 4: Categorical variable imputation performance. Top row: in-sample; bottom row: out-of- sample. Higher F1 is better. MIRACLE in-sample points are missing at some missingness ratios due to failure. Legend entries show ofapple/out-o-saaneee ranks (firs//second number).
13

---

Published in Transactions Machine Learning Research (02/2026)
Missingness Mechanism Impact. MCAR yields lower error (lower RMSE and higher F1) than MAR MNAR, reflecting the challenge of imputing structured missingness. For MAR and MNAR, RMSE does not always increase with higher missingness ratios one might typically expect.
Specifically, for methods like Mean, SoftImpute, and MIWAE, RMSE decreases as missing- ness increases. As detailed in Appendix A.6, empirically verified that this is due to the dilution of extreme values inherent in the logistic missingness generation mechanism (Algorithm T]: at low missingness ratios (e.g., 10%), the mechanism selectively masks outlicrs (which are highly erroncous to impute with the mcan); however, at high ratios (o.g., 50%), the missing set expands to include many data points near the global mean, thereby reducing the average RMSE.
In contrast, F1 scores for categorical data typically exhibit two trends: either remaining nearly constant, indicating a failure to learn patterns, or decreasing linearly, An exception observed in DSAN and variant DSN, where F1 scores drop sharply under high missingness, suggesting instability in such settings.
Method Comparison. For continuous variables (Figure 3 MissForest typically performs best at low missingness ralios (e.g., 10%) but its performance steadily declines as the missingness ratio increases. In contrast, deep learning methods such as ReMasker, DSAN, and DSN outperform MissForest under MCAR, MAR, and MNAR when the missingness ratio is high. For categorical variables (Figure D. MissForest continues to outperform other models al low missingness ratios regardless of the missingness mechanism. However, as the missingness increases, ReMasker and Buter begin to achieve the highest F1 scores. DSAN and DSN, while strong performers for continuous variables, show noticcably worse performance categorical data, with F1 scores degrading significantly as missingness increases. In summary, MissForest is highly sensitive to the missingness ratio rather than to the type of missingness or variable; it is among the top performers under low missingness (up to 20%), but deep learning methods often become more effective as missingness becomes more severe.
DSN outperforms DSAN in average ranking across most cases, suggesting limited benefit from the self-attention mechanism. In particular, as shown in Figure 3 while DSAN achieves the best in-sample performance for continuous imputation, DSN outperforms it on average in out-of-sample evaluations, indicating that the attention layer in DSAN introduces a higher risk of overfitting compared to DSN, a phenomenon also observed in a previous study by Dehimi Tolba (2024).
Analysis. As shown in Figures and 4
Overfitting Assessment. Out-of-sample RMSE and F1 scores closely match in-sample results, indicating minimal overfitting for most methods.
Failure Analysis: MIRACLE and MOT. A notable observation in our benchmark is the instability and high error rates of MIRACLE and MOT. We attribute these failures to three key factors:
MOT: Sinkhorn Instability and Geometric Mismatch. MOT relies on Sinkhorn iterations to approximate Wasscrstein distances. This approach is known to suffer from
14

---

Published in Transactions Machine Learning Research (02/2026)
numerical instability if the entropy regularization parameter (e) is not meticulously tuned for each dataset (Feydy et al. 2019 Furthermore, MOT relics on Euclidean distances in the
data space, which Zhao et al.
[2023 argue fails to capture the complex manifold structure
of tabular data, resulting in goometrically close but semantically wrong imputations.
MIRACLE: Optimization Complexity and Divergence. MIRACLE involves a multi- objective optimization problem involving causal DAG learning and moment matching regularizers. First, this introduces high hyperparameter sensitivity; as shown in Zhao et al. 2023 (e.g., their Appendix Figures 8-9), MIRACI.E. frequently yields "emply results" (execution failure) or extreme errors when default hyperparameters do not match the data scale. Second, MIRACLE relies on iterative refinement from a baseline (e.g., Mean imputation). In our datasets, initial baselines often have large errors (see Appendix A.6); this poor initialization can lead the causal graph learning astray, causing the refinement loop to diverge even under MAR mechanisms.
Assumption Violation in MNAR. Finally, specifically for MNAR, MIRA- CLE assumes the absence of self-masking missingness (Assumption in Kyono et al. 2021) ). Our MNAR settings inherently involve self-masking. Violating this assumption renders the moment regularizer invalid, further destabilizing the model.
MIRACLE:
6.3
Downstream Task Performance
MCAR
MAR
MNAR
Average
ROC-AUC Score
Degradation (%)
-5
-5
-5-
Classification
-10
-10
-10
-10-
-15
-15
-15
-20
-20
RMSE increase (%6)
125
150
125
150
125
150
125
Regression
100
100
100
100
75-
50
50
25
25
0.5
0.5
0.2
Missing ratios
Missing ratios
Missing ratios
Missing ratios
Me
Mis
DSANI/S
Hypprrimpute//2
MOT
DiffPuter
+ MIWAE Remaskenajja
MICEG/A
GAIN
DSNEN
MIRACLEW/-
Softimpute
TabCSDiyS
Figure 5: Top: ROC-AUC degradation for downstream classification using Random Forest. Bottom: RMSE increase for downstream regression using Random Forest. Lower ROC-AUC degradation and smaller RMSE increase indicate better preservation of predictive performance. Legend entries show classification cetoormannc//rerreeeeeeee performance ranks (firt/second number).
As shown in Figure 5 we report the downstream impact of imputation on both classification and regression tasks, using Random Forest models as illustrative examples. Unless otherwise specified,
15

---

Published in Transactions Machine Learning Research (02/2026)
downstream models are run with library default hyperparameters, with only minimal necessary adjustments (c.g.; setting class www.weee"aalaanneee of 42. All explicitly chosen hyperparameters are documented in the released scripts in our code repository; any parameter not listed there is kept at its default value.
for classification) and fixed random seed
Missingness Mechanism. The results show that missingness under MCAR and MAR leads to slightly lower performance degradation than MNAR. ROC-AUC scores decrease and RMSE increases with higher missingness ratios, highlighting the adverse effect of missing data on downstream performance.
Robustness Analysis. A clear alignment observed between raw imputation quality (RMSE/F1) and downstream results: methods with lower degradation also achieve top imputation scores. For instance, DSAN and DSN perform best on continuous variables, while ReMasker, MissForest, and DiffPuter excel in categorical imputation, all showing minimal downstream impact. Those top-performing imputation methods maintain their rankings in both regression and classification settings. In two other datasets, while some methods' rankings vary, ReMasker and Hyperlmpute consistently perform well across all datasets.
Sensitivity Analysis. Figure 5 reports the downstream performance using Random Forest models; similar trends are observed when we replace Random Forest with Lincar Regression, XGBoost, or LightGBM (see the "Experiment result" folder in the supplementary material). To more directly assess the stability of method rankings across different downstream models, we visualize the ranks of all 14 imputers using bump charts for regression and classification in Figure 6a and Figure 6b] respectively. Across Logistic/ Linear Regression, Random Forest, XGBoost, and Gig,, the curves in both bump charts are largely parallel, with only minor local crossings. Top-performing methods remain in the upper part of the ranking for all four models, while the weakest methods stay in the lower part, and no imputer exhibits dramatic shifts from top to bottom or vice versa.
We further quantify this consistency using Kendall's W defined in Eq. For the classification task, the rankings induced by 4 models yield a Kendall's W = 0.91; for the regression task, the rankings give an even higher Kendall's W = 0.95. These values indicate strong concordance among the four downstream models in both settings. Taken together, the bump charts and Kendall's W statistics suggest that our main conclusions about the relative performance of imputation methods are not sensitive to the particular choice of downstream model or task type.
6.4 Computational Efficiency
As shown in Figure 7 the runtime efficiency analysis indicates that the imputation time is relatively stable across varying missingness ratios and missing mechanisms for most methods. Statistical methods (e.g., Mean/Mode, Mode, MICE) and traditional machine learning methods (e.g., Hyperlmpute, MissForcst) demonstrate significantly faster performance, approximately an order of magnitude faster than deep learning- based methods. Among these methods, MissForest. stands out for its strong performance on continuous variables at low missingness ratios, solid categorical imputation accuracy, and exceptional time efficiency, making it well suited for large-scale practical applications.
16

---

Published in Transactions Machine Learning Research (02/2026)
SubSDIC Downstream Regression Ranking Bump Chart
SubSDIC Downstream Classification Ranking Bump
P2
0
dra ditputor
dkaa Oaan
detuuden
PS
00ta, Jomosser
0353, cirpuer
da3k, ryporrype
77
dara,mice
data
dstt_mice
dimurttiee
610
data 10bo0d
#10
e12
data MOT
#12
#13
#14
emuustampus
+14
Unoor Regnession
Randon Forost
XGBoost
UgMdaM
Logili Regnasiio
Rundon Foresl
XXBoool
LgniBBM
(a) Regression
(b) Classification
Figure SubSDIC Downstream Task Ranking Bump Chart. For regression, Kendall's W 0.9473 across models; and for classification, Kendall's W 0.9132 across models. The overall ordering is highly consistent across downst.ream models.
MCAR
MAR
MNAR
Average
Time (/seconds)
6000
6000
6000
6000
4000
4000
4000
4000
2000
2000
2000
2000
0
0.2
0.4
0.4
Missing ratios
Missing ratios
Missing ratios
Missing ratios
Mean/Mode + MissForest3 + DSAN --+ Hyperimpute ---- MOT
DifPutteR ........ MIWAE
MICE@
GAIN22
DSN MIRACLE2
Softlmpute
TabCSDI
Remasker
Figure 7: Comparison of imputation runtime.
For more complex missingness scenarios, ReMasker proves to be a powerful imputation method for both continuous and categorical data, offering competitive efficiency compared to other deep learning approaches and resulting in minimal degradation in downstream task performance.
6.5 Consistency Analysis
We evaluate the consistency of the rankings across the three datasets using Kendall's coefficient of concordance (W) described in Eq. (4.. As shown in Table 2 agreement is strong in scenarios A and D, moderate in B, and lower in C. Considering the generally consistent performance across scenarios, we center our discussion above on SubSDIC. This datasel publicly accessible, facilitates both classification and regression tasks, represents the other datasets well, and prevents repetitive reporting across all three.
17

---

Published in Transactions Machine Learning Research (02/2026)
Table 2: Ranking agreement across 3 datasets measured by Kendall's W (higher is better).
Scenario
W
Consensus
A: RMSE (numeric, out-of-sample)
0.857
Strong
B: Fl (categorical, out-of-sample)
0.629
Moderate
C: ROC-AUC degradation (RF)
0.411
Weak
D: Time efficiency
0.904
Strong
6.6 Analysis
Overall, by aggregating the average rankings of 14 impulation methods (13 on the CPHS dataset) on three missingness mechanisms and five missingness ratios, across three datasets and four representa- tive tasks (1) Imputation RMSE rank, (2) Imputation F1 rank, (3) downstream regression rank, and (4) downstream classification rank using Random Forest, we find that Hyperlmpute and MissForest consistently achieve the best overall performance with a clear margin over all other methods. This suggests that although deep learning based models such DSAN, DSN, and ReMasker perform competitively, traditional methods like Hyperlmpute and MissForest deliver more consistent and superior overall results. Our findings therefore reinforce the strong practical compelitiveness of traditional machine learning based methods for missing data imputation, consistent with prior studies (Lalande & Doya, 2022; Zhang et al.) 2025 Suh Song, 2023 Jolicoeur-Mertinen et al. 2024 Jäger et al.j (2021). Furthermore, our benchmark also suggests that incorporating attention layers may increase the risk of overfitting, in line with the observations of Dehimi & Tolba 2024
6.7 Practical Recommendations
Based on our benchmarking results, we offer the following guidelines for method selection:
Optimal Balance of Efficiency and Performance: For general applications, MissForest offers the best trade-off wetween accuracy and elficiency. It achieves superior raw imputation performance at low missingness ratios and demonstrates exceptional stability in downstream tasks. While HyperImpute also efficient and effective at low ratios, its downstream utility is less consistent compared to Fossso.est.
Prioritizing Imputation Performance: When prioritizing performance over computational cost, deep learning methods outperform basclines missingness increases (typically 30%):
Categorical Variables: At high ratios (2 40%), ReMasker achieves the highest Fl scores, followed by DiffPuter.
Continuous Variables: Under MCAR, ReMasker remains the top performer. However, for complex mechanisms (MAR/MNA) at high ratios, DSAN and DSN provide the most accurale ruction.
In summary, we suggest a hicrarchical strategy: use MissForest for low missingness (< 30%) due to its efficiency and robustness; switch deep learning beyond 30% when imputation performance is preferred. Specifically, ReMasker and DiffPuter are robust for general/categorical data, while DSAN and DSN excel in continuous imputation under complex patterns. Regarding downstream
18

---

Published in Transactions Machine Learning Research (02/2026)
predictive stability, MissForest, ReMasker, DtefPuter, DSAN, and DSN constitute the top-5 methods that most reliably preserve predictive signals. To ensure the robustness of these recommendations, we conducted rigorous statistical significance tests across varying scenarios; detailed analysis and p-values are provided in Appendix A.9.1
7
Conclusion
This work presents comprehensive benchmark study across three large-scale socioe- conomic survey datasets both real and synthetic that reflect key characteristics of the domain: longitudinal, hierarchical, large-scale, and non-iid. Using these datasets, we systematically evaluate diverse imputation methods under controlled missingness mechanisms, varying missingness ratios, and across continuous and categorical variables. Beyond imputation accuracy and downstream task performance, we also assess computational efficiency, providing a well-rounded evaluation of each method's practicality.
Conclusion:
Our results confirm the strong performance of classical approaches obscrved in prior studies, while emphasizing the value of multimetric evaluation, including downstream task impact and efficiency, for understanding real-world applicability. The proposed benchmark offers a realistic, robust testbed for missing data research structured sociocconomic contexts. By releasing the SubSDIC dataset and evaluation framework, we support reproducible research and foster progress in addressing complex missingness patterns in the survey domain.
Limitations & Future Work: While CPHS and SynthCPHS provide robust validation of our conclusions, third-party licensing restrictions prevent public dataset release. We note that certain baselines were evaluated using default hyperparameters, though we acknowledge this may conser- vatively estimate their potential. Graph-based imputers such as GRAPE and IGRM are excluded because their public implementations perform imputation in a learned graph-embedding space rather than the original feature space. Predictions are returned node and edge representations, and reconstruction losses (c.g., RMSE/MAE) are computed in the latent space. Our benchmark requires per-variable imputations in the original domain and trains downstream models on the reconstructed features. Making these methods compatible would require adding an extra layer that maps graph embeddings back to each column (including discrete variables), thereby violating our fixed-budget and fairness constraints. Finally, we recognize the substantial computational demands of comprehensive benchmarking, particularly for modern deep architectures. Future work will (1) release this framework as an open-source Python package and establish a community leaderboard on PapersWithCode for standardized evaluation, (2) develop prediction models based heterogencous graphs that might be better aligued with the structural properlies of socioeconomic survey data, and compare graph-based imputers within that study, and (3) continuously incorporate emerging imputation techniques.
Acknowledgments
The authors would like to acknowledge the use of the University of Oxford Advanced Rescarch Computing (ARC) facility carrying out this work (DOI: 10.5281 (5221/xenodo.22533.. This work was also supported by the German Federal Ministry of Research, Technology and Space (BMFTR) under project Eventful (grant number 01IW23005). SF supported by the EPSRC (grant
19

---

Published in Transactions Machine Learning Research (02/2026)
EP/V002910/2). AC acknowledges funding from a UKRI AI World Leading Researcher Fellowship (grant EP EP/W002949/1).
References
Taher Abdelnaby, Tingyu Feng, Zhang Tiantian, Xiaoming Jiang, Wang Yuming, Zhaojie Li, and Changhu Xue. Impact of frozen storage physicochemical parameters and quality changes in cooked crayfish. Heliyon, 10(11), 2024.
Hervé Abdi. The Kendall rank correlation coefficient. Encyclopedia of measurement and statistics, 2:508-50 510, 2007.
Dimitris Bertsimas, Colin Pawlowski, and Ving Daisy Zhuo From predictive methods missing data imputation: An optimization approach. Journal of Machine Learning Research, 18(196); 1 39, 2018a. URL http://jmlr.orge http://aah..orr//pppppra////1/10
Dimitris Bertsimas, Colin Pawlowski, and Ying Daisy Zhuo From predictive methods to missing data imputation: an optimization approach. Journal of Machine Learning Research, 18(196):1-33 2018b.
Dimitris Bertsimas, Arthur Delarue, and Jean Pauphilet. Simple imputation rules for prediction with missing data: Theoretical guarantees vs. empirical performance. Transactions on Machine Learning Research, 2024.
Felix Biessmann, Tammo Rukat, Phillipp Schmidt, Prathik Naidu, Sebastian Schelter, Andrey Taptunov, Dustin Lange, and David Salinas. DataWig Missing value imputation for tables. Journal of Machine Learning Research, 20(175):1-6, 2019.
Partha Chatterjee and Aakash Dev. Labour market dynamics and worker flows in India: Impact of COVID-19. The Indian Journal of Labour Economics, 66(1):299 327,, 2023.
Jiahua Chen and Jun Shao. Nearest neighbor imputation for survey data. Journal of official statistics, 16(2):113, 2000.
Jacob Cohen. Statistical power analysis for the behavioral sciences. L. Erlbaum Associates, Hillsdale, N.J, 2nd ed. edition, 1988. ISBN 9781134742776.
Koen De Maere, Steven De Haes, Michael von Kutzchenbach, and Tim Huygh. Tdentifying the enablers and inhibitors of organizational learning in the context of IT governance: an exploratory Delphi study. Information Systems Management, 39(3):241-268, 2022.
Nour El Houda Dehimi and Zakaria Tolba. Attention mechanisms in deep learning: Towards explainable Artificial Intelligence. In 2024 6th International Conference Pattern Analysis and Intelligent Systems (PATS), pp. 1-7. IEFE, 2024.
Tianyu Du, Luca Melis, and Ting Wang. Remasker: Imputing tabular data with masked autoencoding.
In The Twelfth International Conference on Learning Representations, 2024. URL https:// openreview
Mark Elliot. Final report on the disclosure risk associated with the synthetic data produced by the SYLLS team. Report 2015, 2, 2015.
20

---

Published in Transactions Machine Learning Research (02/2026)
Dominik Maria Endres and Johannes E Schindelin. A new metric for probability distributions. IEEE Transactions Information theory, 49(7):1888-1860, 2003.
Jean Feydy, Thibault Séjourné, François-Xavier Vialard, Shun-ichi Amari, Alain Trouvé, and Gabriel Peyré. Interpolating between optimal transport and mmd using sinkhorn divergences. In The 22nd international conference on ortificial intelligence and statisties, pp. 2681-2690. PMLR, 2019.
Trevor Hastic, Rahul Mazumder, Jason D Lee, and Reza Zadeh. Matrix completion and low-rank svd via fast alternating least squares. The Journal of Machine Learning Research, 16(1):3367--3)) 2015.
Nandini Jagannarayan and Asha Prasuna. An empirical analyses of determinants of health expendi- ture in rural amravati in march 2019, 2020 and 2021. Available at SSRN 4975312, 2024.
Sebastian Jäger, Arndt Allhorn, and Felix Bießmann. benchmark for data imputation methods. Frontiers in big Data, 4:693674, 2021.
Daniel Jarrett, Bogdan C Cebere, Tennison Liu, Alicia Curth, and Mihaela van der Schaar. Hy- perimpute: Generalized iterative imputation with automatic model selection. In International Conference Machine Learning, pp. 9916-9937. PMLR, 2022.
Alexia Jolicoeur-Martincau Kilian Fatras, and Tal Kachman. Generating and imputing tabular data via diffusson and flow-based pradient-bossted trees. In International Conference on Arlificial Intelligence and Statistics, pp. 1188-1296. PMLR, 2024.
Graham Kalton and Daniel Kasprzyk. Imputing for missing survey responses. Proceedings of the section on survey research methods, American Statistical Association, volume 22, pp. 31. American Statistical Association Cincinnati, 1982.
Rajat Kathuria and Aakash Dev. Technological advancement and employment changes: Recent trends in the Indian economy. The Indian Journal of Labour Economics, 67(3):637-660, 2024.
Markelle Kelly, Rachel Longjohn, and Kolby Nottingham. The UCI machine learning repository. https://archive.. ics uci. edul 2025. Accessed 2025-10-22.
Trent. Kyono, Yao Zhang, Alexis Bellot, and Mihaela van der Schaar. Miracle: Causally-aware imputation via learning missing data mechanisms. Advances in Neural Information Processing Systems, 11223866238811 2021.
Florian Lalande and Kenji Doya. Numerical data imputation: Choose knn over deep learning. In International Conference on Similarily Search and Applications, pp. 3 10. Springer, 2022.
Do-Hoon Lee and Han-joon Kim. selfatteention-based imputation technique for enhancing tabular data quality. Data, 8(6):102, 2023.
JiaHang Li, ShuXia Guo, RuLin Ma, Jia He, XiangHui Zhang, DongSheng Rui, YuSong Ding, Yu Li, LeYao Jian, Jing Cheng, et al. Comparison of the effects of imputation methods for missing data in predictive modelling of cohort study datasets. BMC Medical Research Methodology, 24(1):41, 2024.
Jianhua Lin. Divergence measures based on the shannon entropy. JEEE Transactions on Information theory, 3711)1145-15,, 2002.
21

---

Published in Transactions Machine Learning Research (02/2026)
Roderick JA Little and Donald Rubin. Statistical onalysis with missing data, volume 793. John Wiley & Sons, 2019.
Frank Massey Jr. The Soannogoro--mmirnon test for goodness of fit. Journal of the American statistical Association, 46(253):68-78, 1951.
Picrro-Alexandre Mattei and Jes Frellsen. Miwac: Deep generative modelling and imputation of incomplete data sels. International conference on machine learning, pp. 4413 4423. PMLR, 2019.
Imke Mayer, Aude Sportisse, Julie Josse, Nicholas Tierney, and Nathalie Vialaneix. Remiss-lastic: a unified platform for missing values methods and workflows. arXiv preprint arXiv:1908. 04822, 2019.
Bruce D Meyer, Wallace KC Mok, and James X Sullivan. Household surveys in crisis. Journal of Economic Perspectives, (144):199226,, 2015.
Xiaoye Miao, Yangvang Wu, Lu Chen, Yunjun Gao, and Jianwei Yin. An experimental survey of missing data imputation algorithms. IEEE Transactions on Knowledge and Data Engineering, 35 (7):6630-665) 2023. doi:
Boris Muzellec, Julie Josse, Claire Boyer, and Marco Cuturi. Missing data imputation using optimal transport. In International Conference on Machine Learning, pp. 7130-7140. PMLR, 2020.
Beata Nowok, Gillian M Raab, and Chris Dibben. synthpop: Bespoke creation of synthetic data in R. Journal of statisticol software, 74:1-26 261, 2016.
Beata Nowok, Gillian M Raab, and Chris Dibben. Providing bespoke synthetic data for the UK longitudinal studies and other sensitive data with the synthpop package for R. Statistical Journal of the IAOS, 33(3):785-796, 2017.
Jesim Pais and Vikas Rawal. Cmic's consumer pyramids houschold surveys: An assessment. In The Indian Forum, pp. 16, 2021.
Paszke, Pytorch: An imperative style, high-performance deep learning library. arXie preprint urXiv:1912.01 703, 2019.
George Paterakis, Stefanos Fafalios, Paulos Charonyktakis, Vassilis Christophides, and loannis Tsamardinos. Do we really need imputation in AutoML predictive modeling? ACM Transactions on Knowledge Discovery from Data, 18(6):1-66 2024.
Jason Poulos and Rafael Valle. Missing data imputation for supervised learning. Applied Artificial Intelligence, 186 March 2018 ISSN 1087-6545. doi: URL h/dp://dx.... doi org.10......... 1tt.//00.1000008835144444 2018. 1448143
Regina T Riphahn and Oliver Serfling. Item non-response on income and wealth questions. Empirical
economics, 30(2):521 538, 2005.
Donald B Rubin. Inference and missing data. Biometrika, 63(3):581 592, 1976.
Donald B Rubin. Multiple imputation for nonresponse in surveys, volume 81. John Wiley & Sons, 2004.
22

---

Published in Transactions Machine Learning Research (02/2026)
Esther-Lydia Silva-Ramirez, Rafael Pino- Mejias, and Manuel López- Coello. Single imputation with multilayer perceptron and multiple imputation combining multilayer perceptron and k-ncarest neighbours for monotone patterns. Applied Soft Computing, 29:65-74, 2015.
Aivin V. Solatorio and Olivier Dupriez. Realtabformer: Generating realistic relational and tabular data using transformers, 2023. URL ptttp://aardv..or//aaa/222222
Anmol Somanchi. Missing the Poor, Big Time: A critical assessment of the consumer pyramids household survey. Retrieved December, 12:2021, 2021.
Daniel J Stekhoven and Peter Bühlmann. Miss Forest non-parametric missing value imputation for mixed-type data. Bioinformatics, 28(1):112- 1111118,11111., 2012.
Heajung Suh and Jongwoo Song. A comparison of imputation methods using machine learning models. CSAM (Communications for Statistical Applications and Methods), 30(3):331 341, 2023.
Yige Sun, Jing Li, Yifan Xu, Tingting Zhang, and Xiaofeng Wang. Deep learning versus conventional methods for missing data imputation: A review and comparative study. Expert Systems with Applications, 227:120201, 2023.
Stef Van Buuren and Karin Groothuis-Oudshoone mice: Multivariate imputation by chained equations in R. Journal of statistical software, 45:1 67, 2011.
Zhenhua Wang, Olanrewaju Akande, Jason Poulos, and Fan Li. Are deep learning models superior for missing data imputation in large surveys? evidence from empirical comparison. arXiv preprint Stt:2103.09316 2021.
World Bank. Synthetic data for an imaginary country, full population, 2023, 2023. URL https:
/ wwwrroddta.wwrldaann... org/index.. wttpiinwww.ppppppppeeeeeeeee whtppppppp//ppp. phtp/catallog/study//.... 200___223____000000_ SYNTH-CEN-EN v01
Xinyu Yang, Yu Sun, Xinyang Chen, et al. Frequency-aware generative models for multivariate time series imputation. Advances Neurul Information Processing Systems, 37:52595 52623, 2024.
Jinsung Yoon, James Jordon, and Mihaela Schaar. Gain: Missing data imputation using generative ive adversarial nets. In International conference on machine learning, pp. 5689-5698. PMLR, 2018.
Hengrui Zhang, Liancheng Fang, Qitian Wu, and Philip S Yu. Diffputer: Empowering diffusson models for missing data imputation. In The Thirteenth International Conference on Learning Representations, 2025.
He Zhao, Ke Sun, Amir Dexfouli, and Edwin V Bonilla. Transformed distribution matching for missing value imputation. In International Conference on Machine Learning, pp. 42159-4444 42186. PMLR, 2023.
Shuhan Zheng and Nontawat Charoenphaklee. Diffusson models for missing value imputation in tabular data. Tn NewrIPS 2022 First Table Representation Workshop, 2022. URI. https: //openreview aet/forrmm/1=444@FFFXXCe
23

---

Published in Transactions Machine Learning Research (02/2026)
A
Appendix
Appendix Contents
A.1
Ethical Considerations and Limitations
24
2
Code
25
A.3
Configurations
25
Datasets
25
A.5
SubSDIC: Feature Descriptions and Construction Details
................................
25
A.6
Analysis of Missing Value Extremeness and RMSE Trends
26
A.6.1
Methodology
26
A.6.2 Results and Interpretation
27
................................
A.7
Temporal Dependency Analysis
28
A.7.1
Methodology: Pseudo-Pant Analysis
28
A.7.2
Baseline Temporal Dependency
29
A.7.3
Impact of Missingness and Imputation
29
A.8
Implementations and Hyperparameters
29
A.8.1
Implementation of Models
29
A.8.2
Hyperparameter Settings of Models
30
A.8.3
Hyperparameter Sensitivity and Convergence Analysis
30
A.9
Experimental Results
31
1.9.1
Statistical Significance Analysis
32
A.9.2
Downstream Task Performance: Light
32
A.1 Ethical Considerations and Limitations
SynthCPHS was generated with the goal of maintaining data utility while ensuring strong privacy protection. We leveraged the synthpop framework for fully synthetic data generation, approach known for its statistical disclosure control properties (Nowok et al. 2017). By design, the synthesized records contain no actual individuals, and thus the risk of re-identification or sensitive attribute disclosure is extremely low. Prior evaluations support this: Nowok et al. (2017) note that fully synthetic data pose minimal disclosure risk, and an independent risk analysis by Elliot (2015) similarly found the disclosure risk in syuthpop-generate additional safeguards (such excluding any accidentally replicated unique cases) can be applied further reduce even the perceived risk of identification.
dataset to be "very small". practice,
24

---

Published in Transactions Machine Learning Research (02/2026)
Nevertheless, this synthetic dataset is not intended as a substitute for the original CMIE CPHS data. For real-world policy and sociocconomic rescarch, direct access to the authentic CPHS data remains indispensable, only the original data provide fully reliable and legally accountable insights for decision-making.
A.2 Code
Code is available here: Git.Hub
Reproducibility Guide: To facilitate reproduction, we provide the SubSDIC dataset in the supplementary material. Researchers can reproduce the full benchmark results on SubSDIC by following the README file included in the repository. The workflow requires only minimal configuration (updating file paths marked in the scripts) to exccute preprocessing, imputation, and evaluation.
Please note that reproducibility is restricted the SubSDIC dataset. The CPHS dataset is proprictary, and the SynthCPHS dataset, due to its high statistical fidelity to the original commercial data, is also subject to distribution restrictions. Consequently, while the codebase supports all datascts, external exccution is currently limited to SubSDIC.
Data Availability Note:
A.3 Configurations
We conduct all experiments with the following setup:
Operating System: Rocky Limux 8 (a rebuild of Red Hat Enterprise Linux 8)
CPU: 2x AMD EPYC 7763 64-Core Processor 1.8GHz (128 cores in total)
RAM: 1000 GiB
GPU: 4x NVIDIA A100-SXM-80GB GPUs (each with 6912 FP32 CUDA cores)
Interconnect: Dual-rail Mellanox HDR200 InfiniBand
Cluster: 90 Dell PowerEdge XE8545 servers
Software: CUDA 11.4, Python 3.9.20, PyTorch
Paszke (019 2.6.0
A.4 Datasets
We include only those datasets that have been used in at least three imputation studies. The full names and links for these datasets are provided in Table
A.5 SubSDIC: Feature Descriptions and Construction Details
Table lists the 19 variables, including the target variables, in the dataset along with the corre- sponding SDIC "fake" survey questions created collect the data. Features prefixed with 'cat indicate categorical variables, while those prefixed with 'con_ indicate continnous variables.
25

---

Published in Transactions Machine Learning Research (02/2026)
Table 3: Related imputation benchmark datasets.
Dataset (link)
Full name
Brief context
California Housing
California
Honsing
Regression of median house value for Califor- nia districts, using numeric socioeconomic and geographic covariates.
Prices [Zhang et al.
2025
Du et
al.
2024
Jarrett
et al.
2022
Letter Recognition
Letter Recognition
Zhang
Multiclass classification of handwritten letters based on extracted shape pixel features.
et al.
2025
Du ct a..
2024
Jarrett et al. 2022
Yoon
et al.
2018
Credit Card Clients
Detault of Credit Card
Binary classification of credit-card default us- ing demographic, repayment and stltemmttt variables.
Clients Zhang et al. 2025
Du ct al. 2024 Yoon ct. al.
2018
Online News
Online News
Popularity Yoon
Regression / classification of article popularity from content and metadata features (social media, keywords, etc.).
[Zhang al. 2025
et al.
2018
Concrete Strength
Compressive Strength (Du et al.
Concrete
Regression of concrete compressive strength based on mixture composition and curing age.
2024
Jarrett et al. 2022
Zheng
& Charoenphakdee 2022
Wine Quality
Wine Quality (Du et al.
Quality prediction of red/ red/white wines from physicochemical measurements (regression ordinal classification).
2024 Jarrett ct al. 2022
Zheng & Charoenplakdee
2022
Diabetes Health
Diabetes Health Indica- tors (Du 2024 Jar
Binary classification of diabetes status using health survey indicators (BMI, lifestyle, comor- biditics).
rett et al. 2022 Zheng &
Charoenphakdee 2022
SpamBase
SpamBase (Du
2024
Binary classification of email spam based on word and character frequency features.
Jarrett et al.
2022
Yoon
et al.
2018
A.6 Analysis of Missing Value Extremeness and RMSE Trends
we observed a counterintuitive trend for methods relying on mean-centering, where the RMSE decreases as the missingness ratio increases under MAR and MNAR mechanisms. To validate that this is structural property of the missingness generation rather than an experimental artifact, conducted quantitative analysis of the "extremeness" of the missing values.
In Section 6.2
A.6.1 Methodology
We define the extremeness of missing value its absolute standardized deviation from the global mean (Z-score). For a continuous fcature with mean μy and standard deviation øj, the
26

---

Published in Transactions Machine Learning Research (02/2026)
Table 4: Detailed Feature Description of SubSDIC
Feature Name
Question Construct (Hl=households, I-mdiiidua)
cat hid
Household identifier / Machine Generated (H)
cat geol
Geographic area Admin (H)
cat geo2
Geographic area Admin 2 (H)
cat urbrur
Urban or rural indicator of household location (H)
con hhsize
Household size, i.e., number of individuals in the household (H)
cat statocc con exp 09
Does the household own, rent, or occupies this dwelling for free? (H)
How much does the household spend per year on? (H)
con_exp__1_ 10
How much does the household spend per year on? (H)
con_tot_at cat relation sex_ con_age cat marstat
Total monthly household expenditure across all categories (H)
What the relationship of [name] to the head of household? (1)
Is [name] male or female? (I)
How old is [name]] (I)
What [name's] ss] marital status? (I)
cat relgion cat set_scohoo__teeeddd_ee__e
What the religion of [name]] (I)
[name] attending school or preschool? (I)
con con_yrs_ yrs_school cat ______sstauus cat _cc_opcuationn
How many years has [name] attended school? (I)
What [name's] status of activity? (I)
What is/ was [uame's] main occupation? (I)
cat _____dd___tteee attain
What the highest level of school that [name] has completed? (I)
extremeness E. for the set of missing entries Mj at specific missingness ratio is calculated as:
ij
(5)
M,
iEM,
A higher E indicates that the missing values are, average, further from the center of the distribution (i.e., outliers). We computed this metric across all continuous features in the SubSDIC dataset under MCAR, MAR, and MNAR mechanisms across five missingness ratios.
A.6.2 Results and Interpretation
Figure illustrates the relationship between the missingness ratio and the extremeness of the masked values.
The results reveal distinct behaviors:
The curve is flat, indicating that the distribution of missing values is consistent with the global distribution (E x 0.75) regardless of the ratio. This explains why Mean Imputation performance is relatively stable under MCAR.
MCAR (Blue Line):
MAR and MNAR (Orange and Green Lines): There is a significant downward trend. At low missingnoss ratio (10%), the logistic generation mechanism is highly
27

---

Published in Transactions Machine Learning Research (02/2026)
Extremeness of Missing Values vs. Missing Ratio (SubSDIC)
Average Absolute Deviation (z: (z-Score) (Higher Moore Extreme Vallues Missing)
Mechanism
MCAR
MNAR
Missingness Ratio
Figure Average absolute deviation (Z-score) of the ground truth values of missing entries across different missingness ratios. Under MAR and MNAR, the "extremeness" of missing data decreases the ratio increases.
selective, masking primarily observations with extreme values (outliers) that yield the highest probabilities. Consequently, is high (> 0.85). As the ratio increases to 50%, the threshold for missingness lowers, and the mechanism includes a larger proportion of data points lying closer to the global mean, thus lowering the average extremeness (E' drops to Ri 0.80).
A.7 Temporal Dependency Analysis
To evaluate whether our benchmark datasets and missingness mechanisms account for the longitu- dinal characteristics socioeconomic surveys, we performed a quantitative analysis of temporal dependencics using the Autocorrelation Function (ACF).
A.7.1 Methodology: Pseudo-Panee Analysis
Since individual household identifiers were removed from the CPHS and SynthCPHS datasets to ensure they are not usod as semantic predictive fcatures, we constructed pseudo-panel analyze temporal signals. Households were aggregated by their Homogeneous Region (HR) (cat_HR), geographical unit designed to cluster households with similar socio-economic characteristics, and by survey round (con_WAVE_NO),
To quantify the global temporal dependency, we adopted two-step approach:
1. HR-Level Autocorrclation: First, we computed the Lag-1 autocorrclation (px) for cach specific Homogencous Region individually. Let øk,e represent the mean value of a feature within region k at wave t. The autocorrelation for region h is calculated
Ehe - (A - Ø4)
(6)
EE (xk. 3² 2 - 32
where Fy is the temporal mean of the feature for region k.
28

---

Published in Transactions Machine Learning Research (02/2026)
2. Global Aggregation: We then reported the average autocorrelation across all valid regions to represent the datasct-level temporal dependeney:
Pk
(7)
K
k=1
1
where is the total number of HRs and K 102.
A.7.2 Baseline Temporal Dependency
We utilized the effect size benchmarks established by Cohen 1988)
(Chapter 3), where correlation
of |r| 0.30 is categorized as medium effect and |r] 0.50 as large effect.
As detailed in Table 5 we select the top 10 features ranked by Porig. The original CPHS data exhibit significant temporal dependencies: income-related features typically show autocorrelation coefficients above 0.30, with some member-level income variables reaching as high as 0.476. These results confirm that the datasets contain moderate-to-strong longitudinal signals, distinguishing them from traditional independent and identically distributed (i.id.) tabular benchmarks.
A.7.3 Impact of Missingness and Imputation
We further analyzod the autocorrelation of the masked (incomplete) data, presenting results a 30% missingness ratio assess structural preservation. We define the Relative Drop (6) using the following equation:
8 Porig
Pmask
(8)
Porig
where Porig and Pmask denote the average cohort-level Lag-1 autocorrelation in the original and masked datasets, respectively.
Our analysis revealed that under MAR and MNAR, temporal dependencies are highly preserved, with relative drops often smaller than 5%. In some MNAR scenarios, a slight increase in ACF (negative d) was observed, likely due to 'survivor bias,' where the missingness mechanism selectively censors highly volatile observations, artificially smoothing the observed cohort means.
Furthermore, we evaluated the ACF of the datasets with imputation. We observed that most imputation methods yielded ACF values higher than those in the masked data, demonstrating their capability to restore temporal dependencies. Comprehensive results including ACF values for every feature across the complete dataset, the masked datasets, and the datasets imputed by all 14 methods provided in the Supplementary MMterial.
A.8
Implementations and Hyperparameters
A.8.1
Implementation of Models
We implemented all 14 imputation methods based on open access GitHub repository following:
Mcan/Mode Implemented using the NumPy package.
MOT
MMuzelle al. 2020 http:///ghhaaaccoooorrraaalllllt
29

---

Published in Transactions Machine Learning Research (02/2026)
Table 5: Comparison of HR-Level Autocorrelation (Lag-1) Before and After Missingness Generation (Missingness Ratio = 30%)- The 5 column indicates the percentage decrease in autocorrelation relative to the original data.
Original
MCAR (30%)
MAR (30%)
MNAR (30%)
Feature Name
ACP (Aowig)
Prosk
6 %
Preserved?
Prosk
8 %
Preserved?
Pronk
5 %
Preserved?
OF
ALL, S3CS
0.452
5.0%
Yes
0.476
0.0%
Yes
0.483
-1.5%
_N MEM_FRN WWW__FMA_ALLLLA ALL__A8__S8S
0.406
0.430
7.7%
Yes
0.461
3.2%
Yoe
0.460
1.3%
Yes
con INC, MEM_FRN_ALA ALL, SRCS
0.100
0.376
6.0%
Yes
0.378
5.5%
Yee
0.378
3.5%5
Yes
_on_TOT_TRCC _No_2
0.386
0.355
8.0%
Yes
0.361
6.5%
Yos
0.372
3.6%
Yes
con con_INC__ OF AAA___N__ALLLSSMME
0.361
0. 0.329
8.9%
Yes
0.317
3.9%
Yee
0.316
4.255
Yes
con _on_TOT_TKC_ TXC_3
0.355
0:335
5.6%
0.331
6.8%
0.335
5.6%%
con_INC_OF SALFRYLALL_SRCS
0.335
0.298
11 .00
Partial
0.312
6.9%
Yee 0:268
14.0%
Pactial
con TNC, OF HH [_____RMA,,,... _tC.,
0.323
0. 0,282
12.7%
Partial
0.283
12.4%
Partial
0.278
13.9%
Partial
con TOT [KC__KK_1
5.1%
Yes
0.268
13.1%
Partial
0.264
Partial
con_TT, T3_ 18_ TRAVEL. T8AVEL
0.250
0.281
3.1%
Yes
0.282
2.8%
Yes 0.274
5.5%
Yes
. MissForest Stekhooen & Bühlmann 2012
Implemented using the missforest package.
DSAN
Tee &
Kim
2023
http:///gttuub.commooos---tee
Structued -aat--aat-allla Ass tree//eeeeeeeeeeeee master
DSN
Lee & Kim 2023 Developed from DSAN by removing the attention layer.
TabCSDI (Zeng & Charoenphakdee
2022
http:://giaalo.co//pttttreeerrrr
TabCSDT
Remasker (Du al.
2024::
DiffPuter (hhng et al.
2025
For Hyper Impperr (Jarrett et al. 2022
MICE Van Buuren & Groothuis Oudshoorn
2011), MIRACLE Kyono et al. 2021). Soft (Hastic et al. 2015), MIWAE (Mattei
& Frellsen 2019), and GAIN Yoon et al. 2018), we use implementations at: https:
The codes for all methods are available in the anonymous GitHub repository.
A.8.2 Hyperparameter Settings of Models
Most of the methods included in our benchmark recommend using single set of hyperparameters across different datasets. For such methods, we adopt the default hyperparameters provided in their official GitHub repositories and ensure sufficient training epochs or steps to achieve convergence of the training loss. The anonymous GitHub repository provides the implementation with the default hyperparameters applied across all methods.
A.8.3 Hyperparameter Sensitivity and Convergence Analysis
To assess the impact dofault hyperparameters on our benchmark rankings, we conducted a focused analysis on GAIN, a generative model known for its sensitivity to hyperparameler settings. We selected the SubSDIC dataset under MNAR (30% missingness) as the testbed, as GAIN exhibited a notable performance drop in this specific sctting (RMSE 1.50) as shown in Figure
30

---

Published in Transactions Machine Learning Research (02/2026)
Convergence Verification. First, we verified that the default training duration was sufficient. We extended the training from the default 1,000 epochs to 10,000 epochs and monitored the total training loss. As shown in Figure 9 the loss curves for multiple random seeds stabilize well before 1, 1,000 epochs, confirming that the default settings do not suffer from under-fitting.
GAIN Train Loss Convergence
2.50
Sample 0
Sample 1
2.25
Sample 2
2.00
Sample 3
Total Train Loss
Sample 4
1.75
Mean Loss
1.25
1.00
0.75
2000
4000
6000
8000
10000
Epochs
Figure 9: Total training loss convergence of GAIN on SubSDIC (MNAR 30%) over 10,000 epochs. The model reaches convergence well within the default 1,000 epochs.
Grid Search and Ranking Stability. rameters for GAIN:
We performed grid search over two critical hyperpa-
nater {0.1, 0.3, 0.5, 0.7, {20, 40, 60, 80, 100}
The results indicated that the optimal configuration (with Siin_rate-02. and loss_ai ___ppappaa {60, 80, 100)) achieved an RMSE of 1.20, significant improvement over the default RMSE of 1.50.
Impact on Benchmarking Conclusions: While tuning improved GAIN's absolute performance and raised its rank by 3 positions, remained siguificantly outperformed by the top-tier methods identified in our benchmark. Specifically, models such MissForest, DSAN, DSN, ReMasker, and DiffPuter all achieved RMSE values below 1.0 the same task. This suggests that while hyperparameter tuming is beneficial, the relative performance gap between the most effective tabular imputation mothods and others is robust and primarily driven by model architecture capabilities rather than hyperparameter configurations. The experiments results can be found in the folder of the supplementary material named "Experiment_reest SubSDDCCIIputattee iimputation_perormancce performance/ OOt-
sample".
A.9 Experimental Results
All the experimental results, including KS lest, JS divergence, and Kendall's W, can be found in the folder of the supplementary material named "Experiment result". The following sections present only selection of represcntative results which are not listed in the main text.
31

---

Published in Transactions Machine Learning Research (02/2026)
A.9.1 Statistical Significance Analysis
To ensure the statistical rigor of our benchmarking results and recommendations, we conducted pairwise t-tests (based on random seeds) for all 14 methods on the SubSDIC datasct. This analysis generated 91 pairwise combinalious across experimental settings (3 mechanisms x ratios) and task types (continuous and categorical imputation), yielding total of 2,730 p-values. We found that 84% of these comparisons showed statistically significant differences (p < 0.05). The full set of p-values is available in the Supplementary_ Matteeerttttry_MMMaallll..
Based on the practical recommendations outlined in Section 6.7, we highlight the statistical ver- ification of the top-performing methods (MissForest, ReMasker, DiffPuter, DSAN, DSN, HyperImpute) in four critical scenarios.
Continuous Variable Imputation (RMSE) We validated the imputation performance for continuous variables, with detailed p-values provided in Table 0
1. Low Missingness (10%, MAR/MNAR), MissForest is significantly superior to other top-tier models (including ReMasker, DSAN, and HyperImpute) with 0.05. This statistically confirms its dominance and suitability as default choice in low-missingness regimes.
2. High Missingness (50%, MAR/MAR /MNAR). DSN significantly outperforms traditional basclines and other deep models (p < 0.05). However, the dilference between DSN and DSAN is not statistically significant (p > 0.05). This aligns with our observation in Figure 3 that their performance is comparable, suggesting that the attention layer in DSAN does not yield a significant advantage over DSN in this specific context.
Categorical Variable Imputation (F1 Score) We conducted similar tests for categorical variables (detailed tables omitted for brevity, full results in Supplementary Material):
1. Low-to-Medium Missingness (S 20%, All Mechanisms). MissForest significantly outper- forms all 13 other methods (p < 0.05), demonstrating exceptional robustness for categorical data in this regime.
2. High Missingness (50%). ReMasker significantly outperforms other methods (p < 0.05), with the exception of DiffPuter. The difference between ReMasker and DiffPuter is not statistically significant (p > 0.05), which is consistent with Figure 4, where both generative models demonstrate similarly high performance at extreme missingness ratios.
A.9.2 Downstream Task Performance: LightGBM
To further examine the sensitivity of downstream evaluation ion to model choice, we additionally evaluate LightGBM on the SubSDIC dataset, the only dataset in our benchmark that naturally supports both a classification and a regression task. LightGBM is assessed two modes:
1. using imputed data produced by each of the 14 imputation methods;
2. using LightGBM's native handling of missing values naïve "no-imputation" bascline.
32

---

Published in Transactions Machine Learning Research (02/2026)
Table 6: P-values for pairwise comparisons of top-tier methods on SubSDIC (Continuous Variables). Bold scenarios highlight the superior performance of Method A. (p < 0.05 indicates statistical significance).
Task Type
Scenario
Method A (Hypothesis: Better) Method
p-value
Significant.
Continuous Veriable Imputation (Metric: R.MSE)
MAR 10%
MissForest
ReMasker
0.0418
Yes
MAR 10%
MissForest
DSAN
0.0346
Yes
MAR 10%
MissForest
DSN
0.0044
Yes
MAR 10%
MissForest
DiffPuter
0.0000
Yes
MAR 10%
MissForest
Hyperlmpute
0.0001
Yes
Low Missinguess
MNAR 10%
MissForest
ReMasker
0.0061
Yes
MNAR 10%
MissForest
DSAN
0.0222
Yes
MNAR 10%
FissForest
DSN
0.0338
Yes
MNAR 10%
MissForest
DiffPuter
0.0089
Yes
MNAR 10%
MissForest
HyperImpute
0.0015
Yes
MAR 50%
DSN
ReMasker
0.0025
Yes
MAR 50%
DSN
MissForest
0.0005
Yes
MAR 50%
DSN
DifPuter
0.0078
Yes
MAR 50%
DSN
Hyperlmpute
0.0022
Yes
MNAR 50%
DSN
ReMasker
0.0021
Yes
High Missinguess
MNAR 50%
DSN
MissForest
0.0086
Yes
MNAR 50%
DSN
DiffPuter
0.0000
Yes
MNAR 50%
DSN
Hyperlmpute
0.0069
Yes
As shown in Figure 10 the "no-imputation" (marked as "w/o imputation") baseline yields the
worst performance across all missingness mechanisms and across most missingness ratios, for both classification and regression tasks. In classification, the ROC-AUC degradation under native value handling is substantially larger than that of any imputed alternative. Likewise, in regression, the RMSE increase under the "w/o imputation" baseline dominates that of all imputed variants.
Notably, even the weakest imputer in our pool outperforms LightGBM's native missing-value strategy under most experimental configuration. This finding provides a clear practical implication: in the context of sociocconomic survey data, explicit imputation consistently beneficial and should not be replaced by relying solely on GBM's built-in handling of missing values.
Together with the sensitivity analyses under /logit/l regression, Random Forest, and XGBoost in Section 0.3 these results further demonstrate that our conclusions regarding the relative ordering of imputation methods are robust to the choice of downstream predictive model.
33

---

Published in Transactions Machine Learning Research (02/2026)
MCAR
MAR
MNAR
Average
Classificatioo cation ROC-AUC Degradation (%)
-5-
-5
-10
-10
-10
-15
-15
20
-20
-25
-25
-25
RMSE Increase (%)
125
125
150
125
Regression
100
100
75
75
75
50
50
50
25
0.1
0.2
0.3
0.4
0.1
0.2
0.3
0.4
0.5
0.1
0.2
0.3
0.4
0.5
0.1
0.2
0.3
0.4
0.5
Missing ratios
Missing ratios
Missing ratios
Missing ratios
MissForestB/1
DSANG/3 DSN@/6
Hypertmpute@ly@ MIRACLE
MOT
DiffPuteri//2 TabCSDI2j-
MIWAE-19
MICE8/O
GAIN
Softimpute
Remasker2/0
w/o imputation
Figure 10: Top: ROC-AUC degradation for downstream classification using LightGBM. Bottom: RMSE increase downstream regression using LightGBM. Lower ROC-AUC degradation and smaller RMSE increase indicate better preservation of predictive performance. Legend entries show classification performance/regression performance ranks (firs//second number).
34


Reviews:
[{"review_id": "Li2Bc9egNQ", "reviewer": "Reviewer_bSNe", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This submission addresses the critical gap in benchmarking missing data imputation methods for socioeconomic surveys, a domain characterized by longitudinal, hierarchical, high-dimensional, and non-i.i.d. data with complex missing mechanisms. The key contributions include: (1) a comprehensive benchmark integrating three datasets (real CPHS, synthetic SynthCPHS, and public SubSDIC) that reflect real-world socioeconomic survey characteristics; (2) systematic evaluation of 14 imputation methods across three missingness mechanisms (MCAR, MAR, MNAR) and five missingness ratios (10%-50%), covering both continuous and categorical variables; (3) multi-metric assessment encompassing imputation accuracy, downstream task performance (classification and regression), and computational efficiency; and (4) provision of an open-source framework and public dataset (SubSDIC) to support reproducible research.\n\n\n### Key Strengths\n1. Fills a critical niche by focusing on socioeconomic surveys.\n2. Uses both real and high-fidelity synthetic datasets, addressing privacy constraints of proprietary survey data.\n3. Adopts a holistic evaluation approach, moving beyond imputation accuracy to include downstream task impact and computational efficiency.\n4. Comprehensive coverage of missingness scenarios (including the understudied MNAR mechanism) and diverse imputation methods (statistical, iterative ML, deep learning, hybrid) ensures generalizability of findings.\n\n### Weaknesses\n1. Graph-based imputation methods are excluded without sufficient justification, limiting the scope of comparative analysis.\n2. No statistical significance tests are conducted when comparing imputation methods, making it unclear whether performance differences are statistically meaningful or due to random variation.\n3. Figure 3 shows a counterintuitive trend: the 'Mean' curve’s RMSE decreases as missingness ratios increase, which contradicts typical expectations and lacks sufficient explanation to support its credibility.\n4. Table 1 lacks detailed descriptive text, including clear definitions of column headers and brief context for each dataset, hindering reader's understanding of the benchmark dataset comparisons.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Yes. The authors provide rigorous empirical evidence to support their claims: \n1. Distribution similarity between CPHS and SynthCPHS is validated via KS test (for continuous variables) and JS divergence (for categorical variables), with negligible differences and high p-values.\n2. Downstream task impact is assessed using ROC-AUC degradation and RMSE increase, with results replicated. \n3. Computational efficiency is measured via wall-clock time, with clear distinctions between fast traditional methods and slower deep learning approaches. \n4. Ranking consistency is verified using Kendall’s coefficient of concordance, demonstrating strong agreement across datasets for key metrics.\n\nHowever, the lack of significance tests for method comparisons and unexplained trends in Figure 3 slightly weaken the robustness of the evidence.", "audience_yn": "Yes", "audience_explanation": "Yes. TMLR’s audience includes researchers and practitioners in machine learning, statistics, and applied data science, all of whom stand to benefit from this work. \n1. For ML researchers, the paper provides a rigorous benchmark for evaluating imputation methods on structured, real-world data.\n2. For applied researchers and policymakers working with socioeconomic surveys, the findings offer actionable guidance on selecting imputation methods that balance accuracy, downstream utility, and efficiency. \n3. The open-source framework and public SubSDIC dataset enable further research on missing data in high-stakes domains.", "requested_changes": "1. Provide additional justification for excluding graph-based imputation methods. If their output is incompatible with the evaluation framework, explain how this incompatibility arises and whether it could be resolved (e.g., post-processing to align with tabular format).\n2. Conduct statistical significance tests to compare performance differences between imputation methods. Report p-values and clarify which method comparisons are statistically significant, especially for top-performing methods.\n3. Provide a detailed analysis of the counterintuitive trend in Figure 3.\n4. Enhance Table 1 with comprehensive descriptive text.", "broader_impact_concerns": "The paper includes a brief ethical consideration section (Appendix A.1) addressing privacy protections for SynthCPHS.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Fa9vLNLaDt", "reviewer": "Reviewer_rUAa", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Disclaimers: I am not a specialist in missing data, but I am familiar with machine learning benchmarks and empirical methodology; my review focuses on the soundness and usefulness of the benchmark rather than fine-grained statistical theory.\n\nThis paper presents a comprehensive benchmark of missing-data imputation methods for large-scale socioeconomic survey data. The authors construct three datasets: (i) a proprietary real panel survey (CPHS) from India, (ii) a high-fidelity synthetic version of CPHS (SynthCPHS), and (iii) a publicly shareable subset of the World Bank’s SDIC synthetic dataset (SubSDIC). They simulate missingness under MCAR, MAR, and MNAR mechanisms at five missingness ratios (10–50%), and evaluate 14 classical and modern imputation methods. Evaluation covers imputation accuracy on continuous and categorical variables, impact on downstream classification and regression tasks (via AUC degradation and RMSE increase), and computational efficiency. The results show that classical methods such as MissForest perform very well at low missingness, while recent deep methods like ReMasker, HyperImpute, DSAN/DSN, and DiffPuter become competitive or superior under higher missingness or more complex settings. The authors also examine ranking consistency across datasets and release SubSDIC plus an evaluation framework to facilitate future research.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper is mainly about dataset construction and benchmarking, not about novel algorithms. I reviewed it in the following aspects, 1) whether the tasks are helpful 2) how the datasets are designed and constructed; 3) whether the benchmarking is thourough and reasonable; 4) whether the benchmarking reveals something helpful for future studies.\n\nFor these questions, I think \n1) Missing-data imputation in large socioeconomic surveys is an important applied problem;\n2) The combination of real CPHS, a synthetic CPHS-like dataset, and a publicly shareable SubSDIC dataset is well thought-out;\n3) The benchmark includes 14 methods spanning simple baselines, classical iterative algorithms, distribution-matching approaches, and several recent deep and diffusion-based methods;\n4) MissForest as a strong, efficient default at low missingness; deep models becoming preferable when missingness is severe; DSN outperforming DSAN in out-of-sample settings—that will be genuinely useful to applied researchers choosing an imputation method.", "audience_yn": "Yes", "audience_explanation": "For researchers who need large-scale socioeconomic survey data for their experiments, the paper is helpful on providing a benchmark. For researchers and practitioners who need to understand what are good algorithms and methods available for missing-data imputation scenarios, the benchmarks of 14 methods are helpful as evidences and advices.", "requested_changes": "1. Clarify how a third party can reproduce at least a substantial subset of your experiments using only publicly available components;\n2. Hyperparameter tuning protocol and fairness across methods. It seems that many hyperparameters are set as defaults.\n3. Sharpen the practical recommendations for practitioners. The empirical results contain many useful patterns (e.g., MissForest / HyperImpute remaining strong and efficient at low missingness, deep models outperforming at high missingness). To make the paper more actionable for applied users, I recommend adding a short “Practical Recommendations” subsection.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "aZi3OKjA3U", "reviewer": "Reviewer_Qn8w", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a large-scale, domain-specific benchmark for missing-data imputation in socioeconomic surveys. The authors construct and evaluate imputation methods on three structured datasets: the proprietary CPHS dataset, a synthetic replica, and a publicly available survey-like dataset. The benchmark systematically compares 14 classical, statistical, and deep-learning imputation algorithms across MCAR/MAR/MNAR mechanisms, five missingness levels, continuous and categorical variables, downstream predictive tasks, and runtime.\n\nStrengths:\n- Socioeconomic survey imputation is critical yet under-studied relative to healthcare or UCI-style tabular data. A reproducible benchmark addressing longitudinal, hierarchical, non-IID structures fills a real gap.\n- The paper evaluates a large set of methods belonging to different families, across mechanisms, missingness ratios, variable types, and downstream tasks (continuous/categorical imputation + downstream classification/regression).\n- Findings reinforce the empirical value of classical methods and highlight when deep learning approaches excel or fail.\n- Providing the full evaluation pipeline is a valuable contribution to the community.\n\nWeaknesses:\n- The paper repeatedly highlights that socioeconomic surveys are longitudinal, and this is presented as one of the main motivations for creating a domain-specific benchmark. However, the longitudinal structure does not appear to be explicitly modeled, exploited, or stressed in the methodology. Missingness is generated feature-wise via logistic models that do not clearly incorporate time dependencies, nor is it explained whether missingness mechanisms preserve realistic longitudinal patterns. Additionally, none of the evaluated imputation methods are time-aware or sequence-based, and the results section does not analyze whether the temporal dimension influences imputation difficulty. As a result, it is unclear whether the benchmark truly evaluates the challenges unique to longitudinal survey data, or whether the datasets are effectively treated as cross-sectional tables. Clarifying how longitudinal dependencies were handled, both in missingness generation and in model evaluation, would strengthen the contribution and align the experiments more closely with the paper’s stated motivation.\n- Some methodological details could improve in clarity, particularly regarding missingness generation and default hyperparameter usage.\n- Limited discussion of failure cases (e.g., why MIRACLE or MOT performed so poorly), which would help contextualize results.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The authors use rigorous empirical evaluation and provide extensive metrics. Results are consistent across datasets, and statistical comparisons (KS/JS divergence) support the validity of the synthetic benchmark.", "audience_yn": "Yes", "audience_explanation": "The work sits at the intersection of ML for tabular data, imputation, and synthetic data generation. It provides a valuable benchmark for researchers developing new imputation methods, especially those targeting realistic tabular settings.", "requested_changes": "- Clarify how longitudinal structure is handled. Since longitudinality is highlighted as a main motivation, the paper should explain whether missingness generation preserves time dependencies and whether the evaluation reflects temporal structure.\n- Clarify missingness generation mechanisms in the main text. Appendix A.6 provides the generation details, but the paper should explicitly summarize the logistic-model–based procedure for MAR/MNAR, how parameters β are learned, whether the same logistic models are used across datasets or re-estimated each time, how temporary imputations work under MNAR. This is central to the benchmark’s validity and should not rely solely on appendices.\n- The paper acknowledges that default hyperparameters were used for many baselines, but this may introduce bias. The authors should clarify which methods were most sensitive to hyperparameters, justify why deep learning models were not tuned, and discuss whether performance rankings might shift with tuning.\n- Discuss failure modes of specific models providing short diagnostic section to help contextualize the benchmark.\n- The authors use RF and XGBoost, which is reasonable, but ideally they should justify these choices over linear models or neural networks and clarify whether models were tuned or used with defaults. Additionally, comparing against a naïve “no-imputation” baseline or noting widespread gradient boosting methods like LightGBM that natively handle missingness could enrich the evaluation.\n- A slightly more detailed explanation of why models like GRAPE or IGRM are incompatible with the evaluation framework would improve clarity.", "broader_impact_concerns": "No concern.", "additional_comments": "- How sensitive are results to the choice of downstream models? RF/XGB are robust, but some imputation methods that leverage nonlinear interactions may perform differently under linear models.\n- Could the benchmark be extended to assess imputation uncertainty? Several deep generative models produce uncertainty estimates and incorporating proper scoring rules could open an interesting dimension.\n- Do the authors plan to provide a leaderboard extension of this benchmark?", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)