Rethinking the Diffusion Models for Missing Data Imputation: A Gradient Flow Perspective
Authors: Zhichao Chen
Haoxuan Li
Fangyikang Wang
Odin Zhang
Hu Xu
Xiaoyu Jiang
Zhihuan Song
Hao Wang
Venue: NeurIPS 2024 poster (2024)
Keywords: Missing Data Imputation
Gradient Flow
Reproducing Kernel Hilbert Space
Functional Optimization
Primary area: probabilistic_methods
TL;DR: We propose a novel, easy-to-implement, numerical tabular data imputation approach based on joint wasserstein gradient flow.

Abstract: Diffusion models have demonstrated competitive performance in missing data imputation (MDI) task. However, directly applying diffusion models to MDI produces suboptimal performance due to two primary defects. First, the sample diversity promoted by diffusion models hinders the accurate inference of missing values. Second, data masking reduces observable indices for model training, obstructing imputation performance. To address these challenges, we introduce $\underline{\text{N}}$egative $\underline{\text{E}}$ntropy-regularized $\underline{\text{W}}$asserstein gradient flow for $\underline{\text{Imp}}$utation (NewImp), enhancing diffusion models for MDI from a gradient flow perspective. To handle the first defect, we incorporate a negative entropy regularization term into the cost functional to suppress diversity and improve accuracy. To handle the second defect, we demonstrate that the imputation procedure of NewImp, induced by the conditional distribution-related cost functional, can equivalently be replaced by that induced by the joint distribution, thereby naturally eliminating the need for data masking. Extensive experiments validate the effectiveness of our method. Code is available at [https://github.com/JustusvLiebig/NewImp](https://github.com/JustusvLiebig/NewImp).

Rethinking the Diffusion Models for Missing Data Imputation: A Gradient Flow Perspective
Zhichao Chen'
Haoxuan Li
Fangyikang Wang'
Odin Zhang
Hu Xu'
Xiaoyu Jiang' Zhihuan Song Hao Wang"
iZhejiang University Pekiigg University University of Washington
"Guangdong University of Petrochemical Technology
j20320422zju. edu. en hxli@stu. pku. edu cn bangfangyikknggz)) edu. cn odinz@us edu seu_zju@zju.. edu. en jiangxiaoyudzji edu.uuuuuuuuuuuuuu.u..uu..u. edu. en songzhihuanēzju edu. hachaowEzju. edu.cn
Abstract
Diffusion models have demonstrated competitive performance in missing data imputation (MDI) task. However, directly applying diffusion models to MDI produces suboptimal performance due to two primary defects. First, the sample diversity promoted by diffusion models hinders the accurate inference of missing values. Second, data masking reduces observable indices for model training, obstructing imputation performance. To address these challenges, we introduce Negative centrooy-regulaate Wasserstein gradient flow for Imputation (NewImp), enhancing diffusion models for MDI from a gradient flow perspective. To handle the first defect, incorporate negative entropy regularization term into the cost functional to suppress diversity and improve accuracy. To handle the second defect, demonstrate that the imputation procedure of Newlmp, induced by the conditional distribuutio-rellteed cost functional, can equivalently be replaced by that induced by the joint distribution, thereby naturally eliminating the need for data masking. Extensive experiments validate the effectiveness our method. Code is available https///ggiubbccoo//m/uuwvvieee
Introduction
Missing data a pervasive problem for data analytics in diverse scenarios, including e-commerce [29, 30, 57], healthcare [51, 56], and process industry [33, 58]. For instance, in healthcare, patient monitoring devices may fail or lose connection, leading to missing vital signs data. Similarly, in industrial processes, sensor signals may be incomplete due to inevitable mechanical shock. These incompletenesses hamper data integrity and impede subsequent analysis. Therefore, accurate missing data imputation (MDI) is critical for enabling reliable analysis and decision in real-world applications.
Recently, diffusion models (DMs) have emerged as a powerful tool for MDI [66]. Specifically, these models first estimate the (Stein) score function of the missing data conditioned on the observed data, subsequently reformulating the imputation problem as a generative task grounded in the learned score function. These works are initiated from [51] and evolve to incorporate crafted model architecture [33] and learning objectives [38, 73] for enhancing the accuracy score estimation [41]. Celebrated for their advantageous capability to model data distributions and generate high-quality synthetic data [41, 50, 65), diffusion models have been a prevalent approach to MDI.
Despite the successes of diffusion models, we argue that directly applying diffusion models to MDI results in suboptimal performance due two primary limitations. First, diffusion models perform
"Corresooning author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

imputation by sampling from a learned score function, which inadvertently promotes diversity in the imputed values. This increased diversity contradicts the accuracy required for precise imputation of missing data [38]. Second, the training process involves masking a portion of the observed data as labels. The selection of masking strategy significantly impacts imputation accuracy and is inherently challenging to optimize [51]. Moreover, the masked data during training often differ in missing mechanisms from those encountered during testing, resulting in a discrepancy between training and inference phases that degrades performance. Consequently, difffusinn models introduce unintended diversity and impose data masking, both of which impede effective imputation.
To tackle these challenges, we introduce a novel DM-based MDI approach termed Negative Entropy- regularized Wasserstein Gradient Flow Imputation (NewImp). Specifically, handle the first issue, we revisit MaM-d MDI task within the Wasserstein Gradient Flow (WGF) framework, derive the associated cost functionals, and identify that they implicitly promote diversity the imputed values. Building on this insight, we incorporate a negative entropy-regularized (NER) cost functional to suppress imputation diversity and enhance accuracy. Furthermore, we derive closed-form imputation procedure based on the proposed cost functional within the reproducing kernel Hilbert space (RKHS), After that, we further prove that within the WGF framework, the imputation procedure of NewImp, induced from the cost functional associated with conditional distribution, can be induced from another cost functional associated with joint distribution equivalently, within which we merely need to estimate the joint distribution during the model training stage, thereby naturally eliminating the need for data masking.
Contributions. The main contributions of this paper are summarized as follows:
We demonstrate that directly applying diffusion models to MDI causes suboptimal performance, as they prompt unintended diversity and require data masking, both impeding accurate imputation.
We propose NewImp, novel DM- RM-based MDI approach under the WGF framework which intro- duces an NER cost functional to suppress unintended diversity. Based on this, we further prove that the imputation procedure of Newlmp can be induced from equivalent foint-distribution--rllae functional, and consequently introduce an imputation procedure that sidesteps the data masking.
We conduct various experiments over public numerical tabular datasets to demonstrate the superior- ity of the NewImp method over prevalent baseline models.
2 Preliminaries
2.1 Problem Formulation
Suppose (ddel)) NRD represents an ideal numerical tabular dataset without any missing entries, where N and D denote the number of samples and features, respectively. The observed dataset is expressed as: (obo) X (ideal) c M + NaN O (INDD- - -M,, where @ denotes the Hadamard product, 1NXD is a matrix of ones of size x D, and M E {0. NxD is a binary mask that indicates the presence (I) or absence (0) of data in each entry. The task of MDI involves imputing the missing entries in X (obs), This is achieved by constructing a matrix X X (obs) O X (imp) o (INxD-MM)
where X (imp) is the matrix containing the imputed values.
The missing mechanism can be classified into three categories [44]: Missing Completely at Random (MCAR), Missing at Random (MAR), and Missing Not at Random (MNAR) (Detailed information about missing mechanisms is given in Appendix E. Notably, in the MNAR setting, it is generally difficult identify the missing data distribution without additional assumptions and constraints [22].
Hence, our discussion primarily focuses on numerical tabular data with MAR and MCAR settings.
2.2 Diffusion Models and Its Application for MDI Task
Diffusion models function by gradually corrupting data towards tractable noise distribution, such as standard Gaussian, and subsequently reversing this corruption to generate samples [50]. Specifically, the forward corruption process is modeled as a discretization of a stochastic differential equation (SDE) over time T: dx,- = J(X,))7 gwd where f(X,) is drift term, Jr is volatility term, and dWr standard Wiener process. The solution to this SDE creates a continuous trajectory of random variables X, T The density function g. of X, adheres to the Fokker- Planck- Planck-Kolmooeeeeeee Kolmogorov (FPK)

---

equation: Bg: = - (gff(XT)) 2 Vg, (see Theorem 5.4 in reference [47]). The reverse ör process is governed by: dx, = [f(X,) of ((X-) + grdW, [3], where Vlog x)) represents the score function, which often parameterized by neural networks.
Diffusion models treat MDI as a conditional generation task. The score function, Vlogp(X), is ( ( X (cho) [51], and the MDI task is executed by generating samples based on this conditional score function. The key challenge is to obtain an estimation V Ximis log (
defined specifically for MDI as V Ximis: log
( ( X (dbs)) that approximates x log p( ( X (()b)). Given that the true X (miss) is unknown, existing DM-based approaches utilize mask matrix to drop some observable data as labels. However, the specification of the mask mechanism, determining the effectiveness of V log (((()))ss) X (obs)) is challenging since it should align with the data missing mechanism in the testing dataset [51]. which may be unknown in practice.
2.3 Wasserstein Gradient Flow
Wasserstein space (P is defined as the set of distributions with finite second-order moments. Consider cost functional Foust (P2(PDD - R; the celebrated Wasserstein gradient flow (WGF) is an absolutely continuous trajectory (g+)>>0 in this space, which evolves over time T to minimize Fous efficiently. This dynamic is governed by the continuity equation:
0gr - (u-g+), #t - 5.F
Ōr
where u, R. + Rp is time-dependent velocity field [2], whose input is sample X RD; dFum 8gr
denotes the first variation of Foust with respect to 4r- On this basis, the evolution of X over time 7 in (2(R can be modeled by the ordinary differential equation (ODE):
dX dr
(2)
However, simulating this ODE is challenging since u, involves the estimation of Zr, which involves solving the differential equation dgr Dr (r) that proves to be not analytically solvable [16].
3 Motivations
3.1 Diffusion Models Secretly Foster Diversity
Based on the notations defined in Section 2.1, we can first define the following cost functional for MDI task according to the maximum like- lihood estimation principle, where we want to find the value with the highest probability:
0.5
0.5
X (imp)
arg max log p( X (miss) X (dhs)), (3)
2
(min)
where p(x (miss)
is the estimation of X (obs) via neural network [52]. No-
X (obs))
01
pl X (miss)
(a) Optimal Value.
(b) Results by DM.
tably, can treat X (miss) samples from "proposal distribution' r(x (miss)), ), and formu- late the following optimization problem based on variational inference [27, 70]:
Comparison of the optimal point in green triangle and the results obtained by diffusion mod- els in white scatters. See details Appendix B.
Figure 1:
E ( (log Pi X (miss)
X ((ahs)) )}
arg max x ((iss)
where we aim to sample some X (miss) samples from proposal distribution r(X (miss)), realize the maximum log-likelihood estimation over the sampled results, and 'optimize" the proposal distribution r(x (miss) that is represented by samples X (miss) Notably, in Eq. (4), we use the spirit from previous references represented by [32], where optimizing the samples X (miss) is equivalent to optimizing the distribution r r(x (miss))
3

---

Referring to Eq. (4), is observed that the MDI task can be formulated as an optimization problem.
This prompts a pertinent question: If the conditional distribution log X {miss) |X (obb)) is estimated accurately, what would happen if we directly apply diffusion models to solve the optimization problem corresponding MDI task? To explore this, we consider hypothetical scenario: Suppose we are optimizing a cost functional related to a direesiimanional Dirichlet distribution on the simplex A
T( PK) 1=1 I ((PE)
3
arg max on EA?
Σ flog
Σ 1=1
- (Px-1) log a, H = 8, pl = [2.5, 5.0).
where I are variables, H is variable number, Pa p=lk== is concentration parameter, and (-) is gamma function. We compare the analytically derived optimal value with the results from diffusion models in Fig. 1. The diffusion model's results tend to surround but do not exactly reach the optimal value, suggesting that there might be implicit, diversity-nccouraging terms integrated into the diffusion models objectives that produce the observed inaccuracies. Identifying and modifying these regularization terms is crucial for enhancing the efficacy of diffusion models for MDI tasks.
3.2 Negative Entropy Regularization Term for Diversity Suppression
In this section, we identify and refine the terms in diffusion models' objectives that prompt unintended diversity and impede accurate imputation. We observe that the inference process in diffusion models adheres to the FPK equation, which is a specialized form of the continuity equation in WGF (see Sections 2.2 and 2.3). This alignment inspires us to reframe diffusion models within the WGF framework, enabling the derivation of their underlying cost functionals. By doing so, we can compare these functionals with the objective functional for MDI in Eq. (4)².
Proposition 3.1. Within WGF framework, DM-based MDI approaches can be viewed as finding the imputed values X (bmp) that maximize the following objective:
arg max
(x (mm)) [log p( X (miss)
(abs)
} + th( X (miss) + const,
(5)
r(X (mis))
where 'const' is the abbreviation of constant, and v6 (miis) is scalar function determined by the type of SDE underlying the diffusion models.
VP-SDE: v X (miss))
- THI r (miss)))
}
W F (mso)) ( X (mmissy) X (mmiss)) 2
VE-SDE: 1 X (miss)) = 1 X (miss)) 2
sub-VP-SDE: (XX (miss)) 1 [HI
X (mmss)))]
+ E ( (mis))
{
1 [
[ [
}} } 0.
47+
where H[(rXx(mizs) == =- frr("" ( (miss) log r( X (mizs))
dx (mizs) is the entropy term, "r is determined
by noise scale Br: "T: == (1- ex-e (2 2 3. dsss))
> 0,0
< 31 < ..... < Br < 1.
Proposition 3.1 reveals that diffusion models inherently optimize an objective functional that largely aligns with (4), but they secretly include an additional term ( (miss) > 0. This term makes Eq. (5) an upper bound on Eq. (4), i.e., maximizing the cost functional in Eq. (5) does not guarantee to
maximize the MDI objective in Eq. (4). Furthermore, the entropy term included in the models fosters
sample diversity, which may compromise the accuracy required in MDI tasks [53, 38]. To address this issue, we propose incorporating a negative entropy term as ()(xX (miss)). );
v (ximmss)) = -XHI[r(X
(6)
where À= > 0 is predefined regularization strength, and consequently we can define our NER cost functional for MDI task as follows:
FNER EB, (m))) [log P X [miss)
X (obs)) )] AHI(( (X (miss)))
The objective functional in Eq. (7) provides a lower bound of Eq. (4). Therefore, maximizing FNER
guarantees maximizing Eq. (4). Meanwhile, JNNE effectively reduces the unintended diversity term, contributing to an improvement in imputation accuracy.
"This paper primarily considers three types of stochastic differentiall equations (SDEs): variance preserving (VP-SDE), variance exploding (VE-SDE), and sub-VP-SDE, which cover the majority of diffusion models according to Song al. [50]

---

Implementation of the NewImp
4.1 Optimizing the FNER within WGF Framework
In this section, we aim to optimize FNE within the WGF framework [46, 69). To this end, we plug Eq. (7) into Eq. (1), producing the velocity field below that drives the ODE Eq. (2):
((-NNER
u x (miss)) = -V X 8r( ( (
X Ximia) log p X (miss)
X (abs) + AV
X Xini log r( X (miss))) }}
However, as stated in Section 2.3, implementing this ODE in computer code is intricate due to the intractability of the density function (X (miss).. Fortunately, by restricting the velocity field within the Reproducing Kernel Hilbert Space (RKHS) defined by the kernel function u( ( E
K ((X (miss))
X
(miss))
) an alternative ODE minimizing JNER can be implemented in Proposition 4.1 [35,
31] which sidesteps the intractable ((x (1
Proposition 4.1. Suppose u(x ((mis))) is velocity field regularized by the RKHS norm under the
following conditions: 1). The kernel function satisfies: limi K (X
(
X x(mise) = 0.
2). The density r(ximiss) is bounded. Then, the velocity field that minimizes the cost functional
FNE | (anu) [log p(x
(mis)) X (obs))] (H(r((((mis)) can be given by:
AV (mis)) K(X (miss)
X- x((mis))
(((x(mmss) = E ry ((
(8)
x (wis) log p( X
(mise)
X (obs)
)1
K(X (miss)
X
((miss)
where the expectation term E, (((((m)) can be efficiently estimated using Monte Carlo approximation, K (X,X)) is set as the radial basis function (RBF) kernel.
4.2 Sidestepping Mask Matrix: Conditional Modeling via Joint Modeling
Simulating the ODE Eq. (2) with Eq. (8) necessitates an accurate estimation of p( X (miss) ( [X((bss) However, this modeling is challenging due to the diverse choices of masking matrices. More specifically, the accuracy of the estimated conditional distribution p(X (miss) X (obs)) heavily relies
on the selection of these matrices, and these matrices should be consistent with the data missing mechanism in the testing dataset, which may be unknown in practice [51]. To bypass this difficulty,
we suggest substituting the conditional distribution p(X (miss) X (abs)) with the joint distribution p( (join)), ), where (jaint = (x (miss) X (obs)). Building on this substitution, the velocity field redefined based on the estimated joint distribution p(X (Goin) as follows:
A (mii) K(X ((X (joint) ((oin))
X (joint)
u( X (joint))
E (niin)
iniso log P( (joint)
I XX
x B
where V imiia log p( X
(joint)
can be obtained by masking the V. Vxcmm log p( X (joint)) with the missing
data indicator matrix M as follows:
X log p( X
(joint)
=V
x tiin log 인( X
Goint)
o (INXD M) + x M,
(10)
and the expression of kernel function term can be directly given based on the expression of RBF:
= =exp(- (jiin)) (jaint)
K
(joint)
X (juint)
(11)
2h
where h is the bandwidth, the values of x (joint) and X (join() are identical, and the tilde notation on x (joint) is merely used to distinguish the variable with respect to which the derivative is taken. On
( (miss)) ) and u(X (miss)) are time-var vareing functions but do not explicitly involve the evolution time 7, thus evolution time is omitted in the input variable.
5

---

this basis, similar to Eq. (10), the gradient term V
x min K ( (nini)
XX" (joint)) can be given as follows:
Eiss K(
(joint)
X )
X- K( X (joint)
(joint)
@ (INXD - M) + + M,
(12)
and since K(X (ixii)) puted by
) x
is a smooth function, X K ((X((iiin)) X ) can be easily com-
based deep learning backends like by PyTorch [42].
Proposition 4.2 demonstrates that the cost functional FiiRN.NER, associated with Eq. (10), and FNER exhibit a constant gap, indicating that optimizing FIR is equivalent to optimizing FNER.
Proposition 4.2. Assume that the proposal distribution r( r(x ( is factorized by r(X = r(X (miss)) )p( (p X(obi)). The cost functional associated with the joint distribution is defined follows:
Fjoint NER == (i [log pi X (Jodne)] ) X (((obu)))) )}
(13)
which leads to the velocity field delineated in Eq. (9) and establishes Foiii-NER as a lower bound for FNER, with the difference being a constant (i.e., Fjjint
FNER- const, const 2 0).
The detailed justification for the factorization r( r(X == r( (
)p( X (obs)) is provided in Ap-
pendix C. Based on this proposition, the following corollary can be obtained:
Corollary 4.3. The following equation holds: ul ( (( ( ((miss))
So far, we know that u( ( can reduce JNFR as effectively as possible, which indicates that the velocity field defined in Eq. (9) can fully substitute for Eq. (8) in optimizing JNER without loss of accuracy. Finally, the imputed value can be obtained by simulating the following ODE:
dx
(14)
dr
4.3 Estimating the Joint Distribution
The remaining problem is determine the estimation of score function V xoom log p( X (ioint)). To achieve this, we employ Denoising Score Matching (DSM) [21, 52] to train the score function V Vuuin log p( ((iin) parameterized by a neural network. Specifically, the learning objective is
designed to minimize the discrepancy between the actual score and the model's predicted score after
introducing Gaussian noise to the clean X
(joint) as X
(joint).
LOSM = 2 9 leiut p [II V
1
E
x tiin log p( Gjoint)
V trinn log da (X
(oint)
X (ooin) |(.)
(15)
Notably, o is variance scale, X (joint)
is obtained by X
(joint)
X
+ E, NO o²I), and
V goain log 9r (X (X (ioint) X((im))
X
Once Xumu log pl X is trained, we can
obtain the imputation value by simulating the ODE based on Eqs. (9) and (14).
4.4 Overall Workflow of NewImp
The computation workflow of NewImp is encapsulated in Algorithm 1. Specifically, we perform
mean imputation to the incomplete matrix X (ahi) producing pre-imputed dataset denoted as X (imp) (step 1). After that, we iteratively conduct DSM training and ODE simulation. In DSM training (steps 3-5), we form (join)) and conduct DSM on it acquire score estimator. In ODE simulation (steps 6-8), we set the starting point and perform ODE simulation, where u is calculated with the score estimator acquired in step 5. The endpoint is treated as the imputation results at the current iteration. After completing T iterations of this process, the imputed dataset X is calculated, where we preserve the observed indices in (obs) (step 10).
5 Experiments
5.1 Experimental Setup
Datasets: We conduct the case study based on eight real-world datasets from the UCI repository. To simulate missing data, we mask the dataset using mask matrix, which realized with a Bernoulli random variable of fixed mean. More detailed information is provided in Appendix E.

---

Algorithm The overall workflow of Newlmp approach
Input: observational data X mask matrix M.
Hyperparameter: loop time: T. simulation time: T, discretization step size 77, bandwidth h, neural network learning rate Ir, training epoch &, neural network hidden unit HU HU..ore.
score:
Output: imputed data X.
1: X (imp) +- Initialize (ob)))) @ (LNxD MM)
Initialization
2: for t = 0 to T do
X (mmiss))
X (imp)
3:
4:
X
X ((miss)
(1NxD- M) x M
5:
V Xivin
log pi X X(joiat)
DSM X X(iiia))
Algorithm
6:
X
((miss))
((mpp)
7: X (miss)
X (miss)
+ So m x (jvint) ( )d
D ODE Simulation by Appendix D.1 with Eq. (9)
8:
X ((mp)
X (miss)
9: end for
X-
((obb)
④ M
X (imp)
@ (INXD M)
Baselines: We compare Newlmp with DM-based MDI models: CSDI for Tabular Data (CSDI_ T) [51],
MissDiff [.1]. In addition, we also select other well-known MDI models like Sinkhorn (Sink) [40].
Transform Distribution Matching (TDM) [72], Generative Adversarial Imputation Nets (GAIN) [67],
Missing Data Importance- Weighted Autoencoder (MIWAE) [39], Missing data Imputation Refine-
ment And Causal LEarning (MIRACLE) [28], and ReMasker [15]. Details concerning experimental settings are given in Appendix E.2.
Implementation Details: In this study, employ two-layer multi-layer perceptron to model V X log p(x ( Each layer is configured with 256 hidden units (HUsome)), and the activation function is set as the 'Swish' function [43]. For DSM training (step 5 of Algorithm 1), the variance scale is set as 0.1, the network is trained by the Adam optimizer [26] with the learning rate of x.0 10 and the batch size is dynamically set to dataset size N. Meanwhile, for the ODE simulation part (step Algorithm 1), we specify simulation time (T) of 500, regularization strength of
(A) 10.0, step size of (1)) 0.1, and bandwidth (h) of 0.5. The loop time T for Newlmp is set as 2. Since only missing indices are updated, the evaluation focuses exclusively on imputation errors for these indices. To this end, modify the mean absolute error (MAE) and squared Wasserstein-2 distance (WASS) according to reference [22] as follows:
ΣN EØ=1
(deal) Xish @ (LNXD M)
MAE
ΣN E (INXD M)s3
M,
WASS = W/
Σ i=1
4 x(
X I..."
m
m.
where W2 denotes the squared Wasserstein-2 distance, M1 == (i subset of M,, with at least one missing value, m, is the number of data points with at least one missing value, and Ax the Dirac distribution (measure) concentrated on X.
M 0} represents the
5.2 Baseline Comparison Results
Table I presents the imputation quality of NewImp and other imputation approaches under the MAR and MCAR scenarios. The primary observations are detailed as follows:
Models with neural architectures such as MIRACLE, MIWAE, and TDM demonstrate superior performance compared to models lacking such architectures. This observation suggests that integrating neural networks into MDI tasks can significantly enhance model performance.
DM-based imputation approaches generally perform worse than other MDI methods. This outcome indicates that despite the incorporation of complex nonlinear neural architectures to boost perfor- mance, employing diversity-orient generative approaches may not align well with the precision requirements of MDI tasks.
7

---

Table 1: Overall performance of MAE and WASS metrics with 30% missing rate.
Secuario Model
BT
BCD
CC
CBV
PK
WOW
MAB
WASS
MAB
WASS
MAE
WASS
MAE
WASS]
|MA
WASS
|MAE
WASS
MAA
WASS
MAE
WASS
CSDLT
0.93
3,44
0.92
18:30
1085
2.22
0.8
3.8
0.70
16866
0.99
15.86
0.65
20.10
0.77
4.15
MissDitf
0.85
2.20
0.91
16.53
0.87
1.59
0.3
587
0.72
13.25
0.92
17.07
0.63
26.25
0.75
6.88
GAIN
0.75
0.65
0.54
1.64
0.67
0.68
0.8
0.56
1:88
0.59
1.90
0.65
5.05
0.68
0.87
MIRACLE
0.62
0.43
0.25
24,07
2.87
0.56
MIWAE
0.64
0.53
0.52
1.54
0,76
0.64
0.2
0.2
0.50
1.87
0.65
1.98
0,55
5.05
0.62
0.75
Sink
0.87
0.92
0.92
3.84
0.88
0.83
0.54
0.88
0.75
2.43
0.94
161
0.65
4.71
0.76
1.04
TDM
0.83
(1.89
0.83
3.47
0.81
0.73
0.76
0.85
0.62
1.96
0.86
3.36
0.59
4.46
0.75
0.99
ReMasker
0.52
0.43
0.43
0.49
0.37
2:23
1.59
3.81
0.59
NewImp
1.21
0.55
CSDLT T
0:73
0.73
15.51
0.85
2.71
0.83
0.76
15.19
0..22
12.42
0:57
19.89
4,11
MissDilf
0.72
3.31
13.01
14.12
GAIN
0.72
0.30
0.38
1,41
0.78
0.73
0.72
0.99
0.57
3.72
0.46
1.70
0.42
3.62
0.73
1,14
MIRACTEE
F
0.52
0.115
0.44
1.94-
0.53
0.35
0.61
0.72
2.90
$2.92
3.38
42.78
0.35
2.71
0.56
0.75
MCAR
MIWAE
0..58
(1.24
0.50
2.55
0.76
0.00
0.83
1.24
0.4
4.95
0.51
2.05
0.48
5.87
0.67 " 0.95
Sink
0.733
0.35
3.36
1,25
0.68
1.15
5.34
ReMasker
0.46
0.11
0.29
1.69
0.55
0.37
0.56
0.6
0.51
4.01
0.48
1.71
0.45
3.96
0.57
0.26
NewImp
0.48
0.15
0.25
0.80
0.47
0.34
0.42
0.44
0.44
3.05
0.32
1.01
0.34
3.66
0.53
0.76
Kindly Note: The best results are bolded and the second best results underliend. "*" marks the results
that NewImp significantly outperform with p-value< 0.05 over paired samples t-test.
Table 2: Ablation results with 30% missing rate.
Scenario
NER
Joins
BT
BCD
CC
CBV
IS
PK
QR
WQQ
MAA
WASS
MAE
WASS
MME
WASS
MAA
WASS
MAE
WASS
|MAE
WASS
MAE
WASS
MAE
WASS
0.96"
3.82"
1.08
20.2"
1.04
5.47"
0.86"
5.81"
0.67"
20.2"
1.06*
15.6"
0.72*
22.5"
0.79"
6.40
MAR
0.54
0.42
0.34
0.22
0.61"
0.40"
0.58"
0.42"
0.43"
1.34
0.46"
1,25
0.57
3.56
0.55"
0.64"
0.96"
3.33"
1.05
20.3"
1.04
5.49
0.86"
5.83"
0.67
20.2"
1.66"
15.6"
0.72*
22.5"
0,79*
6.51
0.52
0.38
0.34
0.82
0.35
0.25
0.31
0.20
0.39
1.31
0.44
1.21
0:45
3.50
0.46
0.55
0.72"
2.11
0.74"
16.7
0.85
3.72
0.83*
5.22
0.74
18.4
0.77
12.7"
0.58*
20.1"
0.76"
5.57
MCAR
0.52"
0.17
0.62"
0.46
0.61
0.71"
3,05
1.09
0.36"
1,74"
0.58"
0.82"
0.72
2.12
0.73
16.8"
0.56
3.75
0.85
5.24
0.74
18.4
0.71
12.7"
0.58*
20.1"
0.76
5.00"
0.48
0.18
0.25
0.80
0.47
0.34
0.42
0.44
0.44
3.05
0.32
1.01
0.34
3,66
0.53
0.76
Kindly Note: The best results are bolded and the second best results are underliend. "*" marks the results
that Newlmp significantly outperform with p-value< 0.05 over paired samples (-test.
Our proposed NewImp method consistently ranks as the best or second-best across most compar- isons under most of the scenarios and datasets. Notably, NewImp significantly outperforms other DM-based MDI approaches, underscoring the effectiveness of our analytical enhancements and innovations in Sections 3.1, 3.2 and 4.2.
5.3 Ablation Study Results
In this section, we conduct the ablation study to assess the contributions of two key components in NewImp: the NER term and the joint modeling strategy (referred as 'Joint'). The results of this study are detailed in Table 2. Analysis of the data between the second and last rows of Table reveals that, for most cases, in the absence of the NER, the proposal distribution r(x (miss)) may become pathological, leading to diminished model performance. Additionally, when comparing results from the first, third, and last rows, it becomes evident that modeling the joint distribution directly, rather than inferring it from the conditional distribution, significantly enhances model performance. This finding underscores the effectiveness of the strategies we have implemented, as discussed in Section 4.2. Overall, the ablation study underscores the critical roles of both the NER term and the joint distribution learning strategy in promoting the performance of NewImp.
5.4 Sensitivity Analysis Results
In this section, we analyze the impact of key hyperparameters within the NewImp approach, including the bandwidth h of the RBF kernel function, the hidden units HUscore in the score network, the weight A of the NER term, and the discretization step size v for simulating the ODE defined in Eq. (9). The profound influence of these hyperparameters on learning objectives and overall performance substantiated by the experimental results presented in Fig. 2. Initially, we explore the effects of varying
8

---

0.50
0.6
0
WASS MAE
0.8
0.45
0.1
WASS
MAE
0.3
0.6
WASS
MAE
WASS
MAE
0.2
0.2
0.35
0.1
0.25
0.5
1.0 2.0 5.0 10.0
32
64
128
256
512
0.1
0.25
0.5 1.0
2.0 5.0 10.0
32
64
128
256
512
h
HU score
h
HU scorr
00:
WASS
MAE
0.4
0.40
0.3
0.3
0.35
WASS
- MAE
0.35
WASS
.. MAE
WASS
MAE
0.2
0.2
1.0
2.0
5.0
10.0
15.0
20.0
0.005
0.05
0.1
0.2
0.5
1.0
2.0
5.0
10.0
15.0
20.0
0.005
0.05
0.1
0.2
0.5
(a) MAR with 30% missing rate at CC dataset.
(b) MCAR with 30% missing rate at CC dataset.
Figure 2: Parameter sensitivity of Newlmp on bandwidth for kernel function (h), hidden unit of score network HUscore. NER weight X, and discretization step 7) for Eq. (9) on CC dataset. Mean values and one standard deviation from mean are represented by scatters and shaded area, respectively.
the bandwidth h. We observe that an increase in bandwidth correlates with a decrease in imputation accuracy. For instance, as the bandwidth increases from 0.5 to 2.0, the MAE and WASS escalate from 0.35 and 0.25 to 0.82 and 0.74, respectively. This trend suggests that excessive bandwidth can lead to an over-smoothed velocity field, expanding the exploration space of the distribution r( ((((joia)) excessively and failing to adequately "concentrate" this distribution, ultimately diminishing examine changes in the score network's 's hidden units. Increasing the hidden units from 256 to 512 appears to decrease imputation accuracy, likely due to overfitting issues associated with larger neural networks. Next, we adjust the strength of the NER term and find that increasing its intensity generally improves imputation accuracy. This supports the necessity of the NER term, further validating its effectiveness. Lastly, we investigate the discretization step size for the ODE. We find that accuracy initially increases with smaller step sizes but then decreases. This pattern is consistent with ODE simulation behavior, where smaller step sizes require longer to converge, potentially resulting in lower accuracy within a predefined time. Conversely, larger step sizes increase discretization errors, adversely affecting accuracy as well.
performance. Subsequently,
Related Works
6.1
Diffusion Models for Missing Data Imputation
The impressive ability of diffusion models to synthesize data [54, 76, 7] has inspired extensive research into their application for MDI tasks [59, 66]. Among the pioneering efforts, the Conditional Score-based Diffusion models for Imputation (CSDI) [51] was the first to adapt diffusion models for time-series MDI, substituting the score function with a conditional distribution and pioneering a novel model training strategy by masking parts of the observational data. Building on this, to address categorical data in tabular datasets, CSDI_T [73] introduced an embedding layer within the feature extractor. To enhance inference efficiency, the conditional Schrödinger bridge method for probabilistic time series imputation proposed modeling the diffusion process as a Schrödinger bridge [10]. Mcanwhile, MissDiff [41] utilizes the missing data information as the mask matrix to improve the model training procedure.
Despite these advancements from the perspective of feature extraction module [1. 64], loss func- tion [41], and model inference approach [60], as pointed out by reference [38], the reconciliation of the inherent diversity-seeking seeking nature of diffusion models' generative processes and the accuracy- centric demands of MDI task remains underexplored. To our knowledge, this paper is the first to elucidate the relationship between diffusion models' generative processes and MDI tasks from an optimization perspective (Sections 3.1 and 3.2), which has not been discovered by previous refer- ence [38]. Based on these insights, we further propose our NewImp approach by designing the NER term to prioritize the MDI accuracy (Section 3.2).

---

6.2 Modeling Conditional Distribution by Joint Distribution
Modeling conditional distribution as joint distribution remains an opening question and has broad potential for application [68, 8, 25]. Conditional sliced WGF [14] first empirically validated that the velocity field of joint distribution and conditional distribution are identical when choosing sliced Wasserstein distance as cost functional. After that, reference [25] extended this relationship and derived the relationship between conditional and joint distribution in various discrepancy metries like f-divergence, Wasserstein distance, and integral probability metrics. On this basis, reference [19] further theoretically proved the equivalence of velocity fields for conditional and joint distribution.
However, the objective of NewImp does not belong to any kind of discrepancy metric [25]. The most similar discrepancy metric negative KL divergence. Notably, negative KL divergence contains diversity-encouragee study is encouraging 'positive' entropy as the regularization term, and the regularization term in our 'negative' entropy, and thus more than directly applying these results to our research is needed. On this basis, our theoretical contribution proves that this joint distribution modeling approach can still be applied when the functional regularized by the negative entropy.
6.3 Wasserstein Gradient Flow for Generative Modeling
WGF [2, 46] has been extensively employed in various domains of machine learning, including generative modeling [17, 12, 74, 63], posterior distribution sampling [55, 35, 32, 34], and domain adaptation [75, 36, 37, 71]. In generative modeling [4, 11, 18], the problem is framed as an op- a f-divergence term that measures the discrepancy between the proposal distribution and the data distribution, alongside an entropy term that promotes diversity in generative results.
timization task, with the objective functional comprising an
WGF is then utilized to address the optimization of this cost functional, with models being constructed during the solution process. However, indicated by our illustrative example (Section 3.1), and further supported by our theoretical analysis (Section 3.2), pursuing diversity in accuracy-oriented tasks such MDI may not be appropriate. Our analysis reveals that the inclusion of an entropy term in the cost functional makes the direct application of diffusion models to MDI tasks unsuitable. Based on these insights, one of our major contributions demonstrating that WGF can be effectively used to analyze and improve the appropriateness of applying diffusion models to non non-generative tasks.
7 Conclusions
This work demonstrated that directly applying diffusion models to MDI resulted in suboptimal performance due to unintended diversity and the requirement for data masking, both of which impeded accurate imputation. To counteract this, we proposed NewImp, a novel diffusion model-based MDI approach within the Wasserstein gradient flow framework, designed to suppress unintended diversity. We developed an easy-to-implemet form for realizing NewImp in computer code by constraining the velocity field within the reproducing kernel Hilbert space. Furthermore, we proved that the imputation procedure of NewImp could be derived from an equivalent joint-distribttion related functional, thereby obviating the need for data masking. Finally, extensive experiments demonstrated that NewImp effectively mitigates these issues and outperforms prevalent baseline models.
Acknowledgement
This work was supported in part by the National Natural Science Foundation of China (NSFC) under Grants 62473103 and 623B2002. The first author Zhichao Chen and the corresponding author Hao Wang would like express their sincere gratitude PhD Candidate Weiming Liu at Zhejiang University for valuable discussions on the implementation of the FPK equation via ODE/SDE.
Dedicated to the t00kh Anniversary of Sun Yat-sen University, the Alma of Zhichao Chen.
10

---

References
[I] Juan Lopez Alcaraz and Nils Strodthoff. Diffusion-based time series imputation and forecasting with structured state space models. Trans. Mach. Learn. Res., 2022.
[2] Luigi Ambrosio, Nicola Gigli, and Giuseppe Savaré. Gradient flows: in metric spaces and in the space of probability measures. Springer Science Business Media, 2005.
[3] Brian DO Anderson. Reverse-time diffusion equation models. Stoch. Process. their Appl., 12 (3):313-322 --131,3666622 1982.
[4] Abdul Fatir Ansari, Ming Liang Ang, and Harold Soh. Refining deep generative models via discriminator gradient flow. Proc. Int. Conf. Learn. Represent., pages 1-24, 2021.
[5] Matthew James Beal. Variational algorithms for approximate Bayesian inference. University of London, University College London (United Kingdom), 2003.
[6] John Charles Butcher. Nunerical methods for ordinary differential equations. John Wiley & Sons, 2016.
[7] Hanqun Cao, Cheng Tan, Zhangyang Gao. Yilun Xu, Guangyong Chen, Pheng-Ann Heng, and Stan Z. Li. A survey on generative diffusion models. IEEE Trans. Knowl. Data Eng., 36(7): 2814 2830, 2024. doi: 10.1 1..1099TKDDEE2224333177777
[8] Jannis Chemseddine, Paul Hagemann, Christian Wald, and Gabriele Steidl. Conditional wasser- stein distances with applications in bayesian ot flow matching. arXiv preprint ar axXiv:2403.1877 pages 1-42, 2024.
[9] Changyou Chen, Ruiyi Zhang, Wenlin Wang, Bai Li, and Liqun Chen. A unified particle- optimization framework for scalable bayesian sampling. arXiv preprint arXiv: 1805. /659, pages 1-13, 2018
[10] Yu Chen, Wei Deng, Shikai Fang, Fengpei Li, Nicole Tianjiao Yang. Yikai Zhang, Kashif Rasul, Shandian Zhe, Anderson Schneider, and Yuriy Nevmyvaka. Provably convergent schrödinger bridge with applications to probabilistic time series imputation. In Proc. Int. Conf. Mach. Learn., pages 4485 1315-4513.210 2023.
[11] Xiuyuan Cheng, Jianfeng Lu, Yixin Tan, and Yao Xie. Convergence of flow-based generative models via proximal gradient descent in wasserstein space. IEEE Trans. Inf. Theory, pages 1-1, 2024. doi: 10. 10109/TTT2224.3222411
[12] Jaemoo Choi, Jaewoong Choi, and Myungjoo Kang. Scalable Wasserstein gradient flow for generative modeling through unbalanced optimal transport. In Proc. Int. Conf. Mach. Learn., pages 8629 6629-8650, 2024.
[13] Hanze Dong, Xi Wang, LIN Yong, and Tong Zhang, Particle-based variational inference with preconditioned functional gradient flow. In Proc. Int. Conf. Learn. Represent., pages 1-26, 2022.
[14] Chao Du, Tianbo Li. Tianyu Pang. Shuicheng Yan, and Min Lin. Nonparametric generative modeling with conditional 8565 8584,,888,441 2023.
flows. In Proc. Int. Conf. Mach. Learn., pages
[15] Tianyu Du, Luca Melis Melis, and Ting Wang. Remasker: Imputing tabular data with masked autoencoding. In Proc. Int. Conf. Learn. Represent. pages 1-23, 2024.
[16] Lawrence C Evans. Partial differential equations, volume 19. American Mathematical Society, 2022.
[17] Jiaojiao Fan, Qinsheng Zhang, Amirhossein Taghvaei, and Yongxin Chen. Variational Wasser- stein gradient flow. In Proc. Int. Conf. Mach. Learn., pages 6185-6215, 2022.
[18] Yuan Gao, Yuling Jiao, Yang Wang, Yao Wang, Can Yang. and Shunkang Zhang. Deep generative learning via variational gradient flow. In Proc. Int. Conf. Mach. Learn., pages 2093-2101, 2019.
11

---

[19] Paul Hagemann, Johannes Hertrich, Fabian Altekrüger, Robert Beinert, Jannis Chemseddine, and Gabriele Steidl. Posterior sampling based on gradient flows of the MMD with negative distance kernel. In Proc. Int. Conf. Learn. Represent, pages 1-32, 2024.
[20] Ya-Ping Hsieh, Ali Kavis, Paul Rolland, and Volkan Cevher. Mirrored Langevin Dynamics, In Proc. Adv. Neural Inf. Process. Syst. pages 1-10, 2018.
[21] Aapo Hyvärinen. Estimation of non non-normalized statistical models by score matching. J. Mach. Learn. Res., 6(24))695-709, 2005.
[22] Daniel Jarrett, Bogdan C Cebere, Tennison Liu, Alicia Curth, and Mihaela van der Schaar. Hyperimpute: Generalized iterative imputation with automatic model selection. In Proc. Int. Conf. Mach. Learn., pages 9916-9937, 2022.
[23] Richard Jordan, David Kinderlehrer, and Felix Otto. The variational formulation of the fokker- planck equation. SIAM J. Math. Anal., (911:1111 1998.
[24] Valentin Khrulkov, Gleb Ryzhakov, Andrei Chertkov, and Ivan Oseledets. Understanding DDPM latent codes through optimal transport. In Proc. Int. Conf. Learn. Represent. pages
1 1-15, 2022.
[25] Young-geun Kim, Kyungbok Lee, and Myunghee Cho Paik. Conditional Wasserstein Generator. IEEE Trans. Pattern Anal. Mach. Intell., 45(6),7088-7211,, 2023, doi: 10.1109/TPAML.222.. 3220965.
[26] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proc. Ini. Conf. Learn. Represent., pages 1-8, 2015.
[27] Diederik P Kingma and Max Welling. Auto-encoding Variational Bayes. In Proc. lnt. Conf. Learn. Represent., pages 1-8, 2014.
[28] Trent Kyono, Yao Zhang, Alexis Bellot, and Mihaela van der Schaar. MIRACLE: Causally- aware imputation via learning missing data mechanisms. Proc. Adv. Neural Inf. Process. Syst., pages 18800-238111 2021.
[29] Haoxuan Li, Kunhan Wu, Chunyuan Zheng, Yanghao Xiao, Hao Wang, Zhi Geng, Fuli Feng, Xiangman He, and Peng Wu. Removing hidden confounding in recommendation: unified multi-task learning approach. Proc. Adv. Neural Inf. Process. Syst., pages 1-13, 2024.
[30] Haoxuan Li, Chunyuan Zheng, Shuyi Wang, Kunhan Wu, Eric Wang, Peng Wu, Zhi Geng, Xu Chen, and Xiao-Hua Zhou Relaxing the accurate imputation assumption in doubly robust learning for debiased collaborative filtering. In Proc. Int. Conf. Mach. Learn., pages 29448- 29460, 2024.
[31] Yingzhen Li and Richard Turner. Gradient estimators for implicit models. In Proc. Int. Conf. Learn. Represent., pages 1-19, 2018
[32] Chang Liu, Jingwei Zhuo, Pengyu Cheng, Ruiyi Zhang, and Jun Zhu. Understanding and accelerating particle-hased variational inference. In Proc. Int. Conf. Mach. Learn., pages 4082-4092. PMLR, 2019.
[33] Diju Liu, Yalin Wang, Chenliang Liu, Xiaofeng Yuan, Kai Wang, and Chunhua Yang. Scope- free global nulti-condition-awwee industrial missing data imputation framework via diffusion transformer. IEEE Trans. Knowl. Data Eng., pages 1-12, 2024. doi: 10. 109/TKDE,2024 3392897.
[34] Qiang Liu. Stein variational gradient descent gradient flow, In Proc. Adv. Neural Inf. Process. Syst., volume 30, pages 1-15, 2017.
[35] Qiang Liu and Dilin Wang. Stein variational gradient descent: A general purpose bayesian inference algorithm. In Proc. Adv. Neural Inf. Process. Syst., volume 29, pages 1-13, 2016.
[36] Weiming Liu, Jiajie Su, Chaochao Chen, and Xiaolin Zheng. Leveraging distribution alignment via stein path for cross-domain cold-start recommendation. In Proc. Ady. Neural Inf. Process. Syst., volume 34, pages 19223-19234, 2021.
12

---

[37] Weiming Liu, Xiaolin Zheng, Jiajie Su, Longfei Zheng, Chaochao Chen, and Mengling Hu. Contrastive proxy kernel stein path alignment for cross-domain cold-start recommendation. IEEE Trans. Knowl. Data Eng., 35(1))5(1))))
11230, 2023. doi: 10.1 10.1109/TKDEE.222222337777
[38] Yixin Liu, Thalaiyasingam Ajanthan, Hisham Husain, and Vu Nguyen. Self-supervision improves diffusion models for tabular data imputation. In Proc. ACM Int. Conf. Inf. Knowl. Manag., pages 1-10, 2024.
Pierre-Alexandrr Mattei and Jes Frellsen. MIWAE: Deep generative modelling and imputation [39]
of incomplete data sets. In Proc. Int. Conf. Mach. Learn., pages 4413 2019.
[40] Boris Muzellec, Julie Josse, Claire Boyer, and Marco Cuturi. Missing data imputation using optimal transport. In Proc. Int. Conf. Much. Learn., pages 7130-7140, 2020.
[41] Yidong Ouyang, Liyan Xie, Chongxuan Li, and Guang Cheng. MissDiff: Training diffusion models on tabular data with missing values. In Proc. Int. Conf. Mach. Learn. Workshop on Structured Probabilistic Inference& Generative Modeling, 2023.
[42] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An imperative style, high-performance deep learning library. In Proc. Adv. Neural Inf. Process. Syst., volume 32, pages 1-12. 2019.
[43] Prajit Ramachandran, Barret Zoph. and Quoe V Le. Searching for activation functions. arXiv preprint arXiv: arXiv:1710.05941, pages 1-13, 2017.
[44] Donald B Rubin. Inference and missing data. Biometrika, 63(3):811552, 1976.
[45] Walter Rudin et al. Principles of mathematical analysis, volume 3. McGraw-hill New York, 1964.
[46] Filippo Santambrogio. (Euclidean, Metric, and Wasserstein) gradient flows: an overview. Bull. Math. Sci., 7:87-1544 154, 2017.
[47] Simo Särkkä and Arno Solin. Applied stochastic differential equations, volume 10. Cambridge University Press, 2019.
[48] Jiaxin Shi, Chang Liu, and Lester Mackey. Sampling with Mirrored Stein Operators. Proc. Int. Conf. Learn. Represent., pages 1-26, 2022.
[49] Yang Song, Sahaj Garg, Jiaxin Shi, and Stefano Ermon. Sliced score matching: A scalable approach to density and score estimation. In Proc. Conf. Uncertainty in Artificial Intelligence, pages 574-584, 2020.
[50] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Proc. Int. Conf. Learn. Represent., pages 1-36, 2020.
[51] Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. CSDI: Conditional score-based diffusion models for probabilistic time series imputation. Proc. Adv. Neural Inf. Process. Syst., pages 24804 24816, 16, 2021.
[52] Pascal Vincent. connection between score matching and denoising autoencoders. Neural Comput., 23(7):6611-1674, 2011.
[53] Dilin Wang and Qiang Liu. Nonlinear stein variational gradient descent for learning diversified mixture models. In Proc. Int. Conf. Mach. Learn., pages 6576-6585, 2019.
[54] Fangyikang Wang, Hubery Yin, Yuejiang Dong, Huminhao Zhu, Chao Zhang, Hanbin Zhao, Hui Qian, and Chen Li. BELM: Bidirectional explicit linear multi-step sampler for exact inversion in diffusion models. In Proc. Ady. Neural Inf. Process. Syst., pages 1-33, 2024.
13

---

[55] Fangyikang Wang, Huminhao Zhu, Chao Zhang, Hanbin Zhao, and Hui Qian. GAD-PVI: A general accelerated weight particle-baser variational inference framework. In Proc. AAAI Conf. Artif. Intell., pages 11466-15473, 2024.
[56] Hao Wang, Jiajun Fan, Zhichao Chen, Haoxuan Li, Weiming Liu, Tianqiao Liu, Quanyu Dai, Yichao Wang, Zhenhua Dong, and Ruiming Tang. Optimal transport for treatment effect estimation. In Proc. Adv. Neural Inf. Process. Syst. pages 5404-5418, 2023.
[57] Hao Wang, Zhichao Chen, Zhaoran Liu, Haozhe Li, Degui Yang, Xinggao Liu, and Haoxuan Li. Entire space coumterfactual learning for reliable content recommendations, IEEE Trans. Inf. Forensics Secur. pages 1-1, 2024.
[58] Hao Wang, Zhichao Chen, Zhaoran Liu, Licheng Pan, Hu Xu, Yilin Liao, Haozhe Li, and Xinggao Liu. SPOT-I: Similarity preserved optimal transport for industrial iot data imputation. IEEE Trans. Ind. Inform., pages 1-9, 2024. doi: 10.1109/11.22243452244
[59] Jun Wang, Wenjie Du, Wei Cao, Keli Zhang, Wenjia Wang, Yuxuan Liang, and Qingsong Wen. Deep learning for multivariate time series imputation: A survey. arXiv preprint urXiv:2402.04059 pages 1-9, 2024.
[60] Xu Wang, Hongbo Zhang, Pengkun Wang. Yudong Zhang, Binwu Wang, Zhengyang Zhou, and Yang Wang. An observed value consistent diffusion model for imputing missing values in multivariate time series. In Proc. ACM SIGKDD lnt. Conf. Knowl. Discovery Data Mining, pages 2409-2418, 2023.
[61] Yifei Wang and Wuchen Li. Accelerated information gradient flow. J. Sci. Comput., 90:1 2022.
[62] Max Welling and Yee W Teh. Bayesian learning via stochastic gradient langevin dynamics. In Proc. Int. Conf. Mach. Learn.. pages 681-688. Citeseer, 2011.
[63] Chen Xu, Xiuyuan Cheng, and Yao Xie. Normalizing flow neural networks by JKO scheme. In Proc. Adv. Neural Inf. Process. Syst. pages 47379-47405, 2023.
[64] Jingwen Xu, Fei Lyu, and Pong C Yuen. Density-aware temporal attentive step-wise diffusion model for medical time series imputation. In Proc. ACM Int. Conf. Inf. Knowl. Manag., pages 2836-2845, 2023.
[65] Ling Yang. Zhilong Zhang, Yang Song, Shenda Hong. Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. Diffusion models: comprehensive survey of methods and applications. ACM Comput. Surv., 56(4):1-39, 2023.
[66] Yiyuan Yang, Ming Jin, Haomin Wen, Chaoli Zhang, Yuxuan Liang, Lintao Ma, Yi Wang, Chenghao Liu, Bin Yang, Zenglin Xu, et al. A survey on diffusion models for time series and spatio-temporal data. arXiv preprint arXiv:2404. 018.4444.88800 18886, pages 1-27, 2024.
[67] Jinsung Yoon, James Jordon, and Mihaela Schaar. GAIN: Missing data imputation using generative adversarial nets. In Proc. Int. Conf. Mach. Learn., pages 5689-5698. PMLR, 2018.
[68] Shipeng Yu, Kai Yu, Volker Tresp. Hans-Peter Kriegel, and Mingrui Wu. Supervised probabilis- tic principal component analysis. In Proc. ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, pages 464-473, 2006.
[69] Chao Zhang, Zhijian Li, Xin Du and Hui Qian. Dpvi: A denamic-weight based variational inference framework. In Proc. Int. Joint Conf. Artif. Intell., pages 4900-4906, 2022.
[70] Cheng Zhang, Judith Bütepage, Hedvig Kjellström, and Stephan Mandt. Advances in variational inference. IEEE Trans. Pattern Anal. Mach. Intell., 2108):2088-203 2026, 2019. doi: 10.1 109/ TPAMI,2018. 0.200088888888 28899774
[71] Yulong Zhang, Shuhao Chen, Weisen Jiang, Yu Zhang, Jiangang Lu, and James T Kwok. Domain-guided conditional diffusion model for unsupervised domain adaptation, arXiv preprint arXiv:2309.14360 pages 2023.
2023.
14

---

[72] He Zhao, Ke Sun, Amir Dezfouli, and Edwin V Bonilla. Transformed distribution matching for missing value imputation. In Proc. Int. Conf. Mach. Learn., pages 42159-42186, 2023.
[73] Shuhan Zheng and Nontawat Charoenphakdee. Diffusion models for missing value imputation in tabular data. In Proc. Adv. Neural Inf. Process. Syst. Workshop on First Table Representation, 2022.
[74] Huminhao Zhu, Fangyikang Wang. Chao Zhang, Hanbin Zhao, and Hui Qian. Neural sinkhorn gradient flow. arXiv preprint arXiv:2401.14069, pages 1-17, 2024.
[75] Zhan Zhuang, Yu Zhang, and Ying Wei. Gradual domain adaptation via gradient flow. In Proc. Int. Conf. Learn. Represent., pages 1-27, 2024.
[76] Zhan Zhuang, Yulong Zhang, Xuehao Wang, Jiangang Lu, Ying Wei, and Yu Zhang, Time- Varying LoRA: Towards effective cross-domain fine-tuning of diffusion models. Proc. Adv. Neural Inf. Process. Syst., pages 1-25, 2024.
15

---

Appendix Contents
Detailed Preliminaries of Wasserstein Gradient Flow
17
Detailed Information for Toy Cases in Section 3.1
18
Theoretical Analysis
19
C.1 Implementation Difficulty Velocity Field
19
C.2 Proof & Discussions of Concerning Propositions and Corollaries
19
D Detailed Explanation for the Workflow of Newlmp Approach
28
D.1 Forward Euler's Method for ODE Simulation Detailed Information for DSM
28
D.2
28
Detailed Information for Experiments
29
E.1
Background & Simulation of Missing Data
29
E,2
Hyperparameter Setting of Baseline Models
30
Additional Empirical Evidence
31
F.1
Toy Case Experiments
................................
31
F.2
Additional Experimental Results with MNAR Scenario
................................
31
F3
Empirical Evidence for Selecting RBF Function
................................
34
F44
Time Complexity Analysis
35
F.5
Convergence Analysis
36
F6
Downstream Task Comparison
42
F77
Baseline Comparison Vary Different Missing Rates and Scenarios
43
G Limitations & Future Directions and Broader Impact
43
G.1
Limitations Future Directions
43
G.2
Broader Impact Statement
44
16

---

Appendix A Detailed Preliminaries of Wasserstein Gradient Flow
In this section, we want introduce the WGF framework and its application scenarios better understand this paper. Before introduction, the following concepts are listed to better understand the WGF framework:
1. Wasserstein Metric: Let P2 PR represent the space of probability measures on RD that possess finite second moments, Formally, this expressed as P2( {u M(BP) | [[]]=]] < oo), where (RRP) denotes the set of all probability measures on IRD Considering any two probability measures μ,, V E ()) we define the Wasserstein-p distance between and v as follows:
Wp {
inf / (ET())))) 22
||x dll
(A.1)
Here, T(2, v)) represents the collection of all joint distributions (couplings) between μ and For every joint distribution æ E [(,, it holds that μ(z) = Sg0 ( y) dy and v Jgp <xx y) dz. The integral on the right-hand side encapsulates the transportation cost in the optimal transport (OT) problem, framed by Kantorovich's formulation, where x denotes the optimal transportation plan.
Furthermore, leveraging Jensen's inequality facilitates demonstrating the monotonicity of the Wasserstein-p distance, affirming that for 1 S p Σ q. the relationship Wp(s, v) < W, v) invariably holds. Building this principle, we articulate the inner product within the measurable space (P2(IP)) W) as delineated below:
(1,, v)n, - -fo (1,, V)Rodjr
(A.2)
2. Gradient Flow in Wasserstein Space: Consider a functional F associated with μ P2(RD). Our objective is to identify the optimal j that minimizes F:
min F(u) const.
(A.3)
MED)(RD)
To facilitate the decrease of F(u), we introduce velocity field My R - RD designed to expedite the reduction of F(u) μ evolves under this field. Utilizing the chain rule yields:
dF((p)
FF())) ôp
upp)
du.
(A.4)
dr
where 8 represents the first variation operator. To ensure the decrease of F(p), i.e. dF(a) the velocity field is defined as:
dr <
8F(p)
(A.5)
The decline of F(4) aligns with the following partial differential equation (PDE) called the continuity equation:
Op = = (mum))
(A.6)
0r Hence, the continuity equation Eq. (A.6), coupled with the velocity field articulated in Eq. (A.5). recognized the Wasserstein Gradient Flow, delineating the steepest descent direction of cost functional F(4) in the Wasserstein space.
Simulation of WGF & Sampling: There are primarily two discretization techniques for the WGF: the forward scheme and the backward scheme.
Forward Scheme: The forward scheme applies gradient descent within the Wasserstein space identify the direction of the steepest descent. For an energy functional F(u) with specified step size 1), the update rule in the forward scheme is formulated as:
5FF(p)
Mr+1 (Id
(A.7)
facilitating an intuitive and direct update mechanism that emulates the gradient flow in the Euclidean space but transposed into the Wasserstein space.
17

---

Backward Scheme: Conversely, the backward scheme, often referred to as the Jordan- Kinderlchrer-Otte scheme [23], represents a more implicit discretization approach. It defines the subsequent distribution M+++ by solving an optimization problem that balances the energy decrease and the transportation cost. This scheme is mathematically denoted as:
M7+11 arg F(D) + W22 (1,,),)),
(A.8)
(EPx(RD)
2n
thereby integrating the energy minimization and transport efficiency into a single variational problem that reflects the inherent structure of the Wasserstein space.
These schemes provide distinct yet complementary approaches to discretizing the dynamics defined WGFs, offering different perspectives and tools for the analysis and computation of these flows.
Leveraging the WGF framework, if designate the functional F to be the KL divergence, it yields a particular formulation for the velocity field.
ŌT x Vlogp log p.
Up
(A.9)
ōμc
On this basis, plug Eq. (A.9) into Eq. (A.6), can the following PDE:
(wVlogp) + Vy.
(A.10)
ÔT
According to Theorem 5.4 in reference [47]. denote the random sample from distribution p as we can obtain the following stochastic differential equation (SDE) called Langevin equation [62] for implementing this gradient flow easily:
dr Vlogp(z) V2dW,,
(A.11)
where dW, is the standard Wiener process (also known as Brownian motion).
Appendix B Detailed Information for Toy Cases in Section 3.1
To investigate what would happen if we directly applied diffusion models to MDI tasks, we consider the following optimization problem:
r
1= Pk
arg EA²
log
Σ (pe 1) log ak.h
H 8, 8, [2.5, 2.5,2.55500 5.0),
k== (pk)
k=1
(B.1)
which corresponds to the density function of Dirichlet distribution, Dir([2.5, 2.5, 5.0]), where lies on the three-dimensional standard simplex A The optimal value of FDir is given by:
P1 Σ
P2
P3
[0.214, 2.211,0055
(B.2)
ビ Ei
Σ Σk=1 Ph
To optimize this cost functional, employ the Langevin equation presented in Eq. (A.11):
da = Va, Foad - V2dW,.
(B.3)
and compare the results to Eq. (B.2) to evaluate effectiveness. Additionally, since the support is on three-dimensional standard simplex 2 to ensure the well-definedness of our approach, we use the mirror descent technique [48, 20]. where the Bregman function is defined as vi(a))
Ek=1 ah.kk log as, ab,k- Moreover, the optimization of
an 1 is conducted by simulating the Langevin equation, which discretized by the Fuler-Maruyama method follows:
a
a7 exp (1) a. JDirla, oz x 1//26) N( T3x3)
T r+1
(B.4)
r+1
T+1 1k=1
With the step size 7) set 5.0 x 10-3, we repeatedly execute Eq. (B.4) 100 times, culminating the results depicted in Fig. (b).
18

---

Appendix C Theoretical Analysis
C.1 Implementation Difficulty of Velocity Field
To the best our knowledge, the difficulty of implementing the velocity field can be given from two perspectives, namely ODE-based implementation and want to discuss these two implementation approaches in detail.
implementation. In this section,
ODE-based Implementation:
1. WGF framework: According to the continuity equation, obtain the following velocity field:
dx (miss)
(miss)
x iniss) log p(x (miss)
X (obx)) + AV X log logr((2 ( (m(ss)s)) } (C.1)
dr
where (i) is based Section 2.3, and (ii) is based on Section 4.1. The expression of the velocity field involves the computation of density term r(X(miss) [31, 9], which is intractable during
practice as we stated in Section 2.3. Based on this, we conclude that implementing this velocity field within the WGF framework difficult.
2. Probability flow ODE: According to reference [50], if we directly plug Eq. (8) into the FPK equation, we can get the following PDE:
Ör (miss))
(u( (miss)
)r( x (miss)))
V Ximos log X (miss)
X (obs) )r( (miss))
AV Vr(X (msss)
V mis log p( X (miss)
X (obs) )r( (X (mmmss))
}]
XV Vr (miss))
(C.2)
2 oo Vr (msss)
-Z Vr (x (miss))
V X log p X (miss)
( (miss)
+ (A
2 o V log r( (
r r(X(imis))
(miss)).
When we set o as we can find that the corresponding ODE is Eq. (C.1), where we are obliged to compute the intractable density r( (miss)).
baseb Implementation:
If we plug Section I into the FPK equation, the corresponding PDE can be given as follows:
dr (r(x (miss))
Ōr
V (u( (mmss))
r X (miss))
(C.3)
V omo log o
X (miss)
X (obs) )r X (miss)
AV Vr(Xx(miss)
where the coefficient before the Laplacian operator V V is -1. To the best of our knowledge, this structure makes deriving corresponding SDE impossible by current approaches.
C.2 Proof& & Discussions of Concerning Propositions and Corollaries
Proposition (3.1), Within WGF framework, DM-based MDI approaches can be viewed as finding
the imputed values X (imp) that maximize the following objective:
arg max x(X (NN))
Ex(X [log ((X (mins)
(ahs) } + 20 X (mics) const,
(C.4)
where 'const' the abbreviation of constant, and ub( (mits)) is scalar fcalatffuncton function determined by the
type of SDE underlying the DMs.
19

---

VP-SDE: 2 ( (miss)) = H ( (m)ss))) + E,, x (m00)) [ (
T
X (miss)))
VE-SDE: 2/ (miss)) H (X (mmiss)) )]
sub-VP-SDE: 25 (miss)) | (x (miss)) + E (x (wews)
X (
T
[X (miss))
where Hl[r(X (X ((imiss)) == - ( (mmis))) log X (miss)) ) xd is the entropy term, 7r is determined
by noise scale B+: B+:'+= == (1 (1- exp( (1-exp(-225) 3,ds)) > 0, < 31 .... BT < L.
Proof. Since there are various approaches for reversing the sampling procedure of DMs, for simplicity. as we emphasized in Section 3.2, we mainly consider the VP-SDE, VE-SDE, and Lub-VP-SDE as analysis objects in this paper.
VP-SDE: According to reference [50], the density evolution of the generative process for VP-SDE be delineated by the following PDE:
(
}
Xemioo
r (miss))
[Rr]
X
(miss)
+ log ((x (miss)
X (obb))
θr
(C.5)
Br V Xinsg
V X
X (miiss))
where 3+ E (0, is the time-varying noise scale. On this basis, according to [24], by changing the variable dr 2/ dr, we get the following equation:
r (miss))
x (miss)
V log pl (miss) x((os))
Ör Ör(X (X
X
(C.6)
2 (mia log r (mis)))
Comparing Eq. (C.6) with Eqs. (A.5) and (A.6), the cost functional to be minimized of this simulation procedure can be given as follows:
x mmsss)
X (miss))
+ log o( X (miss) x (ahs)
FVP.SDE = -
r( ((X((mms)) (mmsss)
4
d d X (miss)
log (x (miss)
+ const
(C.7)
X (misss) T
X (miss))
+ log p(
X (obs)
--Br(ximug
1
2 log r(X (imiss) const
Note that 1 (imiss)) X
(miss)
and f( (miss)) log r X (miss) d X (miss) > O hold, and thus
the proposition for VP-SDE is proved by taking the negative of the abovementioned equation.
VE-SDE: Similarly, based on reference [50], the following PDE can be given to delineate the
density evolution the generative process for VE-SDE:
Ör( X (miss))
fr(
do?
V x imio
(a(isnn)))) (iniss)
innis log (misss) X (ubbs))
dr
(C.8)
do?
V (mia
ximiio (mmiss)).
2 dr
where of is time varying noise scale.
dr [24]. Eq. (C.8) can be reformulated as do
As such, by chaning the variable as dr
dr
follows:
Ör (r (miiss) (mmis))
V Ximmo r( r(X (miss))
Ximma logg (mnis)) (obb))
xcmoo log r( (m(isss)
(C.9)
20

---

Comparing Eq. (C.9) with Eqs. (A.5) and (A.6), the cost functional to be minimized of this simulation procedure can be given follows:
1
FVE-SDE=
r(x (
log r( (X (mixs) log (XX (miss))
X (nbs)) const
dx (miss)
2
(C.10)
log r( (xx(miss)) + log p X (miss) X + const
Note that the entropy function f [r( ( (((iss)) log r X ((miss)) proposition for VE-SDE proved by taking the negative of the abovementioned equation.
) ) ( > 0 holds, and thus the
sub- VP-SDE: Based on reference [50], the following PDE can be given delineate the density evolution of the generative process for sub- VP-SDE:
0r(X (miss)
1 X (miss)
}
r
[8+]
+ 7/ V x Xim log pi( X (miss)
X X(obs)
X giiee
Ör
Br 2
+
74 V X
V X X (miss))
(C.11)
where 7/- == (1- - exp(-2) Lo 3,ds))
> On this basis, by chaning the variable as dr = dr,
we can get the following equation:
(inss)s))
(miss) + V x imo log p (miss) x
dr( Or(x
x (miss))
V x (in
(C.12)
Or
V xima log r( x (miss))
Comparing Eq. (C.12) with Eqs. (A.5) and (A.6), the cost functional to be minimized of this simulation procedure can be given follows:
(miss)
X (miss)
+ 7/r log p( X-
(miss) (obs))
F -U--VD-
/
r(x (miss))
4
dXx (miss)
Yr
log r( (X (imise))
+ const
X (miss))
[x (misss)
+ 7r log p( X (misss)
X (obs))
Lr log r( r(x (miis)) + const
2
1
X (imiss)
[x (miss)] log ( (xx (miss) ((bbs)
47
-EE, ((iis)))
log r( (orr(x(misn (miss))
+ const
(C.13)
d 0 and f [r( ((( log r( X X x (miss) > 0 hold, and thus the proposition for sub-VP-SDE is proved by taking the negative of the abovementioned equation.
Note that X (miss)
T
X (miss)
In summary, the regularization term 4 (misx)) for VP-SDE is Ep(xinisyy [X (miss)) T X (miss)))
[i[(( (miss) ]]
for VE-SDE is a (r((( ( (miss))) and for sub- VP-SDE is (miss)
Ex
T (miss)
(miss))
[ (miss)))
Before proving Proposition 4. want introduce the following lemma delineate the evolution of cost functional FNER along time T:
Lemma C.I. The evolution of FNER along time T be characterized by the following ODE,
assuming that the boundary condition lim [u[ (u(x((fmnn X(miss) )r( (miss)) 0 is satisfied:
dJ
dr (x((ni)))) (X-(mian)) V (miss) log p( X (mise)
x (obs)
AV
x X(mi)) u(x((mis))) J. (C.14)
This boundary condition is achievable, for instance, when r( X (miss)) ) is bounded, and the limit of the velocity field as the norm of X (miss) approaches infinity is zero (limy xoudi u(x u(x(miss) 0).
21

---

Proof. Before proving this lemma, we should recognize that the evolution of X (miss) should promise the probability density function r(x (mis)) unchanged. In other words, the following continuity equation should be satisfied during the optimization of r(x (mis)):
r(x(mi)))
Ör X (miss)
Vx x (min)
[r(X (miss) ul (x (..
(miss)))s))
(C.15)
On this basis, the evolution of FNER along time T, dJMME can be given as follows based on the chain rule:
d_
dr
Ör X (miss)
llog p(x (miss)) X ((bb)) ) + x log r (miss)) ) + 2] dx (miss)
V X (nin)
|r(XX (miss))
(u( x (misss) |] [log (miss)
X (obs)) A log r (x (x (miss)) + NJelX (miss)
(
(r(x (miss))
)u( x (miss))
( [log pi X (miss)
X (abs)) Alog X (miss) + Ajdx (miss)
(r (miss)
( (miss)))
) (V (mis)) [log p X (miss)
X (nbs)) + A log (X (miss)) (miis)) ]]dxX (miss)
/ [u(X X (x(ss)ssss]]
(r( X (miss)
V
x (miss)
log p (miss)
X (ahs)) + Ar (miss) V x (min) log r' (X (miss)
]d (miss)
(m( X (
X (miss)
log p( X (miss)
X (abs)) + (ama) (X (miss) (miss) (miss)
(miss)
(
r( x (miss))
[u"
x (miss)
log p X (miss)
X (abs))
u X (miss) ]] (miss)
X (niss)
X (mis))
= (x (um
(x (miss))
V
x (mi)) log p X (miss) X (abs))
AV
x (m))))
u X (miss)))
(C.16)
where (i) and (ii) are based on integration by parts. More specifically, when condition
lim ooo [u( (((iss)) X [ = is satisfied, for example, r (miss)) bounded, and the limit
(
of the velocity field as the norm of X (miss) approaches infinity is zero (lim | u (missi)) = 0),
we can get the following result [35, 32]:
f x inin ( yu( (x (misss) ) J
(miss)) (miss) 0,
where the left-hand-side can be further decomposed as follows based on the integration by parts:
f V Xtnin [r(x (mis)) (miss)))(
)u( X (miss))
)d
f uu' (x(miss)) V Xinsor
X (miss))
d (miss)
+ V iina 21 ux (miss))) )r (x (miss) dX (miss)
Based on Lemma C.1, we can now start proving Proposition 4.1:
Proposition (4.1). Suppose ul ((x(miss)) is a velocity field regularized by the RKHS norm under the
following conditions: 1). The kernel function satisfies: lim K ( (
(miss)
X X
2). The density r(X r(x(mmis)) is bounded. Then, the velocity field that minimizes the cost functional FNRR E (mm)) [log ((x (miss))
X [ahs))] (x((is((((ss))) can be given by:
AV
x (wis) K ( (miss)
x {miiss)
u( (mmiss)) E (oxi)))
x (mer)) log p(X (miss)
X (abs))
{]
K(X (miss) x (miss)
(C.17)
where the expectation term E xxxx (ond) can be efficiently estimated using Monte Carlo approximation.
22

---

Proof. When the velocity is regularized by the RKHS norm, we can first reformulate Eq. (8) as
follows to find the steepest direction for the sake of improving FNEE:
{ (mm)) (omo)/l" (X (miss) X log p (msss) X (db))
(X (miss)
arg max us E
(C.18)
AV (mia) ul X (miss)
}]} [ (miss)) (
Based on this, assume we have a map function ((x), the kernel function can be given as follows:
((x,y) (0(x)) (()))
(C.19)
Hence, the regularization term that control magnitude of u ( can be given by || (miiis))) 11 and the spectral decomposition of kernel function can given as follows:
K(x,)) v) = (x))
(C.20)
where (x(-)) indicates the orthonormal basis and & the corresponding eigen- value, For any function u( x ( E H. the following decomposition is given:
(iisss)
ui EioEs (Xmmms))
(C.21)
where u, and E ||u,|12
DC.
The learning objective defined in Eq. (8) can be reformulated as follows: u² (x (iiiss)
( [u T (X (miss) V x (min) log pi( X (miss)
X (obs))
arg max al X (iiis)eee E
AV x (miu)
ui A (miss)
| (misss)) |1
||u|
2
(C.22)
log p(x (miss) ((bss) T "p (X (mis)))
E X
&
X (mis))
arg max a(X (((i)))))
DC
AA (miss)
u, V E,,, (X (mnss))
Σ 1|
x
Take the right-hand-side of (i) with-respect-to ut and set it to
get:
V EEEE (x (mm))) [V (mis) log p( (miss))
X (obe)))
(xx (miss))
AV
x (min) di X (miss))
}}} -, 0.
(C.23)
On this basis, us can be given as follows:
" V Ei x( (nin) [[V x (mis) log p( X (miss)
[ db (x((is)))
AV (unss Gi (x
)
(miss)
}}}
(C.24)
and hence, ul
X ((nnsss)) be given follows:
(x (miss))
V (e (miss)) (mmiss))
(C.25)
-vV
x (mo) K ((X (miss)
x (miss))
=E -(x (mla)
V (ma) log p(x (miss)
X (obs)) (ubs)))
K(X (miss)
x (miss)
Proposition (4.2). Assume that the proposal distribution r r(x is factorized by r (foitt) =
(mise)
(mise) )p X (obs)). The cost functional associated with the joint distribution is defined as follows:
Fjoint-NEs NER == Ex [log pi X (joint) X ((foin))) )}
(C.26)
which leads to the velocity field delineated in Eq. (9) and establishes FJRiRINNER as a lower bound for FNER. with the difference being constant (i.e., Fjoln-NNER FNER const, const > 0).
23

---

can Before proving this proposition, we want to first clarify the justification of the assumption that
r(X (ivini) = r(x (
)r())))( (x (ahu) = r r(x (miss) (( (( In this part, we set r(X ((xx(obb)) = p(X (1ht))
Before stating the justification, we should come to the following agreements:
1. Throughout the imputation procedure, X (obs) remains invariant regardless of any modifications to
X (miss)
2. Given this invariance, it is justified to state that r(X (obs) remains constant from the per- spective of particle variational inference represented by reference [35, 34], and consequently ((X((abs) (abs)
X (miss)) = r_x ( reflecting the independence of X (obs)
( from X (miss).
Based this, want show that within the WGF framework, the factorizations r r(x (X((iss))) Ir(X (x(ob)) and r(X (miss)) X ((ub))) (r((X(bbr) are equivalent. To this end, let write down the evolu- tion of r(XX(ohy XX (miss)) along time T as follows when r(x((1s)) X ( is factorized by
r( ((X((bbs)) X (mmis))) r(X(mms)) ( r X (
Or( (( (
ar( X (mmiss)
X (
r X (obb))
(X ((bbs) Ör (r (miss))
X (obs))
Ör (r(X((ab))
+r(x (miss))
X (ibbs)
r(X X(lis)) (( xe yr( (niss))
((X(
where the first underbrace is the Bayesian formula, and the second underbrace is based on the abovementioned Agreement 2. Consequently, can further obtain the following results:
Òr (abs) X (miss)
7 r(X (wh))
(r(X (obs)
X (miss))
r( (miss)
θT
(X (ohs))
OT
(C.27)
r(Xisbil X =i (X(it)))
Ör( X (
=r =x(X((bs)
Similarly, when factorize r [ (obs) x (mmis))) by r(X (ohs) (miss))
r(x (miss) (miss))) (
get the following result:
Ör (abs) (miss))
Ör (((
X (mixs))
(C.28)
x (ob() Òr X (miss)
Comparing Eq. (C.28) to Eq. (C.27), we can demonstrate our justification of the factorization ((x)) X (miss)) = r( ((x (miss) )r( (X (0bs)) Finally, we would like to conclude with a metaphor to further illustrate the plausibility of this mean-filed factorization, which has been widely applied in variational inference [5]:
1. Consider an actor in play, capable of being molded and shaped. Initially, the actor may not fully embody the role, akin to r( ((x (mmmss))) not containing information about (obs)
However just as a director shapes an actor's performance through guidance and rehearsal, all we need to do is ensure that r(x (miss)) is appropriately molded by the directorial guidance (mirrors the continuity equation Ør
- (ur)) of the velocity field u and the script provided by the (mmis))).
0r critic p( X (obs). X (miss) (/miss))pp((
3. As long as r can adapt based on this feedback (akin to the WGF framework), it can overcome the limitations of its initial portrayal (akin to r r(x = r(x (miss) r (x(o)))) )
Based on the abovementioned analysis, we can now start the proof of Proposition 4.2:
Proof. Our proof will be divided into two parts namely 'velocity field derivation' and 'upper bound acquirement'.
24

---

Velocity Field Derivation:
the following continuity equation should be satisfied during the optimization of r( (((miss)
(miss)
Ōr X (miss))
V x (niu))
[r( X (miss)
)ul X (mmsss))]
Ör X (miss))
x p( (0(x(1nng
X (mins)) [r
(miss) )u( (miss)
(miss)))]
x p(x
(C.29)
ÜT
Ör( X (joint)
(r( X (joint) )u( {
((nia)
where (i) is based on the fact that (ohs) remains unchanged during the imputation process. Thus, according to Eq. (C.16), the evolution of Jeiin-NEER along time T, da can be given as follows based on the chain rule:
dr
dF Joint-NER
dr
Ör X (joint)
log p( x (joint)
+ A log r( (X (loint) + 2 dx (sint))
Ör
(r( (join))
)u( X (jonat)
}} [log pi X (boin))
+ A log r( X (jolat) Ajd (soin))
((nis)
(x(x (joine)
(u( (aaint) (
}
x (min)) [log p (joint) + Aloo r(x (joint) + X) Njd. (joint) (join))
(r(x (ioine)
)u( X [
|
x (mis) [log p X (join)) + Aloo rX (jvint) ]] (joins)
((
/ [u( X (ooin))]
[r( x (ointt)
log p( (joint)
+ Ar (joint)
x (min) log (joint)
]d (jiin)
(uiss)
(u{ (riint))
(r( x (oint)
(uiss)
log p( X (joint),
AV
X (miss)
X (jinn))
[d (joint)
(
r( (ilmn)
fu (x (oint) (iiss) log O X (joint)
(mis)) u X
Jd Jd (joint)
(xxmm)) (x (joint) (ma)) log p X (juint)
(ma)) u X (jaint)
(C.30)
where (i) and (ii) are based on integration by parts. More specifically, when condition
lim x poo [u{ (X (ioint)) (x ((iint)) is satisfied, for example, (X(ii))) bounded, and the limit
of the velocity field the norm X (jaint) approaches infinity is zero x u( (X (rinti) 0),
we can get the following result [35, 32]:
/ x Xiniso
[r(X ((oxi))
)u( X (
Id I 0,
where we omit the gradient operator with respect to the observed variables (abs) denoted as V wim,,
because X (obs) remains constant during the imputation process. This constancy implies that the (jain)))(( yu( X [oini))] 0. Consequently, the left-hand-side of this equation can be further decomposed as follows based the integration by parts:
divergence V xiiss [r(x (jaint)
f u (Xx (ioint) Xmma
(miii (ionti) )u( (oint)) ) (ioin)
X (joint) x (jinnt)
+
V Xinia ul X (((iin)))]r
[)r('' (jaint) d (join))
25

---

Similar to the proof of proposition 4.1, we can restrict the velocity field in RKHS and find the steepest gradient boosting direction as follows according Eqs. (C.19) (C.21):
w" (x ((oin))
(2emmn (anny/u [u (X (joint)
x (min) log pi{ X (Goin)
arg max u(X ((Xx))))))
(ais)) u X ((miss)))
}}}
2 |u(( x ((oiun) )
(C.31)
log pi( X (ooum) I "m (X (oint)
(I r(x Goint)
(mio))
arg max M/ (0.X)
ui Eidi( (ooint)
AV (miss)
}1} 2 i=1
|uul133
Take the right-hand-side of (i) with-respect-to ue, and set can get:
V B x (X (0m))
V (maa) log p(x )1 "o Givin))
(join)
AV
X (min) Oi
(X
Guin))
22 0. (C.32) |}
On this basis, of can be given as follows:
(ioint) TT Φ5 (ioin))
x (min) dee X (
" = VErBB x (
[[V
(niss) log pi
AV
[]}
(C.33)
and hence, u(x (Giit)) can be given as follows:
(oiinn Goint)
VE Øs (xx (ivint)
(C.34)
X (mina) K(X (Joint)
x (jint))
(xx(xmn)) (wie))
(mbg) log p(X (joint
K K(X (miss)
X (miss)
Lower Bound Acquirement:
Before starting the proving this part, we should notice that given the unchanged observational
data (obs), the distribution p(X ((ob))) constant. On this basis, consider the definition of FNEr (right-hand-side of Eq. (7)), the first term and the second term are denoted by 'term 1' and 'term 2 for simplicity:
E F (iis)) log P X (miss) x (obs) +1 x
- [il[ (x (misss)
(C.35)
tem
tern
26

---

For term 1, we can obtain the following derivation:
/ r( (miss)) log p( X (miss)
x (obs) )d X (miss)
2 / r( ((xss))))) log (X (miss)
(miss))
(
x (obs) ) X (miss))
pl ((Xiobb)) log pl ( ) (obs)
negative entropy (acgative constant)
= 1ʃ p(x x (obs)
(obb)))((X )r(X (misss) log p( (miss) X (ahs) (dhs) d (miss) dx d (1hs)
/ pl ((ooss) log log (((bs)) (( ((
negative constant
(C.36)
/| p( (((() (r( (misss) log (
(obb))
(miss) X (ahs) d (miss)
dx (uhs)
+ /f r(x (miss)
(miss)) )p( (abs))
log X (obs) (obss)
cL (miss)
dx (obs)
negative constant
p( X (obs)
rr (X (miss)
log p( X (miss)
X (obs))
+ log p( X ((b6)) jod))) dx (miss) d. (obs)
r(x (min)) x (in))
log p(
(mmm) (mis; x (101))
= (mio)) x (cbit) [log p X (miss)
X (ub))) ]-
Similarly, the term 2 can be reformulated as follows:
H X [mis)))] }
Hx (miss))
/ p x (obs))
(obs)) log (X (obs)) (obs)
negative enirupy (negative constant)
= =Iʃ pl (((( (ott)) x (misss) log r( (miss) x (miss))
dx (ubs)
+ /f p(x (ubs) )r( x (min)) log p( X (abs) d (miss))
(C.37)
dx (obs)
negative eniropy (negalive constani)
p( X (obs)) )r (X (miss)
[log ro (miss) (miss))) ) r(X (dt), X (niss)
log p( X (abs) dx (miss)
dxx
((X(
X ((mi)))
--[[(r(X (obs)
x (miss)) ]..
Combine Eqs. (C.36) and (C.37), we obtain the following relationship:
FNER const Fjoint-NER:
(C.38)
and constant const is greater than 0.
Corollary (4.3). The following equation holds: ul (oiii)) uu(x((mu (mix))))))
Proof. This corollary can be easily proven by according to Eq. (C.38);
FNERR
Fjoin-NNER + const
5.NNE
8J FF Joint--NER NER + const
X
5r 6r(X (miss)
X miiss)
5r (miss)
(C.39)
8FNER
6 Joint-NER
(mba)
8r(X (minss)
X imbe)
Sr x (misss)
27

---

Plugging Eq. (C.39) into Eqs. (A.5) and (A.6), we can see that the velocity fields for X (miss) within functional FNEr and Joi are identical.
Appendix D Detailed Explanation for the Workflow of NewImp Approach
In this section, intend to provide detailed information about the implementation of the NewImp approach in Algorithm We will focus on two primary aspects: 1) the numerical implementation of ODE simulation, and 2) the DSM algorithm.
D.1 Forward Euler's Method for ODE Simulation
During step 7 of Algorithm I, we involve the simulation of the ODE defined by Eqs. (9) and (14). To simulate this ODE we use the forward Euler's 's method [6] in this paper for simplicity. Specifically, suppose we have the following ODE:
dæ
dr = f(+)),
(D.1)
and the initial value at = 0 is given Zinit the value at time ny can be derived as follows:
In x0 + [ f(x))dr.
(D.2)
To alleviate the intergal term, the forward Fuler's method attempts to approximate the integral term to summation term as follows:
~ so + f(x)) x (1 0).
(D.3)
On this basis, the value at time T can be obtained by repeating Eq. (D.3) from T = to 7 = T, which is the forward Euler's method.
Algorithm 2 Algorithm for Forward Euler's Method
1: Input: ODE f(x;); start point O; end point T; step size ry; initial value øg.
2: Output: Predieted value 2r
T.
3: j + T-0
b Calculate the Number of Steps
4: for T = 0 0 + 2n,..., 0 jn do
5: Er +- -r- (x X11
6: end for
D.2 Detailed Information for DSM
During step Algorithm we involve the training of omm log p ((iin)) using the function. In this subsection, we aim to further elaborate on the detailed algorithm for the DSM function to uphold the completeness of this manuscript. As mentioned in Section 2.2, the score function
iiii log p( x (oint)) is typically parameterized by a neural network. For simplicity, we denote the parameter set of Xo00 log p( X ((uin) by 0.
28

---

Algorithm 3 DSM for V Xxom log p (Join))
Training
1: Input: joint data X (jiinn))
2: Hyperparameters: neural network learning rate Ir, training epoch &, and neural network hidden unit sorre score-
3: for to & do
4:
X
(pinn)
X (joint)
+ E N(0,001)
Data Noising
& joun log lo (nin)) X(voin)
X (ooin))
X (oin))
5:
6: LDSM Eq. (15)
7:
0 < ApplyGradient V (Ve.dennn
Apply the Gradient with Learning Rate In
8: end for
Appendix E Detailed Information for Experiments
E.1
Background & Simulation of Missing Data
Table E.1:
Detailed dataset descriptions, where 'Dimension' denotes the variate number of each dataset. 'Numer' denotes the total number of item.
Abbreviation Dataset Name
Numer (N) Dimension (T)
BT
Blood Transfusion
748
4
BCD
Breast Cancer Diagnostic
569
30
CC
Concrete Compression
1030
CBV
Connectionist Bench Vowel
990
10
IS
Tonosphere
351
34
PK
Parkinsons
195
23
QB
QSAR Biodegradation
1055
41
WQW
Wine Quality White
4898
11
In this paper, we consider the datasets listed in Table E.1 our experimental datasets. Based on this, according to reference [44], missing data can be classified into three categories: Missing at Random (MAR), where the likelihood of missing data depends solely on observed data; Missing Completely at Random (MCAR), where the absence of data is completely unrelated to any observed unobserved variables; and Missing Not at Random (MNAR), where missingness is influenced by unobserved data. In the cases of MCAR and MAR, the patterns of missing data are considered 'ignorable" because it is unnecessary to explicitly model the distribution of the missing values. Conversely, MNAR scenarios, where missing data can introduce significant biases that are not easily corrected without imposing domain- -pnaii- assumptions, constraints, or parametric forms on the missingness mechanism, present more complex challenges [40, 22). Therefore, discussion is primarily focused numerical tabular data within the MCAR and MAR contexts.
To simulate missing data, we adopt the methodologies outlined in reference [22]:
MAR: Initially, random subset of features is selected to remain non-missing. The masking of the remaining features conducted using logistic model, which employs the non-missing features as predictors. This model is parameterized with randomly selected weights, and the bias is adjusted to achieve the desired missingness rate.
For each data point, the masking variable is generated from a Bernoulli distribution with predetermined fixed mean, ensuring that the probability of missingness is the same across all data points.
MCAR:
Although MNAR scenarios are not the primary focus of this manuscript, we include experiments in this context. Missingness is introduced either by additional masking of the MAR- selected features using Bernoulli process with fixed mean, or through direct self-masking of values using interval-censoring techniques. In this paper, we mainly consider the former strategy. In other words, the mechanism of MNAR we used in this paper is identical to the previously described MAR mechanism, but the inputs of the logistic model are then masked by MCAR mechanism.
MNAR:
29

---

E.2
Hyperparameter Setting of Baseline Models
In this subsection, we want to report the baseline models' hyperparameter settings ensure the reproducibility of our paper:
Batch-Size-Related The batch size for ReMasker set to 64. For other baseline models, is uniformly set 512. (Notably, for Sink and TDM, N 512, the batch size to 21
Boaddn-Unni--ellatee
The MIWAE model features latent dimension of 16 and 32 hidden units.
The TDM model includes 16 hidden units per layer with the number of layers set to 2.
MIRACLE's hidden units set to 32.
For ReMasker, the embedding dimension is 32, depth is 6, mask ratio is 0.5, encoder depth is
6, decoder depth is 4, number of heads is 4, and the multi-layer perceptron ratio is 4.0.
For MissDiff and CSDL_T, the channel size is set 16, the embedding dimension is set 128, and the layer number is set as 2.
For the GAIN model, for both the generator and the discriminator, the hidden size is set to x D, and the number of hidden layers to
Setiuiiion-yyeeppaametteeessss 50 for MissDiff and CSDLT.
The diffusion step is set 100 and the particle number
30

---

Appendix F
Additional Empirical Evidence
F.1
Toy Case Experiments
X
X1
X1 (b) Student's-t
X1 (c) Gaussian Mixture
X1
(a) Standard Gaussian
(d) Skewed Gaussian
Figure F.l: Contours of Various Distributions' Density Value.
Table F.1: Newlmp Performance with Missing Rate at 30%, and 1000 samples generated.
Scenario Distribution Type MAE
WASS
Gaussian
0.769
10.00
Student's s-f
0.737
0.513 10,048
MAR
Gaussian Mixture
0.763
0.419
Skewed-Gaussian
0.422
0.492
Gaussian
0.769
2..44
Student'se s-(
0.698
0.307 10.014
MCAR
Gaussian Mixture
0.824
0.301
Skewed- Gaussian
0.417
0.210
Gaussian
0.778
0.309 100000
Student's- s-t
0.715
0.323 +0,19
MNAR
Gaussian Mixture 0.807 200042
...00
Ekewed-Gaussial
0.421 20.11 0.202
To demonstrate the effectiivnness of the Newlmp method vary different type of distributions, evaluate it across four distinct toy cases, each characterized by different distributions:
Standard Gaussian: (ddel)) ~ NO, T2x2).
Student's-f (a heavy-tailed distribution): X lidcal) ~ St-t(O,
[1
0.5
0.5
0.5 0
0.5
0.1
Gaussian Mixture: X (ideal) 1/2 X 2),
+
([-11,
}
0.5
0.1
0.5
0.3 0 [V([2,-2
0.3
Skewed Gaussian (via exponential transformation): (ideal) exp(c), where N I2x2).
Based on this, we display the contours of their density values in Fig. F.I, and we list the imputation accuracy comparisons for MAR, MCAR, and MNAR scenarios with 30% missing rate in Table F.I. The results indicate that our NewImp approach generally performs better on nta-standard Gaussian type data, underscoring its universality and upplicability. This enhanced performance is attributable to our modeling strategy, which involves modeling the score function of the data [52, 50], which eliminates the need for normalization, and consequently results in the NewImp approach can perform well on complex data distributions, including skewed, heavy-tailed, and mixture distributions.
F.2 Additional Experimental Results with MNAR Scenario
In this section, we expand upon the results presented in Table by including the MNAR scenario, detailed in Table F.2. Additionally, report on the outcomes of ablation study and sensi-
31

---

tivity analysis in Tables F.4 and F.5 and Fig. F.2. These extended results lead to several pertinent observations:
Across three different missing data scenarios, the models consistently exhibit the poorest perfor- mance under the MNAR condition. For instance, in the MNAR scenario, nearly all models show a significant decrease in imputation accuracy and an increase in standard deviation. This supports the assertion made in Appendix E. that addressing the MNAR scenario requires the incorporation of relevant domain knowledge to mitigate biases introduced by the pattern of missing data.
The findings from the ablation study under the MNAR scenario are consistent with those observed in both MAR and MCAR scenarios in Section 5.3. This consistency underscores the importance of including the NER term and adopting the joint distribution modeling approach.
Similarly, the results from the sensitivity analysis under the MNAR scenario align with those from MAR and MCAR scenarios in Section 5.4. This alignment reinforces our interpretations of model performance across different groups of hyperparameters under MAR and MCAR scenarios.
Table F2: Performance of MAE and WASS metrics at 30% missing rate.
Scenario Model
BT
BCD
CC
CBV
IS
PK
QR
WOW
MAB
WASS
MAE
WASS
MAE
WASS
MAE
WASS
MAE
WASS
MMAE
WASS
MMAE
WASS
MAE
WASS
CSDLT
0.51
18:20
282
16.86
15.86
4.13
MissDiif
0.85
2.20
0.91
16.53
0.87
1.59
0.33
387
0.72
13.25
0.92
17.07
0.63
26.25
0.75
6.88
GAIN
0.75
0.65
0.54
1.61
0.75
0.67
0.68
0.6
0.56
138
0.59
1.90
0.65
5.05
0.68
0.87
MTRACLE
0.62
0.38
0.55
1.92
0.43
0.25
0.55
0.46
3.9
35.06"
4.14
34,07
0.46
2.8
0.51
0.56
MAR
MIWAF
0.64
0.52
0.64
5.05
0.75
4.71
1.04
TDM
0.83
0.89
0.83
3.47
0.81
0.73
0.76
0.83
0.62
1.96
0.86
3.36
0.59
4.46
0.73
0.90
ReMasker
0.52
0.52
0.48
1,15
0.60
0.43
0.40
0.37
0.2
2:23
0.61
1.50
0.60
3.81
0.51
0.59
Newlmup
0.52
0,38
0.34
11.82
0.35
0.25
0.31
0.20
0.39
1.31
0.44
1.21
0.45
3.50
0.46
0.55
CSD1 T
0.73
1.93
0.73
15.51
0.85
2.71
0.83
3.79
0.766
15.19
10.22
12.42
1.55
19.89
0.78
4,11
MissDirT
0.72
1.62
0.73
14.30
0.84
1.23
0.82
3.31
0.75
13.01
0.71
14.12
0.56
19.67
0.76
4,95
GAIN
0.72
1.14
MIRACLE
0.52
0.115
0.35
2.9
52.92
42.78
0.35
2.71
0.56
0.75
MCAR
MIWAE
0.58
0.24
0.50
2.55
0.76
0.09
0.33
124
0.64
4.95
0.51
2.05
0.48
5.87
0.67
0.95
Sink
0.73
0.65
0.75
4.39
0.84
0.85
0.82
127
0.75
4.94
0.74
3.36
0.61
5.92
0.76
1.25
TDM
0.68
0.42
0.63
3.57
0.77
0.75
0.77
1.15
0.66
4.20
0.64
2.89
0,52
5.34
0,74
1,20
RoMasker
401
1.71
NewImp
0.45
0.42
0.44
0.32
1.01
3.66
0.76
CSDLT
0.83
0.82"
15.68
0.33
333
0.74
15.54
0.84
12.20
0.62
19.77
4,09
MissiDiif
0.78
14.89
3,53
13.31
16.02
21.62
GAIN
0.77
0.57
0.62
3.94
0.78
0.79
0.78
1.15
0.71
4.85
0
4,20
0.76
10.53
0.75
1.25
MIRACLE
0.63
0.35
0.60
4,26
0.52
0.35
0.63
0.77
3
55.56
3.49
44.76
0.52
5.61
0.58
0.80
MNAR
MTWAE
0.66
0.42
0.56
3.31
0.74
0.63
0.85
130
0.39
4.33
0.60
3.06
0.53
7.21
0.67
0.97
Sink
7.23
1,33
0.76
0.77
0.79
0.64
0.76
4.54
6.45
1,25
ReMasker
0.53
0.28
0.42
1.91
0.54
0.39
0.59
0.6%
0.51
3.40
0.63
3.06
0.47
5.02
0.56
0.77
NowImp
0.60
0.35
0.32
1,46
0.44
0.34
0.46
0.52
0.40
2.68
0.39
1.56
0.42
5:57
0.55
0.81
Kindly Note: The best results are bolded and the second best results are underliend.
"*" marks the results
that NewImp significantly outperform with p-value- 0.05 over paired samples I-test.
32

---

Table F.3: Standard deviation of MAE and WASS metrics at 30% missing rate.
Scenadi
Model
BT
PK
WQW
MAR
WASS
MAF
WASS
MAR
WASS
MAR
WASS
MAA
WASS
MAR
WASS
MAR
WASS
MAF
WASS
USBI 1
4.80-2
2
5 3.91/22
2,033
2:70/2
1-48-1
2.00-2
9.26-22
2.1023
2,100
199-2
3.1131
2,00/2
5.30-1
1.80-2
7/76/2
MisaDoir
4.0E:2
4.9E-1
3.1E-2
2.400
3.4E-2
2.6E-1
19E-2
1.200
5.4E-2
5.6E-1
3.0E-2
2.400
1.9E.2
4.550
1,6E-2
1.3E
GAIN
1.0E
1.62
4.3E-2
2.3E-1
3.4E-2
8.9E22
19E
4.0E-3
5.9E-2
3.4E.1
5.4E22
3.7E-1
6.9E
5.483
3.9
5.8E22
MIRACUE
3,06-2
4.36-1
1.15-2
2.16-2
6.79-1
4-8-1
SAFR
1/49-1
1-56-3
2.8F-1
MAR
MIWAE
5.26-2
2.56-1
6.1822
1.28-1
4/6-2
5.38-2
2-662
3.85-2
2.65-1
196-3
2.76-2
Sink
4.683
2.00
2-0
5.56
2.00:
1
TDMM
1.210-1
8.36--2
3.40-2
201-2
6.21/2
2-70-2
1-20-1
2111-2
ReMasker
1.11--1
3.0E-2
2.3E-2
2.28-2
2
2.70-2
L1E-1
2.26-2
1.96.2
1.18-1
1.9E22
1,68
3,3022
CSDLT
1.06-2
1.5E-1
8.7E.3
5.76.1
8.7E-3
2.28-2
4.E-3
4.6E-2
4.6E.3
3.6E-1
1.162
8.7E.1
3.7E.3
3.15.1
1.28-2
5.0E.2
MissDoft
0.435-5
3-30-1
8:20-3
8. 8.36-1
3.50-3
2.50 2
598--3
N-41-1
7.1023
1.81--1
4.00-3
2
0.20-3
2,400
4.10-3
6,38-1
DAIN
6-18-2
1101-
7.96-3
2 2--2
2.48-2
3-46-2
1:4022
1 101-1
3.712
2,00-2
MIRACLE
2.68.
5.4E.3 3
166 2
19E-1
1.722
1.382
5.26-3
1.4E 2
4.78:2
12250
4.6.2
1.100
1.0E2
1.7E- 1
1.133
5.5E.3
MCAR
MEWAE
3.10
3.4E
4.9
4.5
7.6
1.6 2
4.36 2
9/42
1,2
106
7
9.1E 3
2.78
4.113
3.5-33
Sink
2-56-1
4.5--1
4.76-3
1.45-1
106-2
3-1E-3
396-3
1-26-3
TDM
8.6--
3.1
0.5-3
6-6-4
6-0--
796-3
126-3
1,06-3
3.5-3
9.36-3
51B-3
1.35-2
1
2000-2
0.20-2
2
1.26-2
3:16-2
9.1623
3886-2
4
7-00-
1:70-2
Newtmp
3,30-5
3.70-3
4,00-3
1
1
5,0093
4,80-3
2.20-3
USDUT
2,903
2.21
8.76
7.86
2.36
1,58
7-46-3
1:00/2
5
2:26-2
4.68
2:40.
2:5
MisaDaY
3.76-2
3.7E-1
2.4E-3
9.76-1
3.0E-
2.4E-1
S.3E-1
8.2E-1
1.42-2
3.5E-1
1.0622
2.150
8.7E.3
3,350
4.05-3
5.6.11
GAIN
4.5E-2
1.2E-1
6.2E-2
6.E-11
5.3.-2
K66-2
411-2
9.3E-2
5.5E.3
4.5
2.5-2
4.7E
5.0E22
1.250
4.1E-2
1.0.1
MIRACLE
0.68-2
9-50-2
1996-2
4.70-1
1-10-2
1 18-2
600-3
1766-2
9:3822
3,500
6.96-2
1.70-2
1.70-1
7-5--
126-2
MNAAR
MEWAR
3.98-2
6-48-2
8 3-9--
5-76-2
2.48-2
S6-2
306-2
8.66-
6.68-1
7. 1.28-2
336-2
1.2F22
15511
91 911-1
205-
Sizk
1
6.28 2
1.4 2
1.4
1.E
5,3 3
136. 2
396 2
7.3E.
186.2
3.5
6,EE 3
1.6 1
460
2
TEMM
2.281
0.NE 2
1.4 2
1.26
9.4
3.38 5
136 2
336
2.00:
7.70
186
37T
39E 3
188
115:3
2
RuMasker
2.26-2
1/56-)
1.45-1
1.78-2
7-5-3
9.96-1
146-3
9.58-2
1.78-1
3.5E-2
4.06-1
106-3
-36-1
Newllmp
1.08-1
3.9E-3
1.36.1
1.9822
8.8-3
9.00.
1.38.1
8.56.3
7.1E-
5.6-3
166-0
Table F.4: Ablation study results at 30% missing rate.
Sccaario NER Joint
BT
BCD
CC
CBV
IS
PK
QH
WOW
MAE
WASS
MAB
WASS]
MAE
WASS
MMAE
WASS
MAA
WASS
MAE
WASS
MAE
WASS
|MAE
WASS
0.96
3.82"
1,05
20.2"
1.04"
5.47"
1086"
5.81"
0.67
20.2"
1,06"
15.6"
10,2"
22.5"
1.79"
6.49
MAR
0.1""
0.40
0.58"
0.47"
0.43*
1,34
0.46"
1.25"
0.42"
3.56"
0.55*
0.64"
0.96
3.83*
1.05*
20.3*
1.01"
5.19
0.86"
5.83*
0.67*
20.2"
1.06
15.6"
0.72"
22.5"
0,79
6.51"
0.52
0.35
0.25
0.31
0.39
1.31
3.50
0.46
0.72"
2.11"
0.774
16.7
0.85"
3.72"
1083"
5.22"
0.74"
18.4"
0.71"
12.7
1..88
20.1"
0.766
5.57"
MCAR
XXX
0,17
0.25
0.79
0.62"
0.46"
0.61"
0.46
3,05
0.34
0.36"
3.74"
0.58"
0.02"
0,72'
2.12'
0.73
16.8
0.86*
3.73"
0.83
5.24*
0.74'
18.4'
0.71
12.7'
0.58
20.1"
0.76*
5.60"
0.47
0.34
0.42
0.44
3,05
0.34
3,66
0.53
X
0.81
2.47"
0.89*
18.2"
0.87"
3.5"
1.85"
5.26"
0.69
17.6"
0,87
13.0"
10.44
20.6"
10.77"
5.21"
MIAAR
1.47
0.11"
0.47"
0.64"
0.44
2.79
0.43"
1.88"
0.44
0.60"
D.87"
0.82
2.57
0.89*
18.3
0.87
3.86"
0.85
5.78
0.59
17.7*
0.88*
13.5
0.64
30.7"
0.71*
5.73*
0.44
0.34
0.40
2.68
0.42
5.57
0.55
0.81
Kindly Note: The best results are bolded and the second best results are underliend. "*" marks the
results that NewImp significantly outperform with p-value< 0.05 over paired samples f-test.
Table F.5: Standard deviation of Ablation Study Results with missing rate at 30%.
Sccourto
QB
WgW
MAF
WASS
MAE
WASS
WASS
MAE
WASS
MAE
WASS
WASS
4.5E
6.9E
1
5
4.6E
4.5E.1
4.4E22
3,450
1.4E:
3.9E31
7.2E
2.480
5.0
6.9E:2
1.2E-1
2.7E-1
IJE-1
2.9E-3
4.7E-2
2.5E-1
3.9E.2
2.7E-2
1.3E-]
1.0-2
9.0E-2
1.56-2
2.6E-1
2.46:1
3.6E-2
X
6.19-2
4.18-1
4.89-3
6.8P-1
1.1F-1
5.6F-
4.6P-1
4.5F-1
4.4F-1
3,FF))
1.48-1
3.9F0)
7.3F-1
2.4F3
5.05-3
4.1F-1
2
4,00
2
1
5
6,40
1.622
2.20:2
1.962
1.002
3
1
2.75
1/6/2
3
X
9.08.3
5.68-1
1.06:3
2,38-1
3.68.3
6.08-1
3,36.3
3,06-3
9.98-1
4.3F-1
1.56-3
2.35-1
1.16-0
1,000
4.06-3
1.4523
MCAR
X
5,00033
5
2,8033
4.0002
1.2023
1.50/2
9.4133
1.7152
0.3153
1,26-1
1 1.0022
1,215-
7.36-3
1.96-1
9.36-4
3.96-3
X
1.00:2
2.3E
3.2E.3
2.1E-2
1.5E22
1,0E0
4.0E3
1.4E22
3.38.3
3.78.3
1.96.3
4.6E-1
1.1533
1.88-2
4.16.3
1.8E.3
5.78.1
1.16.1
6.46.3
3.7E-3
4.76.3
1.7E-1
2.36.3
1.16.2
4.28.
1.5E 1
2.38
3.5E:
3.20
1.3E
1.2E
5.30-2
6.903
1.2E
3.26
9.00
1.702
8,36
1
1
MNAR
X
4.0E:2
1.4E-1
3.4E.3
1.3E.1
1.8:22
2.1E22
4.8E.3
1.7E-2
1.0E-3
13E.1
IJE.2
1.7E.1
8.00-3
7.0.11
7.78.3
136-2
1.4F-1
2.4F-1
1.39-1
1.7F-1
198-1
8.1F-1
1/46-1
2.58:
1 1
3
1
1.9033
2
9
1.20/2
9.003
1.36
8.50/3
5,00/2
7.263
6,86
57663
1.76/2
33

---

0.50
0.6
0
WASS MAE
0.8
0.1
WASS
MAE
0.3
0.6
WASS
MAE
WASS
MAE
0.2
0.35
0.1
0.25 0.5
1.0 2.0 5.0 10.0
32
64
128
256
512
0.1
0.25
0.5 1.0
2.0 5.0 10.0
32
64
128
256
512
h
HU score
h
HU score
00:
WASS
MAE
0.4
0.40
0.3
0.35
WASS
- MAE
0.35
WASS
.. MAE
WASS
MAE
0.2
1.0 2.0
5.0
10.0 15.0
20.0
0.005
0.05
0.1
0.2
0.5
1.0
2.0
5.0 10.0 15.0
20.0
0.005
0.05
0.1
0.2
0.5
2
(a) MAR with 30% missing rate at CC dataset.
(b) MCAR with 30% missing rate at CC dataset.
0.40
WASS
MAE
0.35
MAE
0.4
0.30
0.1 0.25
1.0 2.0 10.0
64
128
256
512
h
HU HUsrore score
0.40
0.5
WASS
MAE
0.35
WASS
MAR
0.3
0.30
1.0
2.0
5.0
10.0 15.0
20.0
0.005
0.05
0.1
0.2
0.5
77
(c) MNAR with 30% missing rate at CC dataset.
Figure F.2: Parameter sensitivity of New on bandwidth for kernel function (h), hidden unit of score network HUscore NER weight A, and discretization step 7 for Eq. (9) on CC dataset. Mean values and one standard deviation from mean are represented by scatters and shaded area, respectively.
F33 Empirical Evidence for Selecting RBF Function
In our derivation process, we specifically selected the RBF kernel to satisfy the 'zero boundary condi- tion': lim x K ((x (jonnt) X (joinn) for the sake of avoiding the explicit density estimation of the intractable proposal distribution r( (oiit)))) This selection prompts an additional inquiry: What if we replaced the RBF kernel with another that does not fulfill the 'zero boundary condition"? To maintain the rigor of our analysis, we compared the performance of the Newlmp with alternative kernel functions under identical settings. Consequently, we consider the following types of kernel functions:
linear kernel function (linear): K(X (joint) (join))
(uintt)
[x (oiit))
(joint) T
polynomial kernel function (poly): (xx (joint)
(joinx)
{[X (ooin)) (oint))
T T
sigmoid kernel function (sigmoid): K (x (jiitt)
Goint
= tanh { [X (joint)
(oint)
TT
cosine similarity kernel function (cos): K ((x((oiin))
x (Goint)
{ [[[ Dimlil
[X Uoing
){
[ Goitt
T
sine kernel function (sin): K(X (joint)
(oint)
sin
X (joint)
(joint)
The experimental results are detailed in Table F.6 (For completeness, we also report the results under the MNAR scenario), From the results, it is evident that other kemee functions, which do not meet the 'zero boundary condition', perform significantly worse compared to the RBF kernel. This demonstrates the critical importance of selecting the appropriate kemnel function for achieving accurate imputation results, thereby validating the choice of the RBF kernel for our NewImp approach.
34

---

Table Fó: Imputation accuracy vary different kernels at 30% missing rate.
Scenario Kemel
BT
BCD
CC
CBV
MAE
WASS
MAB
WASS
MAB
WASS
MAE
WASS
MAE
WASS
MAE
WASS
MAE
WASS
MALE
WASS
lincar
0.77
0.73
0.883
5.36
0.85
0.77
0.83
0.97
0.77
2.42
0.50
2.50
0.70
4.58
0.76
1.02
poly
1.12
1,40
1.44
8.36
1.06
1.36
1.08
1.55
1.00
5.5
1.33
5.53
1.28
14
1.02
1.76
MAR
sigmoid
1,02
1.44
0.82
0.74
0.81
0.93
1.30
9.57
7.19
1.04
0.53
0.80
3.14
0.73
0.92
2.37
2.30
4.56
0.98
sin
8.62
8.74
291
15
283
18
191
11
456
10
307
8:00
314
15
357
RBF
0.52
0.38
0.34
0.82
0.35
0.25
0.31
0.20
0.39
1.31
0.44
1.21
0.45
3.50
0.46
0.55
lineur
0.71
0.45
0.87
5.87
0.83
0.84
0.81
1.28
0.8L
5.65
0.83
4.14
0.62
6.31
0.76
1.25
poly
0.94
0.58
1.31
12
0.98
1.21
0.99
1.85
1.11
9.94
1.27
8.64
1.11
19
0.92
1.95
MCAR
sigmoid
0.74
0.48
0.97
8.00
0.81
0.82
0.50
1.23
0.92
7.36
0.94
6.54
0.76
7.6
0.77
1.78
0.43
0.54"
5.51
0.81
5.48
3.06
6.11
1.30
12
RBF
0.48
0.18
0:25
0.50
0.47
0.34
0.42
0.44
0.44
3,5
0.32
1.01
0.34
366
0.53
0.76
linear
0.50
6.36
0.58
1.32
5.36
4.62
7.26
00.66
1.29
poly
0.92
0.87
1.26
11
0.98
1,26
1.01
1,90
1.09
9.55
1.19
7.71
1.14
0.92
2.09
MNAR
sigmoid
0.77
0.59
1.01
8.86
0.50
0.85
0.82
1.27
0.91
7.71
0.99
7.38
0.83
9.27
0.77
1.31
0.73
0.55
0.88
6.19
0.81
0.84
0.82
1.27
0.70
5.36
0.57
4.77
0.68
7.17
0.75
1.34
362
500
12.
358
562
300
0.35
1.46
1.56
5.57
0.55
Kindly Note: The best results are bolded and the second best results are underliend. marks the
results that RBF kernel significantly outperform with p-value< 0.05 over paired samples t-test.
Table F.7: Standard deviation of MAE and WASS metrics vary different kemness at 30% missing rate.
Scenurio
Kemel
BT
BCD
WOW WASS
MAR
WASS
MAF
WASS
MAR
WASS
MAF
WASS
MAR
WASS
MAR
WASS
MAF
WASS
MAF
linex
1.5E-1
2.4E-1
4.5E:2
4.3E-1
2.1E-2
3 2
2
5.0:2
3.4E-2
1.7E.1
2.88-2
3.26-1
3.5E.2
2.6E-1
2.1E-2
3.7E-2
poly
3.9
7.8E-2
8.5E.3
2.4E:1
3.E-2
7.58 2
1.6
4.6E
5.6E 2
5.0E 1
5.26-2 2
6.18-1
6.8B2
5.7E-1
4.7E-2
6.6E2
MAR
4.1--2
1.3--1
2.6E-1
411--2
5.5E-2
5.1E-1
1.40
17663
166-2
6.6--
1.16-2
2.16-2
2.06-2
5.4--3
4.8-2
9-56-2
3.76-2
9.080
1-96.1
1-36-1
1-36-2
4.150
2.36-1
2.18-2
1.96-2
8.78-2
1.7E-1
2.0-2
144-2
4.1-22
6.2-22
2
5.9E1
5.68-1
4.7E1
2860
1.082
2,580
1,422
2.5E-1
2,0E1
2.65-1
1.111
2.06-1
4,2500
1.431
5.8E1
RBF
2,0E-2
4.0E-2
2.68-2
1.1E-1
5.6-2
6.48-2
1.5222
2.28-2
1.9E-2
L.1E-1
1.1E-2
RMB--2
1.9E-2
2.7E-1
1.6E-2
3.3E-2
3.16-2
2.38-1
7.9R-3
4,6833
2.16-2
1.26-2
3.36-1
8.26-3
6.4-3
poly
3.26-2
9.06-3
2.45-1
8.26-3
2.4--2
8.58-3
3.6--2
2.26-2
4.35.1
1.6-2
2.7-11
1.18-2
9.26-3
MCAR
signsid
21122
288
3.3E-2
3.58.3
166--1
8.4-3
7.7--
1.18-2
5.3..3
4.8E.3
2.36-2
2.1022
A.1E-3
1.4E-2
2.00:
L.T.1
2:1822
1.2622
7.06-1
7.78-3
1.5E-1
6.7E-3
1.8E-2
9.1
RBF
1.96-3
5.76-3
6.46-3
1.15-2
3.0
1.1E-1
3.00
4.4E
3.56-1
1.28
4.7E-
4.2633
1.3E
14E
1.48
266
2
2.00
1.56
6.80
5.3E
5.1E
8.4E
MNAR
4.36-2
3-6-2
1-4--1
1.3E-1
2.96-2
1.8-2
3.4E-
2.55-1
3366-1
268-2
5.6-2
6.8-2
8.06-
1.ES
14E3
16E2
2.56.2
2.56.2
6.8.1
3.45.1
306-3
4.36-2
36B-1
1. 1766-
3.08-3
1.46-2
5.48-1
4.55-1
576-3
4.6.3
1.1--2
1.1-22
1.00)
2.2E1
2.58-1
1.8E1
1.400
6.781
3.5E.1
5.0E1
5 3.6E11
1.78-1
18E1
2.88-1
3.3E1
1.251
7.1E1
RBF
255.2
1.0E.1
3.9E-3
1.38.1
196-2
2.
9.4E33
12E-2
9.0E-3
1.38.1
8.5E.3
5.1B-2
7,28-3
6.3E-1
577..
1.7E.2
E4 Time Complexity Analysis
In this section, we present an analysis of the complexity of time for our New approach. The complexity analysis is based the algorithms described in Algorithm 1. We begin by estimating the time complexity of the score function V x log p X (fjimn)). Assuming the number of layers in the neural network that parameterize V isim log X (inim)) is L and each layer has equal number of hidden units denoted as HUscrre the time complexity for the imputation algorithm defined in Algorithm| 1 is detailed as follows:
1. DSK Training Part (step 5): Building on the previous item, the time complexity for the DSM training algorithm defined in Algorithm 3 is given as:
O 4 x x (D x HU + (L-1) HU2 H
(F.1)
where the factor of 4 comprises three distinct components: backward propagation (1), forward propagation (1), and the acquisition of the sample-wise score function (2). Note that the network parameter size is substantially smaller than the number of data points, thereby making the forward computation of the score function the primary factor in time complexity.
2. Imputation Part (step 7):
Score function computation: The time complexity for computing the score function is ex- pressed as:
O O x N x (D x HUsore + (L- 1) x HU2 HU
(F.2)
where the factor accounts for the backward propagation needed during the score function computation.
35

---

(scconds)
Estimate Impute Total
(seconde)
Estimate Impute Total
Time (scconds)
Estimate Impute Total
225
2.58
2355
3.00
325
3,9
375
3000
3225
3.00
A75
1225
logio N
log N
log N
(a) Running time vary MAR. (b) Running time vary N, MCAR.(c) Running time vary N, MNAR.
Time (scconds)
Estimate Impute Tutal
Time (seconds)
Estimate Imputte
Time (scconds)
10'
Estimate
Impute
Total
Total
0.2
1.1
1.5
1.4
1.
log10 D
log10
logio
(d) Running time vary MAR. (e) Running time vary D. MCAR. Running time vary D. MNAR.
Figure F3: Average computation time, where 'Estimate' indicates the 'DSM Training Algorithm' (step 5 of Algorithm 1), and 'Impute' indicates the imputation algorithm (step 7 of Algorithm 1). The scatters and shaded areas indicate the mean and one standard deviation from the mean, respectively.
Kernel function and its gradient: Employing the RBF kemel K(X, X) exp (-1x
|X X
2h
the gradient with respect to X is analytically determined as:
Vx nr3
[K(X, X
[
X
[.J]] O Σ K(X,
j=1
(F.)
The time complexities for calculating the kernel function and its gradient are specified in Eqs. (F.4) and (F.5):
O [N2 x D+N3], O N N
(F.4)
(F.5)
Based on the abovementioned analysis, we explore how computational complexity varies with different dataset sizes N and the number of features D, as shown in Figs. F,3 (a) and (b), respectively. From these figures, it evident that computational time increases with the dataset size N. However, changes in the number of features D do not significantly affect the computation time. This observation underscores that the primary determinant of computational complexity in our context is the dataset size, aligning with our theoretical analysis, which indicates quadratic relationship between time complexity and the size of the dataset N for the 'Imputation' part, and >> D for the 'DSM Training' part, aligning with our theoretical analysis.
Moreover, the data reveals that the total computational time is predominantly governed by 'Estimation' part of our NewImp approach. This suggests that the training of the score function represents critical bottleneck in the efficiency of the Newlmp algorithm. Therefore, accelerating the NewImp algorithm crucially hinges on reducing the computational demands of the "Estimation" part.
F.5 Convergence Analysis
In this section, we want to discuss the convergence of the proposed NewImp approach, prior to delving into this discussion, it is essential to establish a clear definition of convergence:
Definition F.1. A sequence F2.., Fr) is said to be convergent if there exists real number G such that for any given positive number E > 0), there exists positive integer N, such that for all indices n greater than N, the corresponding terms Fn, n 2 N satisfy the inequality Fr, G < s.
Based on Definition F.1, if sequence is either monotonically increasing or monotonically decreasing and bounded (either bounded above or bounded below), then it is guaranteed to converge according
36

---

to the celebrated monotone convergence theorem (Section 3.14 in reference [45]). Based on this, the convergence of the 'Imputtioon part (step of Algorithm 1) and DSM training part (step of Algorithm 1) are proposed in the proceeding parts.
0.9
0.8
MAV
0.8
0.6
259
(a) MAE, BT.
(b) WASS, BT.
(c) MAE, BCD.
(d) WASS, BCD.
0.8
0.8
1.0
MAE
MAE
8.6
0.4
0.4
0.4
0.4
(c) MAE, CC.
(f) WASS, CC.
(g) MAE, CBV.
(h) WASS, CBV.
0.7
0.9
MAE.
0.6
2.0
MAE
0.8
0.5
0.6
0.4
(i) MAE, IS.
(j) WASS, IS.
(k) MAE, PK.
(I) WASS, PK.
0.7
WASS
0.7
WASS
0.66
0.8
0.6
(m) MAE, QB.
(n) WASS, QB.
(0) MAE, WQW.
(p) WASS, WQW.
Figure F.4: Evolution of evaluation metrics along iteration time 7 under MAR scenario at 30%
missing rate. The shaded area indicates the + 1.0 standard deviation uncertainty interval.
F.5.1 Convergence Analysis the Imputation Part
In this section, we explore the convergence of the imputation part as defined in step 7 of Algorithm within our NewImp approach. Based on this, we first prove the following proposition for the convergence the "Imputation' part:
Proposition F.1. The convergence of the imputation part can be guaranteed, given that the discretiza- tion step size yj is small enough.
37

---

0.7
0.5
MAA
MAE
0.6
0.5
250
500
500
250
500
250
500
MAE, BT.
WASS, BT.
MAE, BCD.
(d) WASS, BCD.
0.8
MAI
WASS
MAE
8.6
0.4
0.5
250
500
250
500
(c) MAE, CC.
(1) WASS, CC.
MAE, CBV.
(h) WASS, CBV.
5.0
MAE
MAE
WASS
3.5
0.4
250
500
240
(i) MAE, IS.
(j) WASS, IS.
(k) MAE, PK.
(I) WASS, PK.
116
0.78
MAE
WASS
WASS
0.5
0.60
0.55
250
500
250
500
250
(m) MAE, QB.
(n) WASS, QB.
(0) MAE, WQW.
(p) WASS, WQW.
Figure F55: Evolution of evaluation metries along iteration time under MCAR scenario 30%
missing rate. The shaded area indicates the + 1.0 standard deviation uncertainty interval.
Proof. First, let us reformulate the velocity field as follows:
u( XGoint)
x K( ((X(iiii x (joime)
EE,, -x
imis log pi X (jvint)
T K (xx (jiin))
x Goint)
A uimo log r Gaint)
I1 ((X (ioint)
((iiiii x Goint))
①
omso log Pi (joine)
)] K(XXXoiin (joint) Goinn))
(F-6
AA iniso log r X (juint)
/ r( Gointy
K(X (juitt)
x
(joinn)
d
(oint)
X mniso log p( Grvin,
AA fiimn log r( xx (loint)
/
K (oi
x (joint))
dr xjoino
+ V immo log D( X (Jolon
where (i) is based integration by parts.
38

---

Based on this reformulation, the inner product can be given as follows:
joint joint NER
dt
ò Joint-NER
uc A (ivint)
dr( (x (imiss))
X Xinis))
ör(X (inint))
V (nia] log X (joini)
K (joint)
X (foint)
(F.7)
inios log pi (
X v Ximioo log r( X (oint)
} dr( X (joinn)
dr x (join))
Xinio log p( X (vvin))
0,
where the (i) is predicated on the requirement that the kernel function, K(;;) :), is semi-positive definite. Consequently, according to the abovementioned derivation, we can conclude that the evolution of JERR is monotonic increasing along Furthermore, Fjoin--NER satisfies the following inequality:
F Joint-NNER
Joint-NER (A + 1)LB, (x Giinn) [log r(X (X (joia))]
(F.8)
=KDLL [r (x(x((nnn)) || (ioint))
<0,
which indicates that FJRInt-NER upper-bounded by 0.
According Eqs. (F.7) and (F.8), the cost functional NERR driven by the velocity field u( (ivin)) along converges. Building on this, employing smaller step size results in the iteration curve of JEiR more closely approximating the ODE defined in Eq. (F.7). Consequently, smaller 7) leads to sequence where JJiin-NER monotonically increases, aligning with the theoretical expectations of the ODE behavior.
Unfortunately, directly obtaining NjoRnt-NER is intractable. Nevertheless, can still observe the changes in WASS and MAE across iteration time t to demonstrate the convergence of the 'Impute' part. To this end, we present the convergence trends along in Figs. F.4 to F.6. These figures illustrate that both MAE and WASS generally decrease as the iteration epochs increase and eventually stabilize after 250. This observed behavior supports theoretical findings regarding the convergence of the 'Imputation' part.
39

---

MAN.
0.6
MAE
0.7
250
500
250
500
250
500
259
500
(a) MAE, BT.
(h) WASS,
(c) MAF, BCD.
(d) WASS, BCD.
MAR
0.8
0.7
WASS
MAR
0.8
0.7
12
1.0
0.6
0.8
0.5
0.4
250
500
250
500
250
500
250
500
(e) MAE, CC.
(1) WASS, CC.
(g) MAE, CBV.
(h) WASS, CBV.
4.5
MAE
MAE
3.5
0.5
250
500
250
500
250
500
259
500
(1) MAE, IS.
(j) WASS, IS.
(k) MAE, PK.
WASS. PK.
01.55
0.55
0.60
WASS
WASS
0.50
0.65
1.0
0.55
250
560
250
500
250
560
250
509
(m) MAE, QB.
(n) WASS, QB.
(0) MAE, WQW.
(p) WASS, WQW.
Figure F.6: Evolution of evaluation metrics along iteration time under MNAR scenario at 30% missing rate. The shaded area indicates the + 1.0 standard deviation uncertainty interval.
F.5.2 Convergence Analysis of the DSM Training
Similarly, we can also give the proposition of the DSM training algorithm located in step 5 of Algo- rithm and summarized in Algorithm 3:
Proposition F.2. The convergence of the DSM training algorithm can be guaranteed, given that the learning rate Ir is small enough.
Proof. In the beginning, let reformulate the parameter learning procedure of the DSM training algorithm as follows:
0 = 0, x V VoLDsM(o=0.>
(F.9
which can be further reformulated as follows:
0 ,1+ 0,
- V VoLDsMle=e.
br
0r+1 - 0-
lim
V VoLDMM
(F.10)
1r
Ir
d0
VeLDSM
dr
which indicates that the iterative procedure for LDSM is monotonic decreasing along Meanwhile, note that:
d.pssM
(
d0 dr
(F.11)
dr
Plugging Eq. (F.10) into Eq. (F.11), we can get the following result:
dLpsM = - (VoLusM VoLDsM) < 0,
(F.12)
dr
40

---

Finally, recall Eq. (15), we can know that the following condition holds:
DDMM 2 0.
(F.13)
Building on this, employing a smaller step size Ir results the iteration curve of LNSM more closely approximating the ODE defined in Eq. (F.11). Consequently, a smaller Ir leads to sequence where LDSM monotonically decreases, aligning with the theoretical expectations of the ODE behavior.
Based on this proposition, we plot the evolution of LUSM along time T in Fig. F.7. These figures illustrate that the LISM generally decreases as the iteration epochs increase. This observed behavior supports our theoretical findings regarding the convergence of the DSM training algorithm.
1500
EDsM
EDSM
CDSM
560
180-
1400
W/W
490
100
200
100
100
200
(a) LDSM. MAR, BT.
(b) DDMM MAR, BCD.
(c) COSM, MAR, CC.
(d) CDMM. MAR, CBV.
1700
2000
550-
EDSM
1100-
1900
544
1600
1850
1550
1000
1800
530
(e) LOSM. MAR, IS.
(1) LDSM. MAR, PK.
(g) COSM, MAR, QB.
(h) LOSM. MAR, WQW.
LDsM
1475-
345
COSM
500
100
1450-
340
490
1400
338
490
100
200
100
100
200
(i) LDSM. MCAR, BT.
COSM. MCAR, BCD.
(k) LOSM MCAR, CC.
()) CDSM. MCAR, CBV.
LDSM
1700
CDsM
1150
2050
2000
CDSM
550
545
1600
1650
1100
1900
1950
535
540
1850
100
200
100
200
100
200
108
200
(m) LOMMs MCAR,
(n) CDSM. MCAR, PK.
(0) LOSM MCAR, QB.
(p) COSM. MCAR, WQW.
1500
C0OM
CDSM
345
350
CDSM
500
190
490
180
480
100
200
160
100
200
109
200
(q) LUSM. MNAR, BT.
(r) LUSM, MNAR, BCD.
(s) LUSM. MNAR, CC.
(t) LUSM. MNAR, CBV.
Coss
1700
CosM
11500
2080
1650
1100
1980
5.00
535
1050
100
200
100
160
260
(u) LUSM. MNAR, IS.
(v) LUSM, MNAR, PK.
(w) LUSM. MNAR, QB. (x) LUSM, MNAR, WQW.
Figure F.7: Evolution of LOSM. the loss function of 'Estimation' part along iteration time at 30% missing rate. The shaded area indicates the + 1 1. standard deviation uncertainty interval. The results of LDSM are smoothed by exponential moving average with = 0.60.
41

---

F6 Downstream Task Comparison
To further substantiate the rigor of our manuscript and demonstrate the efficacy of the proposed NewImp approach, we conduct downstream task comparisons as detailed in this subsection. Initially, we evaluate the classification performance on imputed data using the following protocol: 1) Selection of datasets with non-binary labels. 2) Post-imputation, we train a support vector machine equipped with an RBF kernel and an automatic kernel coefficient. We assess the model's performance using 5-fold cross- validation, reporting both the mean and standard deviation of the accuracies across 10 runs with different random seeds. In this procedure, we select classification accuracy as our evaluation metric. 3) Additionally, we include the accuracy of ground-truth data for reference. The comparative results are presented in Table F.8. From Table F88, it can be seen that the NewImp approach generally has the best performance among all baseline models, this phenomenon reflects the superiority of the proposed NewImp approach in a further way.
Table F88: Classification accuracy results at 30% missing rate.
Scenario|
Model
BCD
CBV
IS
QB
WQW
CSDI T
0.6771 0.0691000ER
0,603 1.9-777
0.640"...000
0.441
21.30-1 177
MissDiff
0.83914.988-17 0.40612.778-7
0.90011700-11
0.783
0,490
1.7
GAIN
0.97141.23-17 0.51911888-1
0.94144998-17
0.794
0,512-6.778.1
MIRACLE
0.966
0.579
0.827
0.798
0,488
1.088-11
MAR
MIWAE
0.968 16.4--17
0.567
2200-177
0.939
0.858
0,482_
Sink
0.958
1- 0.0000
0.499
1: 1.39E-17
0.891
1-4 17
0.798
17
0.477
1.39E:17
TDM
0.969 0.969 17
0,581
5.55E-17
0.938
0.796
0.505
2 17
ReMasker
0.973
17778-17
0.517 + 2108-17
0.935 0.0000
0.859
+41661411
0,489
= 1.858117
NewImp
069-416.17E-17
0.595
17.11117
0.947
94998-17
0.860 13.70-17
0.611__..86.17 166-117
CSDI T
0.593 1.99-17
00.066
18
0.609 1.60991.9988 1.398-17
0.656
0.0000
0,441 = 1.398-17
MissDiff
0.756_..70F-17
0.327
- 1.70--17
0.883
0.732
7.40--77
0.478 1.
GAIN
0.964
0.489
+ 1088-117
0.929
2.47E-17
0.837 1.171-17
0.491 137001-17
MIRACLE
0.923 17.1-11
0.450
2247E-17
0.709
1.40E-17
0.770
0.00E0
0.474
17
MCAR
MIWAE
0.957 17.98-17
0.451
+ 1299--1
0.917 0.917 5.555-17
0.831
17
0.492 2.0088-17
Sink
0.950 20,0EE
0.446
2088-17
0.877 1694E-17
0.768
1.000D
0.462 222.2-2222220 2.08E-17
TDM
0.961 18
0.486 0.486 3.77
0.922 5.55EE-11
0.922
0.836
0.489
_ 1.
ReMasker
122 0.468
0.922
1.85E-17
0.762 1,390
0.479
17
NewImp
0.967 44.93-177
0.494
0.934
+4.98E-17
0.839
12.47E:17
0.495
12.08E-17
CSDLT
0.6587-41188:1
0..78 18208-18
0.608
000000
0.647 0.0000
0.440 1.440%6..94..0. 68
MissDiff
0.800 2.47012
0.332212.47E.
0.884
1 3.70-117
0.749
4.988-77
0.480 14480"1.8588 1.85--17
GAIN
16.7-17
0.475
: 1.5E-117
0.925
0.925
3,700E11
0.837
1717-17
0.493
MIRACLE
0.457]
0.457
0.721 0.721 3,70E-17
0.777
1.93E17
0.481
1.85
MNAR
MIWAE
0.961 144116-11
0.437
1-69E-18
2.78E-7
0.839
+ 17
0.495
+: 17
Sink
0.427 0.946__1.30F-17
0.427
16
0.88221
0.781
0.781121.9022 1.30-
0,449922.8FF11 27.78E117
0.469
TDM
0.96211-6.45-117
0.469 22005-177
0.927 2.7786117
0.927
0.773
0.489 0,489
ReMasker
0.965 12.78-177
0.458
13.7--1777
0.929
5.555111
0.771
17
0.478
Newimp
0.969 14493E--7
0.482
1:2477--1
0.943
14.93E-17
0,847
13.70E
0.497
Ground Truth
0.985 1777-11 (..700
0..989 : 0.0000
0.908
(0.566=2.78:11
Kindly Note: The best results are holded and the second best results are underliend. -qg
marks the results that Newlmp significantly outperform with p-value< 0.05 over paired samples t-test.
Moreover, we also conduct downstream regression task comparisons as detailed in this subsection. Initially, we evaluate the regression performance on imputed data using the following protocol: 1) Selection of datasets with continuous outcome variables. 2) After imputation, we train a support vector regression model equipped with an RBF kernel and an automatic kernel coefficient. We assess the model's 's performance using 5-fold cross- validation. In this procedure, we report both the mean and standard deviation of the mean squared errors (MSE) and mean absolute error (MAE) across 10 runs with different random seeds. 3) Additionally, we include the MSE and MAE on ground-truth data for reference. The comparative results are presented in Table F.9. As indicated in these results, the NewImp approach consistently outperforms most of the baseline models, further validating its superiority.
42

---

Table F.9: Comparison results on the regression task with 30% missing rate.
CC
Model
MAR
MCAR
MNAR
MAE
MSE
MAE
MSE
MMA
MSE
CSDLT
1.4181
20000
10.000
3.07E2 5.6002
1.41E1
0.0000
3.07E21:5.612
1.41EI 1
0.0000
3.07E21
MissDiff
1.25E1
1592F-16
2.31E2
1.532
1.27E1
5.928-16
2.44E2
4-1.228)) 8,822
1.27E1
1955-16
2.45E22 3,0022
GAIN
1.2381
17.80-116
2.24E2
3,6022
1.26E1
11955-16
2.40E22 2.40022
1.6811
36955-16
2.38E2411.982
MIRACLE
1.59E1 15.92-116
3.51
3.51E2
1.60E
15.92-116
3.71E2213.712
1.64E
17899-16
3.80E211 80E2
MIWAE
1.23B1:1222:
2.23E2
3,63E2
1.27E1 26,220116
2.43821
1.27E]
1.2.28-16
2.42E2.440002
Sink
1.88111222B.8
3,48E2
3,4822
1.65E
8888-116
3.85E21
3.8582
1.66E]
14.441-16
3.87E2458888
TDM
1.58E1 44446-16
2.19E22-5.0R3
1.26E1
16666-16
2.38E2
1.66E] 16.66F166
2.37 18622
ReMasker
1.58E1 22.22-166
3.44E2
1.4482
1.62E1
14.44F-16
3.69E2I 60F2
1.64E 18888-16
3.72E2117775
NewImp
1.22E1
11.88-15
2.37E2_3,9E3
1.24E1
13995--16
2.37E2
1.26E1
7.899-16
7.37E2+33982
Ground Truth 1.01EI 15.92-166
1.57E2_3.<22
|9.83E0
1.55)2
1.04E1
1.68E2
Kindly Note: The best results are bolded and the second hest results underliend. "*" marks the results that Newlmp significantly outperform with p-value< 0.05 over paired samples t-test.
E.7 Baseline Comparison Vary Different Missing Rates and Scenarios
In this section, we further present the extended analysis of model performance across varying missing data rates, as detailed in Tables G.1, G.3, G.5 and G.7, and the corresponding standard deviation error results are presented in Tables G.2, G.4, G.6 and G.8. From the comparison results, it can be seen that our NewImp approach generally perform well compared most of baseline models. This phenomenon reflects that the proposed NewImp approach is robust to various missing rates, and further proves its applicability.
Appendix G Limitations & Future Directions and Broader Impact
G.1 Limitations & Future Directions
The limitations and future research directions of this work can be summarized as follows:
Utilization of Kernel Function:
During the derivation of the velocity field, we employ RKHS ensure implementation easiness. However, this regularization term may impose restrictions on the velocity field's direction, potentially limiting imputation accuracy in high-dimensional settings. Additionally, the computational complexity tends to scale quadratically with dataset size increases. Exploring alternative regularization terms [13] to replace RKHS presents a promising direction.
Training of Score Function: As discussed in Section F.4, the runtime of NewImp is predominantly governed by the DSM function. Investigating techniques to reduce the training costs of this part, such as employing sliced score matching [49], represents an intriguing area for future exploration.
The WGF framework currently operates system where each sample is equally weighted. A critical advancement would be the incorporation of second-order systems, such Hamiltonian dynamies [55, 61], and other gradient flows like Fisher-Rao gradient flow [69] that assign variable weights to samples. These adaptations aim to decrease computational times inherently.
Wasserstein Gradient Flow Framework:
first-order
RD Support Assumption: In our manuscript, for ease of derivation, we assume that the distribution we model has support on the real number domain IR,,, which limits the direct application of New tabular data with categorical variables. This limitation can be alleviated by employing the mirror descent approach. Specifically, for a categorical variable with D categories, the distribution belongs the Dirichlet distribution whose support lies on the simplex A AD-T On this basis, we can define the Bregman function as the entropy function: ((() = Σ (X, log X, X,) and apply mirror descent using this Bregman function to handle the categorical domain effectively. Notably, similar approaches have been successfully applied in works focusing on constrained domain sampling, as exemplified in [48]. We have implemented a comparable scheme in Section 3.1 and Appendix B.
43

---

G.2 Broader Impact Statement
MDI and DMs are pivotal areas within machine learning, each boasting a wide array of real-world applications. While numerous applications exist, this paper does not single out any specific ones; instead, it focuses on addressing fundamental challenges in these fields. This study significantly advances the application of DMs for MDI by tackling prevalent issues such as inaccurate imputation and challenging training processes. We believe that the insights garnered here can be applied to related domains, such as probabilistic time-series forecasting and image inpainting, where accuracy is often more critical than diversity in results. A common challenge across these domains the nuanced need for precision over variety, which can lead to overlooked opportunities in model application and development. Our proposed method provides a fresh perspective on these tasks through an optimization lens. It evaluates the appropriateness of directly applying existing diffusion models to these tasks and, where necessary, proposes the derivation of novel algorithms. This approach not only enhances the understanding of the underlying mechanisms but also paves the way for more targeted and effective solutions in the future.
Table G.I: Performance of MAE and WASS metrics at 10% missing rate.
Scemario Model
BT
BCD
CC
CBV
IS
PK
QR
WOW
MAR
WASS
MAE
WASS
MAE
WASS
MAE
WASS]
MAA
WASS
|MAE
WASS
|MAE
WASS
MAE
WASS
CSDLT
1.22
6,09
1.04
23.73
0.93
3.43
0.33
4.06
0.70
16.73
1.36
23.27
0.26
24.16"
0.83
5.06
MissDiif
1.08
3.53
1.02
22.26
0.92
1.55
0.84
5.74
0.70
12.43
1.25
21.05*
0.74
28.80
0.82
7.58
GAIN
0.97
1,11
0.51
0.87
0.84
0.64
0.71
0.50
0.3
0.56
0.66
1.31
0.55
1.69
0.70
0.63
MIRACLE
0.65
01.2
0.31
0.14
0.50
0.26
337
1938
4.46
23.51
0.45
1.23
0.35
MAR
MIWAE
0.74
0.31
0.33
0.7
0.76
1.78
2.84
0.56
Sink
1.09
1.50
1,00
2.53
0.92
0.67
0.84
0.67
0.71
1.03
1.25
3.52
0.75
2.69
0.82
0.81
TDM
1.02
1.49
0.80
2.22
0.78
0.54
0.73
0.54
0.57
0.78
1.14
3.26
0.68
2.61
0.77
0.75
RoMasker
0.61
1,00
0.644
1.04
1.00
0.85
0.47
0.23
0.65
1.01
0.77
1.20
0.76
2.31
0,51
0.41
New
0.40
0.17
0.31
0.43
0.23
0.11
0.32
0.07
0.33
0.50
0.46
0.74
3.30
0.37
0,29
CSDLT T
0.71
1.99
0.72
16.40
0.56
2.58
0.81
3.69
0.75
16.01
0.733
12.20
0,56
20.77
0,78
4.02
MissDiir
0.70
14.57
0.86
1.31
0.74
13.37
0.72
17.05
19.35
5.44
GAIN
0.66
0.33
0.36
0.76
0.47
0.60
0.54
0.53
1.14
0.38
0.40
0.39
1.06
0.73
0.61
MIRACLE
0.44
0.10
0.36
0.43
0.28
0.36
0.53
0.33
3.68
26.51
3.78
18.22
0.29
0.66
0.47
0.35
MCAR
MIWAE
0.49
0.115
0.46
0.33
0.71
0.43
0.79
0.68
0.00
1.81
0.30
0.74
0.45
L811
0.65
0.31
Sink
0.70
01.50
0.75
1.50
0.83
0.53
0.81
0.72
0.75
1.69
0.75
1.24
0.59
1.94
0.76 4 0.66
TDM
0.62
0.24
0.43
0.59
0.99
1,60
0.61
ReMasker
0.39
0.11
0,46
0.68
1.04
0.33
0.48
0.22
0.70
1.74
0.55
0.80
0.69
2.94
0.55
0.39
NewImp
0.38
0.09
0.19
0.18
0:27
0.11
0.23
0.10
0.42
1.01
0.25
0.28
0.30
L.06
0.40
0.30
CSDLT
0.95
0.88
17.57
0.86
2.93
0.53
3.97
0.75
16.50
1.01
13.26
21,44
0.79
4.26
MiscDief
01.91
2.47
0.88%
16.20
0.35
1.39
0.4
3.70
0.75
13.39
0.99
19.32
0.66
23.29
0.77
5.95
GAIN
0
0.61
0.42
0.20
0.79
0.55
0.09
0.54
0.1
1088
0.51
0.92
0.47
L.99
0.75
0.70
MIRACLE
0.55
0.21
0.45
1.29
0.37
0.16
0.53
0.34
3.82
25.83
1.19
23.82
0.39
1,26"
0.49
0.40
MNAR
MTWAE
0.65
2.94
0.60
Sink
0.87
0.76
0.88
2.48
0.66
0.2
082
0.5
073
0.99
2.70
0.68
3.29
0.78
0.75
TDM
0.80
0.71
0.75
2.11
0.74
0.50
0.73
0.63
0.9
1.4
086
2.39
0.59
2.63
0.75
0.71
RcMasker
0.49
0.30
0.53
0.98
1.02"
0.3
0.4
0.2
0.66
161
0.67
1:25
0.64
2:72
0.56
0.48
Nop
0.45
0.17
0.24
0,35
0.26
0.12
0.24
0.10
0.39
0.94
0.37
0.61
0.43
2,27
0.42
0,37
Kindly Note: The best results are bolded and the second best results are underliend.
"9g marks the results
that NewImp significantly outperform with p-value< 0.05 over paired samples I-test.
Table G.2: Standard deviation of MAE and WASS metries at 10% missing rate.
Sccauio Model
IT
BCD
CC
CRV
PX
QR
WGW
MAE
WIS55
MAE
WX555
WASS
MAE
WA55
1MAE
WASS
MAE
MAE
WXS5
MMAE
WISSS
CSDLT
S.408-2
4.705-1
5338-2
4.0360
2.21B-2
2
2.236-1
1966-1
2.78-2
143ED
5.85-1
3.5 3.5920
4,30E-2
3.316-1
2.676-2
1188-1
Miscii
4340-3
3340-1
5786-3
47680
1
3785-
3716-1
1.900
4678-3
1098-1
3180-3
33500
3
7,5000
7.876-3
|5500
MIRACTE:
6.550-1
175-1
196-2
1380-1
1
1.45-22
198-
198-1
1350
1940-
9050
1.085-2
7.110-2
7.196-1
1855-2
MXR
MIWAT
8.770-3
195-
1338-1
4.70
196--
5088-3
6.0003
7.848-1
31190-1
Smn
5.200-2
200-1
1410-1
166--
3470-3
4.00-3
999-
1100-2
1.548-1
31908-3
199-3
ReMakkr
2.30E
2
1678-1
LA3E-
1
2.546:1
194E-2
8 8758-2
161E-1
1.19-1
3.01
5.R0E 2
1.KE:
3.168-2
3.4E-2
New
5.636-2
5.58-22
2258-2
9033-22
2368-2
2
9:448-3
4.758-3
128-2
2.5 2
4.058 2
1.44
2
5.746-1
1.706-2
1,3K6-2
Masihil
4.88--2
7-805-1
6-100-3
10460
2.008-2
1
2.116-2
4.2000
K412-5
3.1
5.665-1
138-1
GAIN
4.300-2
02200-2
K-18-5
5.136-2
L-10-2
1 192225
5988-2
2.03622
191022
3.RE-S
5.00B-2
2.276-2
2805-2
MIRACLE
31178-2
2.138-2
1.516-2
5.16-2
3 2400
1448-1
3-41-22
3985-1
6.00E-2
33716-3
1.345-2
SIR
3140--
1.478-1
636B-
81 1155
7.370-2
5.770-3
TDM
2007-7
178-3
2118
138--3
1.418-2
5710-2
5.601-1
6900-3
Newflop
7.878-3
166--
1966-5
2.400-1
1900-
4.5003
8
1.530-1
1.015-1
6900-1
OSDUUT T
1.478-3
4.54E.1
1716-2
2,2760
3.5IE-2
1.746-
1238-2
1 1.5ED
1.418-2
2
1.3E
4.22E)]
1,436-0
2
420
GAIN
7.008-2
2015-1
1506-2
6.2662
534E
117E5
1.365
2.5EE
308E-2
MIRACCEES
51270-2
2800-2
4-106-2
1--10-1
8 KINESS
13800
1901-11
6.140-1
2.00-3
MNAA
NIWALE
02310-2
3.600-1
1-310-2
4 483000
151h-1
128-22
3
1-101-1
448-2
8.208-1
1.28-2
6316-1
1:688-1
2011/2
1300L-1
9.365-1
1688-2
2.418-2
6880-22
6320-2
1-788-1
2.118-1
3
2 2400-3
7-28-22
1411-1
9.602-2
199E-2
3
3:300-1
118-2
NewSmmp
3,640-2
7/786-2
4186-3
2
9
1.10-1
1.188-
5646-2
1096-2
9.11522
L-08-2
3.340-1
9.428-3
44

---

Table G.3: Performance of MAE and WASS metrics at 20% missing rate.
Scmmriio Model
BT
BCD
CC
CBV
IS
PK
QU
WOW
MAB
WASS
MAE
WASS
MAR
WASS
MAE
WASS|
MAA
WASS
|MAE
WASS
MAA
WASS
MAR
WASS
CSDLT
0.99
4,07
0,98
19.60
0.50
2.97
0.56
0.60
16.70
1.07
13.54
0.70
20.67
0.78
4.36 *
MissDiif
0.91
2.19
0.96
18.37
0.87
1.37
0.7
571
0.70
13.36
1.05
17.50
0.69
24.45
0.78
6.64
GAIN
1.02"
1.11
0.53
1.35
0.74
0.58
0.71
0.63
0.8
1.0
0.55
1.02
0.59
3.02
0.75
0.82
MIRACLE
0.69
(1.6
0.47
1.14
0.38
0.18
0.54
0.37
3.90
30.49
3.99
23.48
0.45
1.91
0.51
0.48
MAR
MIWAE
0.68
0.61
0.56
1.51
0.08
0.52
0.82
139
0.59
3.81
0.65
0.67
0.93
0.88
0.72
1.04
3.21
3.49
0.89
TDM
0.88
1,06
0.87
3.07
0.77
0.61
0.78
0.74
0.59
1311
0.95
2.95
0.63
3.32
0.75
0.84
ReMasker
0.51
0.42
0.52
1.16
Tun
Tan
0.50
0.32
0.60
1.49
0.65
39
0.76
3.98
0.55
0.51
NewImp
0.55
0.53
0.31
0.68
0.28
0.16
0.27
0.13
0.35
0.83
0.42
0.78
0.51
2.91
0.45
0.46
CSDLT
0.74
2,055
0.73
16.83
0.85
2.6
0.82
3.73
0.76
15.03
0.74
1188
0.57
20.29
0.78
4.05
MisaDief
0.75
1,64
0.74
14.08
0.84
1:25
0.82
3.71
0.74
13.09
0.73
16.22
0.56
22.27
0.76
5,46
GAIN
0.70
0.31
0.36
0.96
0.60
0.09
0.72
0.52
2.23
0.39
0.26
2.13
0.82
MIRACLE
0.48
44.75
0.53
MCAR
MTWAE
0.53
0.18
0.48
1868
0.73
0.35
0.1
0.93
0.2
3.18
0.48
1.26
0.47
3.92
0.65
0.73
Sink
0.73
0.40
0.75
2,90
0.84
0.68
0.82
0.97
0.75
3.50
0.73
2.14
0.60
3.81
0.76
0.94
TDM
0.67
0.33
0.62
2.57
0.74
0.56
0.75
0.84
0.64
2.73
0.61
1.72
0.50
3.31
0.75
0.88
RcMasker
0.42
0.50
0.41
4.09
0.55
NowImp
0.22
0.50
CSDLT
0.85
0.86
17.08
0.84
0.83
3.84
0.74
15.92
0..90
11.70
0.64
20.01
0.78
4,10
MissiDier
0
15.69
1.18
0.73
13.54
16.08
21.83
5.69"
GAIN
0
0.48
0.43
1,46
0.72
0.57
0.68
0.74
0.56
2.40
0.55
2.05
0.51
4.08
0.71
0.87"
MIRACLEE
F
0.54
0.19
01.5*
2.02
0.44
0.23
0.57
0.54
1.35
42.40
385
34.51
0.43
2.89
0.55
0.57
MNAR
MIWAE
0.600
01.31
0.53
2,08
0.72
0.56
0.2
1.00
0.57
281
0.63
2.32
0.54
5.52
0.65
0.76
Sink
0.77
0.55
0.09
0.74
3.19
3.98
5.39
0.99
357
4.95
0.94
ReMasker
0.49
0.27
0.48
1.56
man
0.52
0.46
0.55
2.52
0.59
1884
0.58
4.49
0.56
0.62
NowImp
0.48
0.17
0.27
0.76
0.36
0.21
0.33
0.25
0.39
1.79
0.39
1.17
0.43
4.03
0.48
0.54
Kindly Note: The best results are bolded and the second best results are underliend.
"*"
marks the results
that Newlmp significantly outperform with v-value< 0.05 over paired samples I-test.
Table G.4: Standard deviation of MAE and WASS metrics at 20% missing rate.
Model
MAFF
W/S55
MAR
WX55
MAP
WASS
MAF:
WASS
MAA
WASS
MAB
WA.55
MAF.
WASS
MAR
WASS
2,6800
MisDiif
1166-1
7.426-5
3.4850
1.44--
1
1.3162
10260
1246-2
3.32E
3,1130
7.57E
5.5560
3.245-2
8135-1
GAINN
31100-2
4200--
5330-2
2.300-
1940-2
A.48.2
2.7122
6.10-2
4338-3
1.2861
2.88--3
33901-2
5.111-2
3280--1
6.3002
110022
MIRACTEE
2.40--2
1780-1
2320-2
6-20-22
1800-3
1.8822
1.488-1
2.801-2
3008-1
6.300
3310-3
3330
2.10-2
2.500-1
4.90-2
Sin
1.208-3
9.10--
1280-1
3.8882
7,38222
1
1-216-1
3426-3
6.40-22
5:300-2
2
1.00222
3.288-1
3356-2
6.28-22
IDM
1.500-2
1118-1
33906-
7.34222
2-266-
2 2888-2
5996-2
1-31b-2
2. 211061
2.20--2
3.440-1
3.240-2
HcMasker
0-218--
9.02-22
1-410-
9-228-2
1
1-08-1
2-400-1
K
4.90b-2
1-205-1
1.542-1
1:311:0
4
1128-2
CSDUT
4.548-2
5.01E-2
1.00-2
13380
2
1
5.366-3
300E2
8515-3
7.25--1
L.5720
4.54E-
507T
2.54B-3
20E
L756-2
1.46
2,5739
3.KE
7.276-3
GAIN
5676-2
7316-
1
2
2000:
9 20
305E
1.71
125-22
MIRACLE
2.358-2
2.345-2
1336-2
4068-2
158E
121E5
13800
458
7.44E
2
MCAR
MIWKE
2.766-2
1333-22
1.10-22
1:45:
S18E
16483
2
2
5226-3
7
2
TOM
8-408-2
28875-2
6370-5
19868
LD0E22
1300-2
HoMolker
3:200-2
1-10022
1-20-1
1070-
1836-1
1000-1
1 LI9--
1,3010
S190-1
1.50-22
Newlmm
3-258-2
2,408-2
1/48--5
1.510-3
1000
1966
11360
4.720-2
5.60-3
28
1-50-1
43006-1
1.20022
CSDI T
8-208-3
2-30-2
1,8900
998-1
1
1.005
1155
15500
(805)
70000-1
MicCTH
5.800-3
1.330
10001
1 LS0E
2 1002
2 2
1.30-1
1.0000
13411-1
GAIN
3.200-3
3310-3
2 1102
8 8330-2
3.411-1
1
3,002
2AIE-1
5-415-1
1
30VAR
MIWAE
L.4E-2
3.278-2
9.106-3
4256-2
2
1366
25155
1
5.7E
35 3
3.EE
4.35E:
6-2223
2535-2
1006-2
0.838-2
207E-2
1A1E-1
111
2:66:
STNE:
1 19030
1
3.TE
305E-1
7046-3
2358-2
TOM
1
2-00-1
1516
2.806-1
5786-3
1.56-2
1
Now
2,04E-2
1.70E-2
3
1956-2
3
1
1
1
5 55K5.3
6.68 3
1.47E.3
2
6.5EE
431E.3
1
45

---

Table G.5: Performance of MAE and WASS metrics at 40% missing rate.
Scemriio Model
BT
BCD
CC
CBV
IS
PK
QU
WOW
MAB
WASS
MAE
WASS
MAR
WASS
MAE
WASS|
MAA
WASS
|MAE
WASS
MAA
WASS
MAR
WASS
CSDLT
0.92
2,80
0.80
17.47
0.84
2.75
0.52
382
0.70
16.04"
10.99
13.29
10.62
20.06
0.77
4.05 +
MissDiif
0.83
0.87
15.08
0.84
1.37
0.3
4.06
0.70
13.50
0.86
18.23
0.61
23.33
0.75
6,41
GAIN
0.86
0.87
0.56
2.28
0.78
0.76
0.74
0.4
0.52
2.14
0.57
2.46
0.66
6.41
0.72
1.17
MIRACLE
0.665
[1:43
0.72
7.16
0.47
0.32
0.57
0.58
4.02"
61.45
3.48
32.16
0.49
4.35
0.55
0.60
MAR
MIWAE
0.09
0.59
2.10
0.78
0.34
0.53
2.58
2.11
0.52
5.29
0.85
0.94
3.13
3.97
5.60
1.25
TDM
0.84
0.93
0.82
4.31
0.77
0.73
0.78
1.05
0.64
2.57
0.81
3.61
0.58
5.37
0.73
1.19
ReMasker
0.41
0.22
0.43
1.40
0.01
1.05
0.51
0.47
0.51
2.23
0.58
1.96
0.55
3.89
0.52
0.66
NewImp
0.71
0.79
0.38
1.32
0.43
0.35
0.40
0.37
0.40
1.67
0.46
1.68
0.43
1.08
0.50
0.70
CSDLT
0.71
1.76
0.73
15.43
0.85
2.77
0.82
3.91
0.76
14.66
0.72
11.68
0.57
19.81
0.78
4.15
MisaDief
0.77
1.22
0.73
13.89
0.84
1.29
0.81
3.00
0.75
13.14
0.72
12.65
0.56
19.50
0.76
3.89
GAIN
0.74
0.69
0.46
2.61
0.79
0.50
0.79
1.45
0,74
7.30
0.52
2.49
0.63
11.12
0.77
1.52
MIRACLE
0.56
2.74
59.66
51.98
4.28
MCAR
MTWAE
0.62
0.32
0.51
3.50
0.79
0.84
0.84
1.33
0.66
6.68
0.53
2.67
0,50
7,86
0.68
1,17
Sink
0.74
0.57
0.75
5.78
0.84
1.04
0.82
1.00
0.76
6.59
0.74
4.33
0.61
7.62
0.76
1.59
TDM
0.70
0.552
0.65
4,90
0.79
0.93
0.78
1.48
0.67
5.71
0.65
3.74
0.53
7.06
0.74
1.50
RcMasker
0.52
0.17
1.0
0.64
5.64
4.88
0,97
NowImp
0.54
0.45
CSDLT
0.79
0.81
16.41
0.86
2.81
0.83
3,89
0.75
14.67
0081
11.79
0.60
19.51
0.79
4,17
MissiDier
0.75
0.79
14.93
0.84
0.33
3:35
0.74
13.45
0.78
14.07
20.56
4.29
GAIN
0.76
0..2
0.85
7.92
0.36
1.11
0.96
2.16
1.15
13.95
0.84
6.52
1.20
31,45
1.01
2.89
MIRACLEE
F
0.68
0.41
0.77
12.61
0.6
0.53
0.68
1.05
2.79
58.80
1.24
51.35
0.63
12.68
0.61
1.05
MNAR
MIWAE
0.67
0.399
0.58
4,47
0.79
0.38
0.34
157
0.64
6.37
0.59
3.22
0.54
9.12
0.69
1.21
Sink
0.77
0.67
165
0.76
6.46
587
0.63
8.78
0,77
1,64
0.74
0.79
154
5.33
8.31
1.56
ReMasker
0.59
0.32
0.46
3.22
1,00
166
0.66
0.2
0.56
5:53
0.61
4.04
0.87
6.89
0.59
0.98
NowImp
0.64
0.46
0.36
2,34
0.58
0.59
0.56
0.83
0.44
3.92
0.43
2.31
0.44
7.16
0.60
1,10
Kindly Note: The best results are bolded and the second best results are underliend.
"*"
marks the results
that Newlmp significantly outperform with v-value< 0.05 over paired samples I-test.
Table G.6: Standard deviation of MAE and WASS metrics at 40% missing rate.
Model
WXOW
MAFF
WXSS5
MAP
WX55
MAP
WASS
MAF:
WASS
MAA
WASS
MAB
W9.55
MAP:
WAS5
MAR
WAVS
7.588-1
3,608
MisDiif
2.506-2
2066-2
2.1550
2.1EE
1.35.1
1.336-2
13360
235E-2
4.3050
1
4.7560
1.73E-2
1.3-50
GAINN
8.00-1
2.10-2
2-30--
4330-2
100511
1080-1
4328-3
2430-1
5:300-3
7. 2.182-1
6.00522
9-420-1
3.570-2
1.40--1
MIRACTEE
5088-2
3.00:22
8300-2
6.200
1960-3
118822
1.89822
3630-2
1.7083
1910-3
3.303
5.100-2
1.30400
2.83822
Sin
2.48--2
198-2
2015-2
1.40-1
2646-2
344222
9.46-3
3 342822
1-43--1
2.10-22
10015
2.330-1
1.006-2
4
IDM
2.300-2
1-788-2
1220-1
212262
282-2
2306-2
5 3088-2
1.310-1
1.8062
1.082-2
1-980-1
1.470-2
3.6-8-2
HcMasker
8-178-22
1-110-2
19:10-2
4.200-2
1
1-48-1
1-28-1
1226-2
1 1:200-1
4.320-2
2.300-1
5 N
3-170-1
1.540-2
1.91-22
91
2.5
CSDUT
1.266-1
7.18--3
5.IEE-
LASE-2
3
4
7 2266-3
L
5.37633
4.5E-5
11158
4.08-3
2
2.68-22
9.0E-
LA3E-1
3.92
5.466-3
7.00E
GAIN
5:415-2
2646-2
5-456-1
5316
339E
2.796
9555
MIRACLE
3.258-2
1.748-2
1.44-22
K366:
35100
101ES
2500
SIKES
2.206
02153
MCAR
MIWKE
11.58-2
3196
1220-2
1488-
2145:
221
1288
1
1.36--
235E-3
7
221E3
TOM
8-200-2
-38-5
1.40-1
9 115
251
1056
2.000-1
HoMolker
2.540-22
2.580-2
5990-5
1:300-1
2 S
2330
190012
42400-
1.50-22
Newlmm
5.108-2
1.200-22
277365
6210-3
200-
2000
1
5256-1
3806-2
2:341-1
3-10-1
1210:2
CSDI T
7,408-3
1.99622
13100
1.000:
55055
1555-1
3.111
9
3:300-1
6.908-1
MicCTHI
3.535-1
S71R-3
11
6. 6.190
5 SAIT
1 14022
0 530022
4 MMRES
3.3100
1360-3
196-1
GAINN
0780-1
1-
2 1
5.80-1
1-40-11
3DVAR
MIWAE
8.68-2
2.00E-2
2216-2
1318-1
0 02HS
1.48-1
4 1RE
133E
5.9E
5:126.3
2258-2
Saak
5.218-3
5.525-2
791E-3
10RE-1
01 STuss
3
1256
1
644E-5
3.550-1
231E-2
TOM
1-55-2
141E-1
2
4.67E-
4105-3
4.565-2
125E
2915-2
Now-Bar
2.37E-2
1.078-1
415E-3
1338-)
LX3B
2
4
1348
6TH33
9.1E 3
5.68. 3
3.1E
3.2E
1.1E-1
8.02.3
2345
46

---

Table G.7: Performance of MAE and WASS metrics at 50% missing rate.
Scemaiio Model
BT
BCD
CC
CBV
IS
PK
QU
WOW
MAB
WASS
MAE
WASS
MAR
WASS
MAE
WASS|
MAA
WASS
|MAE
WASS
MMAE
WASS
MAR
WASS
CSDLT
0.85
2.61
0.84
17.50
0.84
2.52
0.1
3.79
0.72
15.46
1084
13.20
1.660
19,79
0.78
4.09
MissDiif
0.78
0.82
15.22
0.84
1.34
0.2
3.35
0.2
13667
0.80
15.05
0.59
24.55
0.77
5.95
GAIN
0.74
0.60
0.55
2,80
0.77
0.55
0.74
1.05
0.60
3.06
0.52
2.22
0.67
8.36
0.88
1.84
MIRACLE
0.66
11.49
0.85
7.33
0.2
0.41
0.63
0.78
3:21
50.79
3.88
48.31
0.52
5.27
0.58
0.N7
MAR
MIWAE
0.03
0.50
0.51
2.51
0.79
0.56
3:28
2.26
6.20
1,05
4.40
6.44
1,47
TDM
0.78
0.81
0.79
5.03
0.50
0.90
0.78
1.19
0.68
3:43
0.77
4.03
0.56
6.13
0.76
1,42
ReMasker
0.40
0.28
0.29
1.47
0.98
1.35
0.51
0.54
0.52
2.02
0.51
2.01
0.55
4.70
0.57
0.85
NewImp
0.67
0.67
0.64
2,05
0.51
0.48
0.47
0.55
0.45
2.20
0.48
2.12
0.42
4.72
0.60
1.00
CSDLT
0.73
1.86
0.74
15.79
0.85
2.71
0.82
3.99
0.75
14.56
0.72
11.31
0.57
19.45
0.79
4.31
MisaDief
0.72
1.44
0.74
14.15
0,85
1.46
0.81
2.83
0.75
13.14
0.71
11:70
0.56
20.15
0.76
3.45
GAIN
0.70
0.52
0.50
3.34
0.87
1.29
1.11
3.32
0.92
12.89
0.65
4.72
LOS
31.75
2.22
MIRACLE
0.63
60.64
57.33
1.21
MCAR
MTWAE
0.64
0.38
0.55
4.71
0.81
1.03
0.84
1.77
0.68
8.31
0.56
3.63
0.52
9.83
0.70
1,30
Sink
0.74
0.68
0.75
7.50
0.84
1.28
0.82
1.91
0.75
8:27
0.74
5.64
0.61
9.24
0.76
1.92
TDM
0.72
0..64
0.67
6.36
0.8
1.18
0.79
1.50
0.69
7.53
0.68
5.05
0.55
8.98
0.75
1.85
RcMasker
0.58
0.44
1X0
1222
8:24
2811
2.32
1.16
NowImp
2.23
CSDLT
0.75
2,077
0.78
15.16
0.86
0.82
3.95
0.75
14.54
0,80
12.42
0.60
19,50
0.79
4,25
MissiDier
0.72
0.77
14.78
0,85
1.44
0.52
3.08
13.52
0.77
11.83
3.05
GAIN
0.75
0.74
0.59
12.08
0.93
1.64
1.22
3,5/7
1.67
31.57
0.92
9.02
1.37
49.24
1.20
4.50
MIRACLEE
F
0.83
0.74
1.14
37.08
0.70
0.79
0.74
1.39
2.52
61.17
1.14
62.26
0.71
14.79
0.66
1,31
MNAR
MIWAE
0.73
(1.63
0.61
5.73
0.3
1.08
0.55
1344
0.66
7.5
D61
4.33
0.53
10.68
0.71
1,57
Sink
0.78
0186
8.47
0.76
5.19
6.34
0.63
10.57
2.01"
0.76
0.72
6.04
10.20
1,91
ReMasker
0.61
0.42
0.55
5.22
0.99
1.56
0.76
0.78
9.3
0.63
5.39
0.50
9.25
0.65
1,22
NowImp
0.74
0.84
0.39
3.12
0.69
0.98
0.65
1.23
0.47
5.03
0.48
3
0.44
8.38
0.65
1.47
Kindly Note: The best results are bolded and the second best results are underliend.
"*" marks the results
that Newlmp significantly outperform with p-value< 0.05 over paired samples I-test.
Table G.8: Standard deviation of MAE and WASS metrics at 50% missing rate.
Model
WXOW
MAF
WXSS5
MAR
WXS5
MAP
WASS
MAF
WASS
MAA
WASS
MAB
WA.55
MAP:
WAS5
MAR
WASS
MisDilf
4.506-2
3.45-1
5125-5
1.555
1.24E-
1.0E.1
5.44.
8.50-
376E-2
2.7E
2.0250
3.115
3,7350
2.21E-2
1
GAINN
5.00-1
1222-1
6-100-2
5.000-
4M70-2
9.80-2
2486-3
2-48--1
4.80--
8.40 K.40-2
1771.0
1.00-3
MIRACTE
M.10-2
1940
3240-2
6.2822
1.41-1
2640-2
5988-1
1
5.800-
1
2988:2
Sin
4.00-22
1388--1
8045-5
2.9062
3.2622
6.300-1
3 30862
1
3.8882
4.245-2
2/22/2
7.388-2
IDM
5-400-2
1
122022
23916-2
4
1.138-1
7 7-288-2
1 198622
9416-2
1.34b-2
9 0
2
8-220-2
3022-2
HcMasker
7-290-22
2990-2
1410-1
9-308-2
2.198-2
3.198-1
2
7-24b-2
4.002-1
2-388-1
2.263-2
CSDUT
2.28822
1.58E-1
3.14--3
1,380
2
2578-3
501E 2
5396-3
6
SJMES
1,588-1
L.11E-2
7.6EE-2
2.626-3
3
2105
4
GAIN
2-26-2
1.756
439
5
243ED
135
1.3E
7.1560
LAIE:
671E-
MIRACLE
2.198-2
2.368-2
7616-2
19380
1358:
$
315E0
48653
2 21000
315E
1966
2255
S
MCAR
MIWKE
6.76.
266-2
LISE-
$
$315:
15TE
MIME
31
1986-3
1
TOM
7.888-1
23305-2
2330-5
30123
3.00-1
HoMoiker
5.170-2
660-2
6800-4
4310-1
11 1122
1900-
200
134522
4.30--1
1080-1
2000-2
Newlmm
0-220-2
20000-2
572065
101001
5. 5.06
8 Se0a3
1
2
1.576-1
CSDI T
5.00-3
1.00-2
1119-1
6 0905
1000-1
1.50155
6.718-1
5
MicCTH
3-390-3
1387-1
6700-3
1-40-1
1 1300
9 MA0TE
1 111
10100
LSTTEO
GAINN
3.170-3
1.11-1
21
1.300
7711-3
1.358
LERED
30VAR
MIWAE
2.88E-2
7.04E-2
181E-1
633
1336:
5363
2TH
S
166E
2055
173E
3.51E
4118
3
4.00--
1.225-2
613E-5
1335--1
341E2
2 2:6
0 0005
9248
1256
3 120
754E-5
1.558-1
7.086-3
4008-2
TOM
2-37E-3
1.405-2
5-716-1
1
141E-
6:226-3
2
434E-
5758.1
1
5.48
1955-2
Now-Barp
4.31E-3
1,42--
8366-3
1A38-1
1
1
26733
8788 2
1.48- 2
1.00.
6.32
2.4EE
1.11E
4158.2
47

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The main claims made in the abstract and introduction accurately reflect our paper's contributions and scope. We restrict our application in missing value imputation task in numerical tabular, and our analysis is mainly focused on diffusion models, where the score function is required.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
Our limitations are listed in Appendix G.I.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "L.imiations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
48

---

Answer: [Yes]
Justification: To uphold the rigor of our manuscript, we provide all proofs of our proposition outlined in Appendix C. Besides, all theorems properly cited in the manuscript.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We attempt to list all hyperparameters Appendices and E to ensure reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
cases, in which
5. Open access to data and code
49

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We used the open access UCI datasets, and we uploaded our algorithm in this github link http:://gthhu.comm// hww..//////.uss...i////////////a
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips...
onooooooooooooo icy for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ wtwppsccpppullpippppptpppppppppp wwtppccccc/ccccccccccccccccctalt /gwui/gus//////ee //eodeedmmumii corbbbisssionn wwwssssliiooooooooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We have included all detailed information Appendices and
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: In Tables F.3 F.9 and G.1 G.8, and Figs. F.4 F.7, we report standard deviation errors suitably and correctly defined or other appropriate information about the statistical significance and error bar of the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
50

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for caleulating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (c.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The required information is given in Appendices E.2 and F44
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics ptttts://neuriiie cc httollic//hiishiiideeeoo
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Since it is an algorithm-oriente research, there is no societal impact of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
51

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper does not involve the safeguards issue.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
The creators or original owners assets (c.g., code, data, models), used in the paper, are properly credited. The license and terms of use are explicitly mentioned and properly respected.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
52

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
If assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, pwperswithcae thcode. com/ conddataset has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not not release new assets in this manuscript.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/model as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our experiments did not involve 'Crowdsourcing and Research with Human Subjects'.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our answer is NA since our paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
53

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
54


Reviews:
[{"review_id": "Dj8UvvKYrN", "reviewer": "Reviewer_Gtbe", "summary": "This paper introduces Kernelized Negative Entropy-regularized Wasserstein Gradient Flow Imputation (KnewImp), a novel approach for imputing missing data in numerical tabular datasets. The proposed method addresses two significant challenges in diffusion model-based missing data imputation (MDI): inaccurate imputation and difficult training. KnewImp integrates the Wasserstein gradient flow (WGF) framework with a negative entropy-regularized (NER) cost functional to enhance imputation accuracy and simplify the training process by eliminating the need for complex mask matrix designs. The method's efficacy is demonstrated through extensive experiments, showing superior performance compared to state-of-the-art imputation techniques.", "questions": "1. Can the authors elaborate on how the proposed method could be adapted or extended to handle other types of data, such as categorical or mixed-type datasets?\n\n2. What are the computational requirements for implementing KnewImp in practice, and how does it scale with larger datasets?", "limitations": "The authors have adequately addressed the limitations.", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The paper presents a unique integration of diffusion models with the Wasserstein gradient flow framework, incorporating a novel negative entropy regularization to address specific challenges in missing data imputation.\n\n2. The work is grounded in solid theoretical foundations, providing clear proofs and propositions that establish the effectiveness and validity of the proposed approach.\n\n3.  Extensive experiments on real-world datasets validate the method's superiority, with significant improvements in both mean absolute error (MAE) and Wasserstein distance (Wass) metrics.", "weaknesses": "1. The theoretical concepts and mathematical formulations presented are quite dense and may be challenging for readers not well-versed in advanced optimization and diffusion models. Simplifying these explanations or providing more intuitive descriptions could improve accessibility.\n\n2. While the method is compared to several models, including a wider range of baseline methods, particularly more recent advancements in diffusion-based MDI (e.g., [1,2]), would provide a more comprehensive evaluation.\n\n3. The paper could benefit from a more detailed discussion on the convergence properties of the proposed algorithm, including potential limitations and scenarios where the method might struggle.\n\n[1] Du, Tianyu, Luca Melis, and Ting Wang. \"ReMasker: Imputing Tabular Data with Masked Autoencoding.\" International Conference on Learning Representations, 2024.\n\n[2] Zheng, Shuhan, and Nontawat Charoenphakdee. \"Diffusion models for missing value imputation in tabular data.\" NeurIPS 2022 First Table Representation Workshop.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful advice and valuable questions, we will respond to your concerns point by point.\n## Weaknesses \n> W1: Dense Mathematical\n* We acknowledge that the theoretical concepts and mathematical formulations in our manuscript could be challenging for readers not extensively familiar with advanced optimization and diffusion models. \n * To enhance the accessibility and readability of our work, we will include a new subsection in the appendix of our revised manuscript. This subsection will detail the derivation and meaning of each equation, ensuring that the mathematical underpinnings are more comprehensible. \n\n> W2: Extra Basline\n* **For CSDI_T [1]**:\n  - Its primary contribution involves the introduction of one-hot encoding, analog bits encoding, and feature tokenization to manage categorical variables in MDI tasks on tabular data. Our research, however, **specifically focuses on numerical tabular data and assumes the absence of categorical data**. Due to this distinction, we did not consider CSDI_T as a relevant baseline for our study.\n  -  Nevertheless, the categorical feature extraction module in CSDI_T, once removed, essentially transforms it into the CSDI model, which we have indeed utilized and referenced in our manuscript.\n* **For ReMasker [2]**:\n  - We have included an additional baseline model named ReMasker in our comparisons. Please see the common rebuttal chat window. We will add the results of this baseline model in the revised manuscript.\n\n> W3: Convergence\n* We have included discussions on the convergence located **in Appendix E.3 with theoretical proofs and empirical validations** on all datasets in our manuscript. \n* We will add a footnote in our revised manuscript to demonstrate this point.\n## Questions:\n>Q1:  Mixed Type Data:\n* The key to applying our approach to such datasets involves **decomposing the missing data distribution, $ r(\\boldsymbol{X}^{({miss})}) $**, into a product of distributions for the dense and categorical components under a mean-field assumption common in variational inference [3]. Specifically, we express it as **$ r(\\boldsymbol{X}^{({miss})}) = r(\\boldsymbol{X}^{({miss, dense})})r(\\boldsymbol{X}^{({miss, cate})}) $**.\n* For the dense data component, we continue to employ our KnewImp method. \n* For the categorical part, we can initially model it using a Dirichlet distribution, which naturally supports simplex spaces, the steps are summarized as follows. \n  - First, we can implement **mirror descent [4] with the operator $ \\nabla_x\\psi(x) = \\log{x} $**, which maps the distribution’s support from the simplex $ \\Delta^{\\text{C}-1} $ (where $ \\text{C} $ represents the number of categories) onto $\\mathbb{R}^{\\text{C}}$. \n  - Subsequently, we apply KnewImp in this transformed space ($\\mathbb{R}^{\\text{C}}$). \n  - After that, we can revert the distribution back to the simplex **using the inverse operator $[\\nabla_x\\psi(x)]^{-1} = \\text{Softmax}(x)$.**\n* Besides, our example located in Section 1.2 of the attached PDF has realized this scheme, where the constraint of variables is a three-dimensional standard simplex.\n\n>Q2: Requirement  \n* **Our computational requirement is given in lines 743 to 746**: `all experiments are conducted on a workstation equipped with\nan Intel Xeon E5 processor with four cores, eight Nvidia GTX 1080 GPUs, and 128 GB of RAM.`, and we found that this configuration is enough for WQW dataset with 4898 items.  \n* In addition, **we further provided detailed time complexity analysis in Appendix E.2 theoretically and empirically**, which can mirror its scalability with larger datasets.\n\n---\nThank you for reading our rebuttal! **We hope the above discussion will fully address your concerns about our work, and we would really appreciate it if you could be generous in raising your score.**\n\n---\nRefs    \n[1] \"Diffusion models for missing value imputation in tabular data.\" NeurIPS'22 First Table Representation Workshop.  \n[2] \"ReMasker: Imputing Tabular Data with Masked Autoencoding.\" ICLR'24.  \n[3] \"Variational algorithms for approximate Bayesian inference\", Doctoral Thesis'03.  \n[4] \"Sampling with Mirrored Stein Operators\" ICLR'22", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PFBm8EMpFA", "reviewer": "Reviewer_EhUg", "summary": "The paper proposes a new algorithm for data imputation. The idea is to estimate the score function corresponding to the posterior p(x_miss/x_obs) using DSM and then infer the missing values using a WGF equivalence argued in this paper itself. These alternating steps are repeated until convergence. Simulations on benchmark datasets illustrate the efficacy of the proposal.", "questions": "1. I am not sure how the equivalence in (2) follows. It seems the objective in RHS is independent of X_miss as it is integrated out. objective in RHS is a function of r, p, X_obs. Whereas objective in LHS is a function of X_miss, does not involve r. It will be helpful if this equivalence is clarified. Wrt. to the optimization variable the objective in RHS seems to be a constant. Is there any notation that I am missing? Since it is a basic step and subsequent derivations depend on this critically, I was unable to check the correctness of soem of the steps. \n\n2. Morevoer, (4) seems to clearly show that the regularizer is not a function of X_miss. Then I am not sure how this regularizer, which is essentially a constant, matter.\n\n3. Reading line 158 gives an impression that r is the unknown. However, r does not seem to appear in LHS of (2) . If it does, when what is the relation with p in the objective of (2) ?\n\n4. is the assumptions of prop 3.4 meaningful? If X_Obs are X_miss are independent, then what information will the observations provide for imputation? will the imputation problem remain meaningful ? Please clarify this.\n\n5.", "limitations": "na", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. In empirical comparisons, the proposal seems to beat existing baselines.", "weaknesses": "1. There are some technical concerns I have raised in the next section\n\n2. I found description in sec34 very cryptic. It would have been nice if important portions of related appendix were moved to the main section. Also, will the proposed alternating style algorithm converge? Do we know anything property of this converged solution? Some discussion around the final algorithm would have helped in understanding the methodology better.\n\n3.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments. Before reading our response, we think we should come to the following agreements: \n1. Optimizing the instances $x$ from distribution $r(x)$ is optimizing this distribution $r(x)$, which is the basis of particle-based variational inference like Stein Variational Gradient Descent [1]. \n2. The velocity filed $v$ drives the optimization of the cost functional (the 'function' of function), according to Eq. (A.10), is changing and hard to be 0, until we reach the equilibrium point (nearly impossible at the beginning of the imputation procedure). \n3. **During imputation procedure, $\\int{r(\\boldsymbol{X}^{(miss)})\\mathrm{d}\\boldsymbol{X}^{(miss)}}=1$ is a constant (normalization constraint), but $r(\\boldsymbol{X}^{(miss)})$ and $\\mathbb{H}[r(\\boldsymbol{X}^{(miss)})]\\coloneqq-\\int{r(\\boldsymbol{X}^{(miss)})\\log{r(\\boldsymbol{X}^{(miss)})}\\mathrm{d}\\boldsymbol{X}^{(miss)}}$ are not constant, since $\\boldsymbol{X}^{(miss)}$ is changing**  \n4. Furthermore, $\\log{p(\\boldsymbol{X}^{(miss)}\\vert \\boldsymbol{X}^{(obs)} )}$ is not a constant when we change $\\boldsymbol{X}^{(miss)}$ unless it is a uniform distribution, which is also nearly impossible in practice.\n\n## Weakness:  \n> W1: Technical Concerns\n\nPlease see next part for detailed information.\n> W2: Convergence\n* Our convergence discussions have already been listed in Appendix E.3 theoretically and empirically. \n* For the final solution, all we can know is that it may reach the equilibrium point, where $\\nabla_{\\boldsymbol{X}^{(miss)}}\\frac{\\delta \\mathcal{F}}{\\delta r(X^{(miss)})}=0$ holds. \n\n## Questions:\n> Q1: Understanding of Eq. (2) `It seems the objective in RHS is independent of X_miss ... of r, p, X_obs. Whereas objective in LHS ... X_miss, does not involve r.` \n\nThe LHS is the content at the left side of $\\Rightarrow$, and RHS is the content at the right side of $\\Rightarrow$. Based on this, let's start to investigate Eq. (2) based on Agreement 1:\n*  The LHS indicates that we are finding some $\\boldsymbol{X}^{{(miss)}}$ from some unknown distribution $r(\\boldsymbol{X}^{{(miss)}})$ ($\\boldsymbol{X}^{{(miss)}}\\sim r(\\boldsymbol{X}^{{(miss)}})$), such that we can maximum the likelihood function $\\log{p(\\boldsymbol{X}^{{(miss)}}\\vert \\boldsymbol{X}^{{(obs)}})}$.\n* For optimizers, to the best of our knowledge whether GAMS solvers like GuROBi or Neural Network Solvers like Adam, can only handle scalar learning objectives. \n* Consequently, we can convert it to $\\frac{1}{M} \\sum_{i=1}^{M}{\\log{p(\\boldsymbol{X}^{{(miss)}}\\_i|\\boldsymbol{X}^{{(obs)}}\\_i)}}$, where $M$ is the missing value size. This is the **Monte Carlo (MC) Estimation** of term $\\mathbb{E}_{r}[\\log{p(\\boldsymbol{X}^{{(miss)}}_i)\\vert p(\\boldsymbol{X}^{{(obs)}})}]$ (to our understanding, the `integrated out` you mentioned is MC integration), which is the RHS. \n* Conversion in Eq. (2) is widely used in Quantum MC [2], where they sample some instances from an optimizable distribution and optimize the concerning functional based on these instances. \n\n> Q1: Constant Question `to the optimization variable the objective in RHS seems to be a constant.`\n\nThe functional $\\mathbb{H}[r(\\boldsymbol{X}^{{(miss)}})]$ is changing, not a constant since we mentioned in Eq. (A.10) $\\frac{\\partial r}{\\partial \\tau}=-\\nabla\\cdot(r v)$, with velocity $v$ unless $v=0$ based on Agreement 4. \n\n\n> Q2: r and $\\boldsymbol{X}^{{(miss)}}$\n\nBased on Agreement 1, this question is answered. \n> Q2: Is NER a constant?\n\nBased on Agreements 2 to 4, this question is answered. \n\n> Q3: $r$ is missed at the LHS of Eq. (2)\n\nThe exact expression of $r$ is not our concern, the vital component is $\\boldsymbol{X}^{(miss)}$, the imputed value (Agreement 1). By investigating funcional related to $r$ and optimizing concerning functional (**The $\\boldsymbol{X}^{{(miss)}}\\sim r(\\boldsymbol{X}^{{(miss)}})$ in LHS indicates that $r$ occurs in the LHS of Eq. (2)**), analyze why DM-based MDI approaches do not take effect, and propose improvements is the novelty of this manuscript.\n\n> Q3: $r$ is unkown:\n\nIt is hard to compute $p$ (existence of normalization constant) and $r$, **but we can still progressively increase the value of $\\mathbb{E}_{r(\\boldsymbol{X}^{(miss)})}[\\log{p(\\boldsymbol{X}^{(miss)}\\vert \\boldsymbol{X}^{(obs)} )}] -\\lambda \\mathbb{H}[r(\\boldsymbol{X}^{(miss)})]$ and realize MDI task (with input $\\boldsymbol{X}^{(miss)}$) throughout WGF, that's what we did in Section 3.2 and 3.3**.\n\n> Q3: Relationship between $r$ and $p$\n* Based on Agreement 1, we are representing $\\boldsymbol{X}^{(miss)}$ by $r$. \n*  $r$ is 'some' proposal distribution, and $p$ is the 'evaluator', which 'evaluates wheter $r(\\boldsymbol{X}^{(miss)})$/$\\boldsymbol{X}^{(miss)}$ is suitable', and 'reshape $r$ (realize by reshaping $\\boldsymbol{X}^{(miss)}$) to make it appropriate' based on cost functional $\\mathbb{E}_r[\\log{p}]-\\lambda \\mathbb{H}(r)$.\n* $r$ is akin to an actor, and $p$ is akin to a critic. The imputation procedure is akin to improving the actors ($r$/$\\boldsymbol{X}^{(miss)}$) with guidance (WGF) from the critics ($p$).\n\n> Q4: Assumption $r({\\boldsymbol{X}^{{(joint)}}})\\coloneqq r({\\boldsymbol{X}^{{(miss)}}}) p({\\boldsymbol{X}^{{(obs)}}})$\n\nPlease see point 3) in the common rebuttal chat window, where this decomposition is widely applied in mean-field variational inference [3].\n\n---\nThank you for reading our rebuttal! **We hope the above discussion will fully address your concerns about our work, and we would really appreciate it if you could be generous in raising your score.**\n\n---  \nRefs    \n[1]. \"Stein variational gradient descent: A general purpose bayesian inference algorithm.\" NeurIPS 16.  \n[2]. \"Ab initio solution of the many-electron Schrödinger equation with deep neural networks.\" Physcial Reviews Research  \n[3]. \"Variational algorithms for approximate Bayesian inference\", Doctoral Thesis'03.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "k6ACBAnU8N", "reviewer": "Reviewer_833r", "summary": "This paper considered tackling the Missing Data Imputation (MDI) problem via diffusion models, which treats MDI as an generative problem. As DM-based methods focus on sample diversification rather than accuracy, which is the primary evaluation metric for MDI, the authors proposed one cost functional to discourage diversification in sample generation based on the Wasserstein Gradient Flow framework. Moreover, given that the true values of the missing data are unknown, the authors proposed to replace the joint distribution with the conditional distribution throughout the learning procedure.", "questions": "It seems to the reviewer that the authors have made a strong assumption throughout this paper. Specifically, the assumption in Proposition 3.4 says that the joint distribution $r(X^{\\text{joint}})$ can be factorized as $r(X^{(\\text{miss})})r(X^{\\text{obs}})$, where $r$ denotes the probability density function. Would it be possible for the authors to discuss whether such assumption is realistic or not? Intuitively, it seems that the missing and observed entries can't be utterly uncorrelated.", "limitations": "It seems to the reviewer that there are too many typos in the current version of this paper, especially for the theoretical derivation part in the appendix. For instance, derivation of the equations from line 583 to line 584 seem to have lots of issues - the first step should contain some dot product? \n\nReferences:\n\n[1] Ansari, A. F., Ang, M. L., & Soh, H. (2020). Refining deep generative models via discriminator gradient flow. arXiv preprint arXiv:2012.00780.\n\n[2] Cheng, X., Lu, J., Tan, Y., & Xie, Y. (2024). Convergence of flow-based generative models via proximal gradient descent in Wasserstein space. IEEE Transactions on Information Theory.\n\n[3] Choi, J., Choi, J., & Kang, M. (2024). Scalable Wasserstein Gradient Flow for Generative Modeling through Unbalanced Optimal Transport. arXiv preprint arXiv:2402.05443.\n\n[4] Gao, Y., Jiao, Y., Wang, Y., Wang, Y., Yang, C., & Zhang, S. (2019, May). Deep generative learning via variational gradient flow. In International Conference on Machine Learning (pp. 2093-2101). PMLR.\n\n[5] Heng, A., Ansari, A. F., & Soh, H. Deep generative Wasserstein gradient flows.\n\n[6] Xu, C., Cheng, X., & Xie, Y. (2024). Normalizing flow neural networks by JKO scheme. Advances in Neural Information Processing Systems, 36.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "This paper focuses on two important questions faced by the diffusion model based solver of the missing data imputation problem. Extensive numerical experiments are provided to validate the effectiveness of the proposed methodology.", "weaknesses": "The main issue here is that incorporating Wasserstein Gradient Flow (WGF) with generative modeling doesn't seem to be a new idea. However, it seems that the authors didn't include any related references in section 2.3, which provides a brief review of WGF. For instance, it might be necessary to cite and discuss the following articles [1-6].", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments.\n\n> W1: Contributions & Novelty of This Paper\n\n* Our primary focus is on analyzing Diffusion Model (DM)-based Missing Data Imputation (MDI) using Wasserstein Gradient Flow (WGF, initially designed for functional optimization), **not merely integrating WGF into a generative model**. \n* The central contribution of this work is leveraging WGF as a tool to analyze and demonstrate the limitations of Diffusion Model-based MDI. This led us to redesign a novel functional and develop a unique computational approach for MDI. \n* To the best of our knowledge, Proposition 3.1 has not been documented in previous literature. Specifically, for the VP-SDE model, the functional of the Ornstein–Uhlenbeck process incorporates a variance term $\\mathbb{E}_{r}[\\boldsymbol{X}^{(miss)}]^\\top[\\boldsymbol{X}^{(miss)}]$ to promote diversification. This feature is notably absent in traditional gradient field-based (GF-based) generative models as referenced in [1-6]. \n* In our manuscript, we want to convey the concept that: MDI should not be treated as a generative problem (in our manuscript, we want to discourage diversification, but the generative models will encourage entropy function as per reference [1]: `The differential entropy term` $\\mathbb{H}(r)$ `improves diversity and expressiveness when the gradient flow is simulated for finite time-steps.` Similar entropy terms can be found in Eq. (15) in [1], Eq. (16) in [2], Eq. (3) in [3], Eq (1) in [4], Eq. (4) in [5], and Eq. (5) in [6]). \n* Hence the references [1-6] you recommended about using GF to improve generative models were not considered during the preparation of our manuscript, and our related works subsections located in Section 5 are chiefly organized from the perspective of DM's application in MDI and the application of WGF works which model the conditional distribution by joint distribution. \n* We plan to add another subsection to discuss the application of WGF in improving generative models to demonstrate the applicability of GF to improve generative model-related tasks and cite these references.\n\n> W2: Justification of Decomposition:\n* As for the decomposition, it is not a strong assumption for MDI. \n* **As we mentioned in our manuscript and common rebuttal chat window, our task is to find the missing value $\\boldsymbol{X}^{{(miss)}}$ and the $\\boldsymbol{X}^{{(obs)}}$ will not be changed ($p(\\boldsymbol{X}^{{(obs)}})$ is a constant measure).** \n* Based on this, when we want to sample $\\boldsymbol{X}^{{(miss)}}$ from distribution $r({\\boldsymbol{X}^{{(miss)}}})$, the results are the same as sampling from $r({\\boldsymbol{X}^{{(joint)}}})$ since $r(\\boldsymbol{X}^{{(obs)}})=p(\\boldsymbol{X}^{{(obs)}})$ is a constant measure according to reference [7,8], which is unchanged. \n* **The key is that $p({\\boldsymbol{X}^{{(joint)}}})\\neq p({\\boldsymbol{X}^{{(miss)}}}) p({\\boldsymbol{X}^{{(obs)}}})$, but $r({\\boldsymbol{X}^{{(joint)}}})= r({\\boldsymbol{X}^{{(miss)}}}) p({\\boldsymbol{X}^{{(obs)}}})$ is justified**. \n* Furthermore, similar assumption on the $r$, the 'ansatz'/variational distribution/approximate distribution/proposal distribution, can be found in mean-filed variational inference represented by reference [9].\n\n> L1: Typos on Eq. (B.2): \n\nWe regret any oversight **regarding the omission of the $\\nabla\\cdot$ operator (divergence operator), and velocity term $v_\\tau$ in our description of the continuity equation**. \n* This equation is given as $\\frac{\\partial r(\\boldsymbol{X}^{{(miss)}})}{\\partial \\tau}=-\\nabla\\cdot[v_{\\tau}(\\boldsymbol{X}^{{(miss)}})r(\\boldsymbol{X}^{{(miss)}})]$, the second equality can be obtained similar to the derivation of Fokker-Planck-Kolmogorov equation, where $v_{\\tau}(\\boldsymbol{X}^{{(miss)}})=-\\nabla{\\log{p(\\boldsymbol{X}^{{(miss)}}\\vert \\boldsymbol{X}^{{(obs)}})}}-\\lambda\\nabla{\\log{r(\\boldsymbol{X}^{{(miss)}})}}$ and $\\nabla\\cdot[(\\nabla{\\log{r(\\boldsymbol{X}^{{(miss)}})}})r(\\boldsymbol{X}^{{(miss)}})] =\\nabla\\cdot[\\frac{\\nabla {{r(\\boldsymbol{X}^{{(miss)}})}})}{r(\\boldsymbol{X}^{{(miss)}})}r(\\boldsymbol{X}^{{(miss)}}] =\\nabla\\cdot\\nabla r(\\boldsymbol{X}^{{(miss)}})$. \n* We commit to revising our manuscript to rectify any ambiguities and ensure clearer presentation. \n\n---\nThank you for reading our rebuttal. **Given the above infos, we hope that these points could be kindly considered in the evaluation of our work, and we would really appreciate it if you could be generous in raising your score.** \n\n---\nRefs    \n[1]. \"Refining deep generative models via discriminator gradient flow.\" ICLR'21  \n[2].  \"Convergence of flow-based generative models via proximal gradient descent in Wasserstein space.\" IEEE TIT  \n[3]. \"Scalable Wasserstein Gradient Flow for Generative Modeling through Unbalanced Optimal Transport.\" ICML'24  \n[4]. \"Deep generative learning via variational gradient flow\". ICML'19  \n[5]. \"Deep generative Wasserstein gradient flows\"  \n[6]. \"Normalizing flow neural networks by JKO scheme\" NeurIPS'23  \n[7]. \"Posterior Sampling Based on Gradient Flows of the MMD with Negative Distance Kernel.\" ICLR'24.  \n[8]. \"Nonparametric generative modeling with conditional sliced-Wasserstein flows.\" ICML'23.  \n[9]. \"Variational algorithms for approximate Bayesian inference\", Doctoral Thesis'03.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "x8iTaeCpwx", "reviewer": "Reviewer_fS2D", "summary": "This paper presents KnewImp, a kernelized negative entropy-regularized Wasserstein gradient flow imputation approach to numerical tabular data imputation. The authors argue that existing missing data imputation frameworks based on diffusion models suffer from two major limitations. Firstly, diffusion models primarily focus on sample diversification rather than accuracy, which results in discrepancy between the training objective of the diffusion models and the aim of tabular data imputation. Secondly, existing approaches are trained by masking parts of the observed data and then predicting the masked entries. This results in training difficulty due to the need of designing complex mask matrix selection mechanisms. To address the limitations of existing models, the authors propose a Wasserstein gradient flow based framework, which employs a novel cost functional with diversification-discouraging negative entropy as regularization. KnewImp is derived within the Wasserstein gradient flow framework, reproducing the kernel Hilbert space. To bypass the need of the mask matrix and to make the model easier to train, the authors further develop a novel cost functional based on joint distribution. Experimental results on a variety of real-world datasets show that KnewImp achieves state-of-the-art, outperforming a number of state-of-the-art baseline alternatives.", "questions": "- In line 150, the authors claim that \"This diversification is fundamentally at odds with the precision required in MDI tasks.\" I understand that the need of discouraging diversification in favor of precision is the main motivation of KnewImp. However, I wonder if the authors can provide a more intuition explanation about this claim, or if the discrepancy between accuracy and diversification can be somehow quantified.\n\n- Why did the authors primarily focus on MAR and MCAR rather than MNAR? Is this because of the setting of KnewImp is not particularly suitable for MNAR?\n\n- In Table 2, not all results in bold are best results.", "limitations": "The authors have adequately addressed the limitations and potential negative societal impact of their work, and have provided sufficient justification.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- KnewImp is theoretically sound and achieves state-of-the-art results on a number of real-world datasets.\n\n- The paper is in general well-written and easy-to-follow.", "weaknesses": "- My main concern is that, the authors only presented results in terms of MAE and Wass, but did not show results on downstream tasks. It is therefore unclear whether KnewImp would be effective in real-world scenarios. I would consider raising my score if the authors can provide the result of KnewImp on downstream tasks. The authors can refer to Figure 5 of the TDM paper in terms of downstream results.\n\n- From the ablation study, I think the majority of the performance improvement comes from modeling the joint distribution directly. The diversification-discouraging negative entropy regularization only marginally improves the performance. The main novelty of KnewImp is from the NER objective. However, ablation study suggests that it does not add much to the final performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments, our rebuttal is posted as follows:\n> W1: Downstream Tasks\n\nAccording to your suggestions, we added the performance on downstream classification task similar to Fig. 5 in the TDM paper [1]. **Please see Table 1 located in Section 2.1 from the pdf attached in common rebuttal chat window**.\n> W2: Ablation Study\n\nThank you for your feedback on our ablation study. \n* There seems to be some misunderstanding about the contributions of KnewImp. Our method primarily introduces WGF to analyze and enhance DM-based MDI. This introduces two main innovations: NER and Joint Distribution Modeling. \n* **The function of NER term is that it introduces a functionally effective modification to the existing DM -based MD—Optimizing the NER functional aligns well with optimizing the joint/conditional log-likelihood objective**. \n* Based on this, **the ablation study aimed to show that including NER maintains an effective lower bound for MDI tasks—meaning it does not degrade model performance in general**. \n* Besides, we can also turn to Table E.4, when we add NER term into the model, **the standard deviation of KnewImp with NER is generally smaller than those without NER term**. These results further confirm that NER, although providing marginal performance gains, while enabling the KnewImp has a smaller standard deviation, is crucial for the model's theoretical robustness.\n\n> Q1: Motivation  \n\n* To clarify the trade-off between diversification and accuracy, which is a central theme of our method KnewImp, let's consider a practical example. Suppose we denote the true value by $x$ and the imputed value by $ \\tilde{x}$. The goal in terms of accuracy is to minimize the discrepancy $Dis(x, \\tilde{x})$, where $Dis$ is discrepancy metric. \n* Diversification tends to increase either the variance $\\text{Var}(\\tilde{x})$ or the entropy $\\mathbb{H}(\\tilde{x})$. These measures do not directly involve ground truth $x$. \n* In DMs, **where entropy is used as a term to encourage diversification (our proposition 3.1), the outcome may converge towards a uniform distribution [2]. In such a distribution, every potential value within the support is equally probable as the imputed value, which is undesirable for MDI.**\n\n> Q2: Scenario Restriction\n\nThank you for your question regarding our focus on MAR and MCAR rather than MNAR. \n* We prioritize MAR and MCAR because **MNAR involves complexities due to its dependency on unobserved data, requiring in-depth knowledge of the missingness mechanism which is often challenging to determine [3,4]**. \n* For instance, in privacy-sensitive studies like diabetes medication usage, the non-response is directly linked to the privacy nature of the participants, typical of MNAR scenarios. \n* Given these challenges, we focus on MAR and MCAR for their more straightforward assumptions and applicability. \n* Nevertheless, to ensure a comprehensive analysis, **we include findings related to MNAR in Tables E.1 and E.3 of our manuscript**, demonstrating our approach's applicability under these conditions as well.\n\n> Q3: Marking Mistake \n \nThank you for pointing out our problems, we will revise this table in our revised manuscript.\n\n---\nThank you for reading our rebuttal, **we hope our response alleviates your problem and we would really appreciate it if you could be generous in raising your score.**\n\n---\nRefs  \n[1]. \"Transformed distribution matching for missing value imputation.\" ICML'23.  \n[2]. \"Nonlinear Stein Variational Gradient Descent for Learning Diversified Mixture Models\" ICML'19  \n[3]. \"StableDR: Stabilized doubly robust learning for recommendation on data missing not at random.\" ICLR'23  \n[4]. \"not-MIWAE: Deep generative modelling with missing not at random data\" ICLR'21", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FiJ6TM1lYe", "reviewer": "Reviewer_9XXU", "summary": "This paper addresses two primary issues in Missing Data Imputation (MDI) using Diffusion Models (DMs): inaccurate imputation due to sample diversification and difficult training caused by the complexity of designing the mask matrix. The authors propose a novel approach, Kernelized Negative Entropy-regularized Wasserstein Gradient Flow Imputation (KnewImp), which aims to resolve these issues for numerical tabular datasets.", "questions": "1. How robust is KnewImp to variations in data distributions, such as heavy-tailed, multimodal, or skewed distributions?\n2. Why was the radial basis function (RBF) kernel chosen for the RKHS, and how does the choice of kernel affect the performance of KnewImp?", "limitations": "Yes. Discussed in Appendix F.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper addresses two issues in the domain of DM-based MDI—sample diversification leading to inaccurate imputation and the complex training process due to mask matrix design. By identifying and explicitly addressing these issues, the paper provides a fresh perspective on improving MDI techniques. The introduction of the negative entropy-regularized cost functional is a creative and innovative approach to discourage diversification, aligning the generative model’s objectives more closely with the needs of MDI tasks. The integration of the WGF framework with RKHS to derive an imputation procedure is an original and elegant solution.\n- The paper provides thorough theoretical analyses and proofs, ensuring the soundness of the proposed approach. The extensive experiments conducted on multiple real-world datasets from the UCI repository underscore the robustness and effectiveness of the KnewImp approach. The detailed ablation studies and sensitivity analyses enhance the quality of the research by thoroughly examining the contributions of different components and the impact of key hyperparameters.\n- The paper is well-organized, with a clear delineation of problems, proposed solutions, theoretical foundations, experimental setup, and results. Each section logically flows into the next, making it easy to follow the authors’ arguments.", "weaknesses": "- The choice of kernel in the Reproducing Kernel Hilbert Space (RKHS) can significantly impact the performance of the method. The paper primarily uses the radial basis function (RBF) kernel but does not explore the effects of using different types of kernels or the rationale behind choosing the RBF kernel. A detailed analysis of how different kernel choices affect the imputation quality and computational efficiency would strengthen the technical rigor of the study.\n- The theoretical foundations of KnewImp rely on certain assumptions about the data distribution, such as the smoothness of the underlying density functions. The robustness of the method to violations of these assumptions is not thoroughly investigated. Including experiments that test the method’s performance on datasets with varying statistical properties (e.g., heavy-tailed distributions, multimodal distributions) would provide a more comprehensive assessment of its robustness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments, our rebuttal organized point to point is posted as follows:\n> W1 & Q2: Why we use RBF Kernel Function:\n* The selection of the RBF kernel function was strategically driven by the need to satisfy the following condition: $\\int{\\nabla_{\\boldsymbol{X}^{(miss)}}[u(\\boldsymbol{X}^{(miss)} ,\\tau) r(\\boldsymbol{X}^{(miss)})]\\mathrm{d}\\boldsymbol{X}^{(miss)}}=\\underbrace{\\int{r(\\boldsymbol{X}^{(miss)})\\nabla\\_{\\boldsymbol{X}^{(miss)}}[u(\\boldsymbol{X}^{(miss)} ,\\tau) ]\\mathrm{d}\\boldsymbol{X}^{(miss)}}}\\_{\\mathbb{E}\\_{r(\\boldsymbol{X}^{(miss)})}[ \\nabla\\_{\\boldsymbol{X}^{(miss)}}[u(\\boldsymbol{X}^{(miss)} ,\\tau) ]  ]} + \\int{u(\\boldsymbol{X}^{(miss)} ,\\tau)^\\top\\nabla_{\\boldsymbol{X}^{(miss)}}[r(\\boldsymbol{X}^{(miss)}) ]\\mathrm{d}\\boldsymbol{X}^{(miss)}}  =0$. This condition is pivotal as it allows us to circumvent the direct, explicit estimation of $r(\\boldsymbol{X}^{(miss)})$ during the imputation procedure. \n* A sufficient condition for $\\int{\\nabla_{\\boldsymbol{X}^{(miss)}}[u(\\boldsymbol{X}^{(miss)} ,\\tau) r(\\boldsymbol{X}^{(miss)})]\\mathrm{d}\\boldsymbol{X}^{(miss)}}=0$ is that \"$r(\\boldsymbol{X}^{(miss)})$ is bounded and $\\lim_{\\Vert \\boldsymbol{X}^{(miss)} \\Vert\\rightarrow \\infty}u(\\boldsymbol{X}^{(miss)} ,\\tau)=0$\".\n* Consequently, the key to choosing kernel function is to validate the condition as follows: For $x$, kernel function $\\mathcal{K}(x,x')$ should satisfy the boundary condition: $\\lim_{\\Vert x \\Vert\\rightarrow \\infty}\\mathcal{K}(x,x')=0$. \n* Conventional kernel functions, like linear kernel, sine kernel, polynomial kernel, cosine similarity kernel... cannot satisfy this condition. Thus we choose the RBF kernel in our manuscript similar to previous work [1]. \n* Furthermore, we propose the following experimental results at 0.3 missing rate (to consist with the table in main context, results for MNAR scenario are not posted due to space limit):\n|Scenario|Kernel|BT-MAE|BT-Wass|BCD-MAE|BCD-Wass|CC-MAE|CC-Wass|CBV-MAE|CBV-Wass|IS-MAE|IS-Wass|PK-MAE|PK-Wass|QB-MAE|QB-Wass|WQW-MAE|WQW-Wass|\n|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|\n|**MAR**|linear|0.77|0.73|0.83*|3.36*|0.85*|0.77*|0.83*|0.97*|0.72*|2.42*|0.8*|2.5*|0.7*|4.58*|0.76*|1.02*|\n||polynomial|1.12*|1.4*|1.44*|8.36*|1.06*|1.16*|1.08*|1.55*|1.07*|5.93*|1.33*|5.93*|1.28*|13.63*|1.02*|1.76*|\n||sigmoid|0.89*|1.02*|1.44*|10.8*|0.82|0.74*|0.81*|0.93*|1.17*|12.57|1.3*|9.87*|1.04*|7.42*|0.77*|1.04*|\n||cosine similarity|0.68|0.53|0.8*|3.14*|0.82*|0.73*|0.81*|0.92*|0.71*|2.37*|0.78*|2.37*|0.74|4.56|0.74*|0.98*|\n||sine|8.62*|95.45|8.74*|290.79*|14.83*|281.68*|17.93*|492.97*|10.56*|456.27*|10.25*|306.76*|8*|313.9*|15.14*|386.73*|\n||**RBF**|0.52|0.38|0.34|0.82|0.35|0.25|0.31|0.2|0.39|1.31|0.44|1.21|0.45|3.5|0.46|0.55|\n|**MCAR**|linear|0.71*|0.45*|0.87*|5.87*|0.83*|0.84*|0.81*|1.28*|0.81*|5.65*|0.83*|4.14*|0.62*|6.31*|0.76*|1.25*|\n||polynomial|0.94*|0.88*|1.31*|12.19*|0.98*|1.21*|0.99*|1.85*|1.11*|9.94*|1.27*|8.64*|1.11*|18.71*|0.92*|1.95*|\n||sigmoid|0.74*|0.48*|0.97*|8*|0.81*|0.82*|0.8*|1.23*|0.92*|7.36*|0.94*|6.54*|0.77*|7.74*|0.77*|1.28*|\n||cosine similarity|0.7*|0.42*|0.84*|5.51*|0.81*|0.81*|0.8*|1.24*|0.8*|5.48*|0.81*|3.96*|0.63*|6.01*|0.74*|1.2*|\n||sine|9.76*|95*|7.77*|434.72*|13.53*|331.59*|11.78*|352.67*|8.27*|542.27*|8.62*|384.97*|7.36*|468.07*|10.61*|289.41*|\n||**RBF**|0.48|0.18|0.25|0.8|0.47|0.34|0.42|0.44|0.44|3.05|0.32|1.01|0.34|3.66|0.53|0.76|\n\n    From the results presented in the table, it is evident that **the RBF kernel outperforms other kernels in scenarios where the boundary condition is not met. This observation empirically substantiates the effectiveness of the RBF kernel, thereby validating the justification for using RBF kernel**.\n* We decide to add the abovementioned contents to explain why we mainly consider RBF kernel in our revised manuscript.\n\n> W2 & Q1: Data Distribution Property\n\nThank you for your comments, our responses are listed as follows:\n* We have expanded the experimental results in the attached PDF to include KnewImp performance across various data distributions, such as heavy-tailed, multimodal, and skewed distributions. Please refer to **Section 2.2 in the supplementary PDF provided in the common rebuttal chat window**.\n* The additional results demonstrate that **KnewImp maintains consistent performance when transitioning from Gaussian to Skewed-Gaussian, Student's-t, and Gaussian Mixture distributions**. This underscores the **robustness of KnewImp across diverse data distributions**.\n* We will incorporate these findings into the revised manuscript to enhance its comprehensiveness and clarity.\n\n---\nThank you for reading our response, **we hope the above discussion fully addressed your concerns about our work, and we would really appreciate it if you could be generous in raising your score.**\n\n---  \nRefs  \n[1]. \"Stein variational gradient descent: A general purpose bayesian inference algorithm.\" NeurIPS'16.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
## Overall Response
We are encouraged by the reviewers' acknowledgment of the strengths in our paper, such as its robust performance [Gtbe] [EhUg] [833r] [fs2D] [9XXU], comprehensive experimentation [Gtbe] [fs2D] [9XXU], and clear, concise presentation [Gtbe] [fs2D] [9XXU]. However, we also recognize that there are common concerns raised by some reviewers regarding the 1). motivation for using Gradient Flow (GF) to discourage diversification [833r] [fs2D], 2). the assumptions related to the decomposition $r(\boldsymbol{X}^{(joint)})\coloneqq r(\boldsymbol{X}^{(miss)})p(\boldsymbol{X}^{(obs)})$ [EhUg] [833r], and 3). the adequacy of our experimental validation [Gtbe] [fs2D][9XXU]. To address these concerns, we offer the following clarifications:
## Motivations & Contributions:
The motivation behind KnewImp is to address specific challenges in diffusion model (DM)-based missing data imputation (MDI) tasks:
* **Inconsistency between DM's goals and MDI objectives:** As generative models, DMs inherently aim to diversify data with implicit regularization terms, which intuitively conflicts with MDI requirements that often demand precise values.
* **Design of Mask Matrix for Model Training:** DMs require a mask matrix to formulate conditional distributions [1], and the design of mask matrix is crucial to the imputation accuracy.

Based on this, our major contributions are summarized as follows: 
* **Introduction of GF in DM-based MDI:** We fconceptualize MDI as an optimization problem and employ GF, initially designed for functional optimization [2], to elucidate the shortcomings of DMs in MDI, particularly how DMs inadvertently promote diversification through terms like entropy and variance (Section 3.1).
* **Novel and Effective Cost Functional:** We introduce an effective cost functional that incorporates the negative regularization term, with a rigorously derived implementation strategy (Section 3.2).
* **Sidestepping Mask Matrix Design through Joint Distribution Modeling:** We demonstrate that within the GF framework, it is possible to circumvent the traditional mask matrix design and instead utilize a joint distribution modeling approach (Section 3.3).

## Weaknesses & Questions Response:
### 1). Diversification Discouraging [833r] [fs2D]: 
*  In Section 1.1 of our attached PDF, we analyze two distributions: uniform and normal. The uniform distribution exhibits higher entropy, aligning with the generative models‘ goal where **each value within the support is equally probable** (**maximum entropy may result in uniform distribution [3]**). This characteristic, however, **does not align with the objectives of MDI**, where specific values are often required. 
* Building on this analysis, Section 1.2 of our PDF compares KnewImp's performance when optimizing a cost functional related to a specified Dirichlet distribution. By gradually adjusting the weight of the negative entropy term ($\lambda$) of KnewImp from negative to positive, we demonstrate that increasing accuracy in MDI tasks may require a reduction in diversification. 
### 2). Assumption of $r(X^{(joint)}) \coloneqq r(X^{(miss)})p(X^{(obs)})$ [EhUg] [833r]:
 * We treat $r$ as a proposal distribution **akin to the approximation distribution within the variational inference context**, which is an ansatz meant to be optimized based on a functional related to the distribution $p(X^{(joint)})$.  
 * Thus, **it is crucial to verify the application of $p(X^{(joint)}) \stackrel{?}{=} p(X^{(miss)}) p(X^{(obs)})$, rather than $r(X^{(joint)}) \stackrel{?}{=} r(X^{(miss)}) r(X^{(obs)})$**. Fortunately, **KnewImp does not assume $p(X^{(joint)}) = p(X^{(miss)}) p(X^{(obs)})$ (i.e. $p(X^{(joint)}) \neq p(X^{(miss)}) p(X^{(obs)})$)**. Moreover, the mean-field assumption in variational inference, where $r(X^{(joint)}) = r(X^{(miss)}) r(X^{(obs)})$, is practical according to references represented by [4]. 
* Finally, **$X^{(obs)}$ remains unchanged in MDI, which indicates that $r(X^{(obs)})$ and $p(X^{(obs)})$ are identity and constant**, thus we can replace $r(X^{(obs)})$ with $p(X^{(obs)})$.
### 3). Extra Experiments [Gtbe] [fs2D][9XXU]:
 * We added results from ReMasker model [5] according to reviewer [Gtbe]:
|Dataset & Metric| BT MAE| BT Wass|BCD MAE|BCD Wass| CC MAE|CC Wass|CBV MAE|CBV Wass|IS MAE|IS Wass|PK MAE|PK Wass|QB MAE| QB Wass |WQW MAE|WQW Wass|
|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|
| MAR | 0.55| 0.43*| 0.50*|1.53* | 0.60*| 0.43*| 0.50*|0.40*|0.58*| 2.02*|0.53|1.31| 0.64*| 3.75*|0.53|0.59|
| MCAR| 0.44| 0.15| 0.37*|1.56*| 0.55*| 0.37 | 0.56*| 0.63*|0.55*| 4.10*|0.47*|1.51| 0.47*| 4.14*|0.56*|0.78|
| MNAR| 0.53| 0.26 | 0.42*|2.08*| 0.54*| 0.39*| 0.58*| 0.66*|0.50*| 3.57* | 0.56*| 2.59 |0.50| 5.53*|0.58|0.82|
 * We have included extra experimental results in Section 2.1 of our attached PDF as suggested by reviewer [fs2D], focusing on a downstream classification task similar to the experiments depicted in Fig. 5 of the TDM paper [6], using the `cross_val_score` function from the sklearn package (the rest of the settings are same as TDM paper).  
* We have incorporated additional experimental results concerning the performance of KnewImp across various data distributions in Section 2.2 of the attached PDF, as suggested by reviewer [9XXU].
---  
In all, thank you for considering our responses, and we look forward to any further feedback that might help refine our work.  

---
Refs  
[1]. "CSDI: Conditional score-based diffusion models for probabilistic time series imputation." NeurIPS'21  
[2]. "{Euclidean, metric, and Wasserstein} gradient flows: an overview." Bulletin of Mathematical Sciences    
[3]. "Nonlinear Stein variational gradient descent for learning diversified mixture models" ICML'19  
[4]. "Variational algorithms for approximate Bayesian inference", Doctoral Thesis'03.  
[5]. "ReMasker: Imputing Tabular Data with Masked Autoencoding." ICLR'24.  
[6]. "Transformed distribution matching for missing value imputation." ICML'23.

Meta-review: This paper discusses the applicability of Diffusion Models to tabular data. It makes the case that existing 
Models is not effective for imputation in numerical Tabular data. The paper suggests a novel regularisation term based on negative entropy and discusses a proedure for imputing Missing values. The proposal seems sound and empirically outperforms existing methods. The paper should be of interest to Neurips.
Decision: Accept (poster)