M$^3$-Impute: Mask-guided Representation Learning for Missing Value Imputation

Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-12 18:31:38

Abstract: Missing values are a common problem that poses significant challenges to data analysis and machine learning. This problem necessitates the development of an effective imputation method to fill in the missing values accurately, thereby enhancing the overall quality and utility of the datasets. Existing imputation methods, however, fall short of considering the 'missingness' information in the data during initialization and modeling the entangled feature and sample correlations explicitly during the learning process, thus leading to inferior performance. We propose M$^3$-Impute, which aims to leverage the missingness information and such correlations with novel masking schemes. M$^3$-Impute first models the data as a bipartite graph and uses an off-the-shelf graph neural network, equipped with a refined initialization process, to learn node embeddings. They are then optimized through M$^3$-Impute’s novel feature correlation unit (FCU) and sample correlation unit (SCU) that enable explicit consideration of feature and sample correlations for imputation. Experiment results on 15 benchmark datasets under three different missing patterns show the effectiveness of M$^3$-Impute by achieving 13 best and 2 second-best MAE scores on average.

Main paper: 
 related work: 
 Statistical methods: These imputation approaches include joint modeling with expectationmaximization (EM) [9, 16, 22] , k-nearest neighbors (kNN) [14, 43] , and matrix completion [5, 6, 18, 32] . However, joint modeling with EM and matrix completion often lack the flexibility to handle data with mixed modalities, while kNN faces scalability issues due to its high computational complexity. In contrast, M 3 -Impute is scalable and adaptive to different data distributions.Learning-based methods: Iterative imputation frameworks [1, 2, 15, 20, 23, 24, 35, 41, 44, 45] , such as MICE [45] and HyperImpute [23] , have been extensively studied. These iterative frameworks apply different imputation methods for each feature and iteratively estimate missing values until convergence. In addition, for deep neural network learners, both generative models [27, 29, 36, 50, 51, 53] , such as GAIN [50] and MIWAE [29] , and discriminative models [10, 24, 48] , such AimNet [48] , have also been proposed. However, these methods are built upon raw tabular data structures, which fall short of capturing the complex correlations in features, samples, and their combination [54] . In contrast, M 3 -Impute is based on the bipartite graph modeling of the data, which is more suitable for learning the data correlations for imputation.Graph neural network-based methods: GNN-based methods [40, 52, 54] are proposed to address the drawbacks mentioned above due to their effectiveness in modeling complex relations between entities. Among them, GRAPE [52] transforms tabular data into a bipartite graph where features are one type of node and samples are the other. A sample node is connected to a feature node only if the corresponding feature value is present. This transformation allows the imputation task to be framed as a link prediction problem, where the inner product of the learned node embeddings is computed as the predicted values. IGRM [54] further enhances the bipartite graph by explicitly introducing linkages between sample nodes to facilitate message propagation between samples. However, these methods do not effectively encode the missingness information of different samples and features into the imputation process, which can impair their imputation accuracy. In contrast, M 3 -Impute enables explicit modeling of missingness information through novel masking schemes so that feature-wise and sample-wise correlations can be accurately captured in the imputation process.3 M 3 -ImputeQuestion: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: In Section 4.2, we discussed two cases of MAE degradation for the KIN8NM and NAVAL datasets. It is mainly because 1. Each feature in KIN8NM is independent of the others, so none of the observed features can help impute missing feature values. 2. In the NAVAL dataset, nearly every feature exhibits a strong linear correlation with the other features. While it is true that M 3 -Impute does not achieve the best MAE on these two datasets, our model has outperformed all the other baselines on the majority of datasets. This demonstrates the unique strengths of graph modeling in M 3 -Impute over tabular data modeling in baselines like Hyperimpute. In real-world scenarios, the correlation structure of datasets is often unpredictable, and such extreme cases are relatively rare. Thus, we design a scheme to handle general cases for data imputation tasks. The empirical evidence suggests that our approach has been quite successful and exhibits overall superior performance to the baselines as it can be well adapted to each dataset that possesses different levels of correlations over features and samples.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.• The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.• The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.• The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.• The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.• If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.• While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
overview: 
 We here provide an overview of M 3 -Impute to impute the missing value of feature f for a given sample s, as depicted in Figure 1 . Initially, the data matrix with missing values is modeled as an undirected bipartite graph, and the missing value is imputed by predicting the edge weight êsf of its corresponding missing edge (Section 3.2). M 3 -Impute next employs a GNN model, such as GraphSAGE [17] , on the bipartite graph to learn the embeddings of samples and features. These embeddings, along with the known masks of the data matrix (used to indicate which feature values are available in each sample), are then input into our novel feature correlation unit (FCU) and sample correlation unit (SCU), which shall be explained in Section 3.3 and Section 3.4, to obtain feature-wise and sample-wise correlations, respectively. Finally, M 3 -Impute takes the feature-wise and sample-wise correlations into a multi-layer perceptron (MLP) to predict the missing feature value êsf (Section 3.5). The whole process, including the embedding generation, is trained in an end-to-end manner.Let A ∈ R n×m be an n × m matrix that consists of n data samples and m features, where A ij denotes the j-th feature value of the i-th data sample. We introduce an n × m mask matrix M ∈ {0, 1} n×m for A to indicate that the value of A ij is observed when M ij = 1. In other words, the goal of imputation here is to predict the missing feature values A ij for i and j such that M ij = 0.We define the masked data matrix D to be D = A ⊙ M, where ⊙ is the Hadamard product, i.e., the element-wise multiplication of two matrices.As used in recent studies [52, 54] , we model the masked data matrix D as a bipartite graph and tackle the missing value imputation problem as a link prediction task on the bipartite graph. Specifically, D is modeled as an undirected bipartite graph G = (S ∪ F , E), where S = {s 1 , s 2 , . . . , s n } is the set of 'sample' nodes and F = {f 1 , f 2 , . . . , f m } is the set of 'feature' nodes. Also, E is the set of edges that only exist between sample node s and feature node f when D sf ̸ = 0, and each edge (s, f ) ∈ E is associated with edge weight e sf , which is given by e sf = D sf . Then, the missing value imputation problem becomes, for any missing entries in D (where D sf = 0), to predict their corresponding edge weights by developing a learnable mapping F (•), i.e.,EQUATION)The recent studies that use the bipartite graph modeling [52, 54] initialize all sample node embeddings as all-one vectors and feature node embeddings as one-hot vectors, which have a value 1 in the positions representing their respective features and 0's elsewhere. We observe, however, that such an initialization does not effectively utilize the information from the masked data matrix, which leads to inferior imputation accuracy, as shall be demonstrated in Section 4.3. Thus, in M 3 -Impute, we propose to initialize each sample node embedding based on its associated (initial) feature embeddings instead of initializing them separately. While the feature embeddings are randomly initialized, the sample node embeddings are initialized in a way that reflects the embeddings of the features whose values are available in their corresponding samples.Let h 0 f be the initial embedding of feature f , which is a randomly initialized d-dimensional vector, and defineH 0 F = [h 0 f1 h 0 f2 . . . h 0 fm ] ∈ R d×m . Also, let d s ∈ R m be the s-th column vector of D ⊤ ,which is a vector of the feature values of sample s, and let m s ∈ R m be its corresponding mask vector, i.e., m s = col s (M ⊤ ), where col s (•) denotes the s-th column vector of the matrix. We then initialize the embedding h 0 s of each sample node s as follows:EQUATION)where 1 ∈ R m is an all-one vector, and ϕ(•) is an MLP. Note that the term d s + ϵ(1 -m s ) indicates a vector that consists of observable feature values of s and some small positive values ϵ in the places where the feature values are unavailable (masked out).To improve the accuracy of missing value imputation, we aim to fully exploit feature correlations which often appear in the datasets. While the feature correlations are naturally captured by GNNs, we observe that there is still room for improvement. We propose FCU as an integral component of M 3 -Impute to fully exploit the feature correlations.To impute the missing value of feature f for a given sample s, FCU begins by computing the feature 'context' vector of sample s in the embedding space that reflects the correlations between the target missing feature f and observed features. Let h f ∈ R d be the learned embedding vector of feature f from the GNN, and let H F be the d × m matrix that consists of all the learned feature embedding vectors. We first obtain dot-product similarities between feature f and all the features in the embedding space, i.e., H ⊤ F h f . We then mask out the similarity values with respect to nonobserved features in sample s. Here, instead of applying the mask vector m s of sample s directly, we use a learnable 'soft' mask vector, denoted by m ′ s , which is defined to be m′ s = σ 1 (m s ) ∈ R m ,where σ 1 (•) is an MLP with the GELU activation function [21] . In other words, we obtain featurewise similarities with respect to sample s, denoted by r f s , as follows:EQUATIONwhere σ 2 (•) denotes another MLP with the GELU activation function. FCU next obtains the Hadamard product between the learned embedding vector of sample s, h s , and the feature-wise similarities with respect to sample s, r f s , to learn their joint representations in a multiplicative manner. Specifically, FCU obtains the feature context vector of sample s, denoted by c f s , as follows:EQUATION)where σ 3 (•) is also an MLP with the GELU activation function. That is, FCU fuses the representation vector of s and the vector that has embedding similarity values between the target feature f and the available features in s through the effective use of the soft mask m ′ s . From (3) and (4), the operations of FCU can be written asc f s = FCU(h s , m s , H F ) = σ 3 ( h s ⊙ σ 2 ( (H ⊤ F h f ) ⊙ σ 1 (m s ))) .(5)To measure similarities between s and other samples, a common approach would be to use the dot product or cosine similarity between their embedding vectors. This approach, however, fails to take into account the observability or availability of each feature in a sample. It also does not capture the fact that different observed features are of different importance to the target feature to impute when it comes to measuring the similarities. We introduce SCU as another integral component of M 3 -Impute to compute the sample 'context' vector of sample s by incorporating the embedding vectors of its similar samples as well as different weights of observed features. SCU works based on the two novel masking schemes, which shall be explained shortly. Suppose we are to impute the missing value of feature f for a given sample s. SCU aims to leverage the information from the samples that are similar to s. As a first step to this end, we create a subset of samples P ⊂ S that are similar to s. Specifically, we randomly choose and put a sample into P with probability that is proportional to the cosine similarity between s and the sample. This operation is repeated without replacement until P reaches a given size.Mutual Sample Masking: Given a subset of samples P that include s, we first compute the pairwise similarities between s and other samples in the subset P. While they are computed in a similar way to FCU, we only consider the commonly observed features (or the common ones that have feature values) in both s and its peer p ∈ P \ {s}, to calculate their pairwise similarity in the sense that the missing value of feature f is inferred.Specifically, we compute the pairwise similarity between s and p ∈ P \{s}, which is denoted by sim(s, p | f ), as follows:EQUATIONwhere h s and h p are the learned embedding vectors of samples s and p from the GNN, respectively, and m s and m p are their respective mask vectors. Note that the multiplication in the RHS of ( 6) is the dot product.Irrelevant Feature Masking: After we obtain the pairwise similarities between s and other samples in P, it would be natural to consider a weighted sum of their corresponding embedding vectors, i.e., ∑ p∈P\{s} sim(s, p | f ) h p , in imputing the value of the target feature f . However, we observe that h p contains the information from the features whose values are available in p as well as possibly other features as it is learned via the so-called neighborhood aggregation mechanism that is central to GNNs, but some of the features may be irrelevant in inferring the value of feature f . Thus, instead of using {h p } directly, we introduce a d-dimensional mask vector r f p for h p , which is to mask out potentially irrelevant feature information in h p , when it comes to imputing the value of feature f . Specifically, it is defined byEQUATIONwhere m f is an m-dimensional one-hot vector that has a value 1 in the place of feature f and 0's elsewhere, [• ; •] denotes the vector concatenation operation, and σ 4 (•) is an MLP with the GELU activation function. Note that the rationale behind the design of r f p is to embed the information on the features whose values are present in p as well as the information on the target feature f to impute.The mask r f p is then applied to h p to obtain the masked embedding vector of p as follows:EQUATION)where σ 5 (•) is also an MLP with the GELU activation function. Once we have the masked embedding vectors of samples (excluding s) in P, we finally compute the sample context vector of sample s, denoted by z f s , which is a weighted sum of the masked embedding vectors with weights being the pairwise similarity values, i.e.,EQUATION)where σ 6 (•) is again an MLP with the GELU activation function. From ( 6)-( 9), the operations of SCU can be written asEQUATION)where H P = {h p , p ∈ P} and M P = {m p , p ∈ P}.Algorithm 1 Forward computation of M 3 -Impute to impute the value of feature f for sample s. For a given sample s, to impute the missing value of feature f , M 3 -Impute obtains its feature context vector c f s and sample context vector z f s through FCU and SCU, respectively, which are then used for imputation. Specifically, it is done by predicting the corresponding edge weight êsf as follows:EQUATION)where ϕ α (•) denotes an MLP with a non-linear activation function (i.e., ReLU for continuous values and softmax for discrete ones), and α is a learnable scalar parameter. This scalar parameter α is introduced to strike a balance between leveraging feature-wise correlation and sample-wise correlation. It is necessary because the quality of z f s relies on the quality of the samples chosen in P, so overly relying on z f s would backfire if their quality is not as desired. To address this problem, instead of employing a fixed weight α, we make α learnable and adaptive in determining the weights for c f s and z f s . Note that this kind of learnable parameter approach has been widely adopted in natural language processing [26, 34, 38, 46] and computer vision [8, 55, 56] , showing superior performance to its fixed counterpart. In M 3 -Impute, the scalar parameter α is learned based on the similarity values between s and its peer samples p ∈ P \ {s} as follows:EQUATION)where ∥ represents the concatenation operation, and ϕ γ (•) is an MLP with the activation function etc. Since the datasets are fully observed, we introduce missing values by applying a randomly generated mask to the data matrix. Specifically, as used in prior studies [23, 24] , we apply three masking generation schemes, namely missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR). 1 We use MCAR with a missing ratio of 30%, unless otherwise specified. We follow the preprocessing steps adopted in [52, 54] to scale feature values to [0, 1] with a MinMax scaler [25] . Due to the space limit, we below present the results of eight datasets that are used in Grape [52] and report the other results in Appendix.γ(x) = 1 -1 / e |x|Baseline models: M 3 -Impute is compared against popular and state-of-the-art imputation methods, including statistical methods, deep generative methods, and graph-based methods listed as follows:MEAN: It imputes the missing value êsf as the mean of observed values in feature f from all the samples. K-nearest neighbors (kNN) [43] : It imputes the missing value êsf using the kNNs that have observed values in feature f with weights that are based on the Euclidean distance to sample s. Multivariate imputation by chained equations (Mice) [45] : This method runs multiple regressions where each missing value is modeled upon the observed non-missing values. Iterative values by predicting the weights of the missing edges, each of which is done based on the inner product between the embeddings of its corresponding sample and feature nodes. HyperImpute [23] :HyperImpute is a framework that conducts an extensive search among a set of imputation methods, selecting the optimal imputation method with fine-tuned parameters for each feature in the dataset.Model configurations: Parameters of M 3 -Impute are updated by the Adam optimizer with a learning rate of 0.001 for 40,000 epochs. For graph representation learning, we use a variant of Graph-SAGE [17] , which not only learns node embeddings but also edge embeddings via the neighborhood aggregation mechanism, as similarly used in [52] . We consider its three-layer GNN model. We employ mean-pooling as the aggregation function and use ReLU as the activation function for the GNN layers. We set the embedding dimension d to 128. It is known that randomly dropping out a subset of observable edges during training improves the model's generalization ability. We also leverage the observation and randomly drop 50% of observable edges during training. For each experiment, we conduct five runs with different random seeds and report the average results.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [NA] Justification: This paper does not present theoretical results. We do not assume that the data is missing under MCAR, MAR, or MNAR conditions for M 3 -Impute to be effective.Instead, M 3 -Impute demonstrates robust performance across all three settings.Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.• All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.• Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.• Theorems and Lemmas that the proof relies upon should be properly referenced. 
overall performance: 
 We first compare the feature imputation performance of M 3 -Impute with popular and state-of-theart imputation methods. As shown in Table 1 , M 3 -Impute achieves the lowest imputation MAE for six out of the eight examined datasets and the second-best MAE scores in the other two, which validates the effectiveness of M 3 -Impute. For KIN8NM dataset, M 3 -Impute underperforms Miracle.It is mainly because each feature in KIN8NM is independent of the others, so none of the observed features can help impute missing feature values. For NAVAL dataset, the only model that outperforms M 3 -Impute is HyperImpute [23] . In the NAVAL dataset, nearly every feature exhibits a strong linear correlation with the other features, i.e., every pair of features has correlation coefficient close to one. This allows HyperImpute to readily select a linear model from its model pool for each feature to impute. Nonetheless, M 3 -Impute exhibits overall superior performance to the baselines as it can be well adapted to each dataset that possesses different amounts of correlations over features and samples. In other words, M 3 -Impute benefits from explicitly incorporating feature-wise and sample-wise correlations together with our carefully designed mask schemes. Furthermore, we evaluate the performance of M 3 -Impute under MAR and MNAR settings. We observe that M 3 -Impute consistently outperforms all the baselines under all datasets and achieves a larger margin in the improvement compared to the case with MCAR setting. This implies that M 3 -Impute is also effective in handling different patterns of missing values in the input data. Comprehensive results are provided in Appendix. To study the effectiveness of three integral components of M 3 -Impute, we consider three variants of M 3 -Impute, each with a subset of the components, namely initialization only (Init Only), initialization + FCU (Init + FCU), and initialization + SCU (Init + SCU). The performance of these variants are evaluated against the top-performing imputation baselines such as Grape and HyperImpute. As shown in Table 2 , the three variants derived from M 3 -Impute achieve lower MAE values than both baselines in most datasets, demonstrating the effectiveness of our novel components in M 3 -Impute.Specifically, for initialization only, the key difference between M 3 -Impute and Grape lies in our refined initialization process of feature-node and sample-node embeddings. The reduced MAE values observed by the Init Only variant demonstrate that our proposed initialization process is more effective in utilizing information between samples and their associated features, including missing ones, as compared to the basic initialization used in [52] . In addition, we observe that when FCU or SCU is incorporated, MAE values are further reduced for most datasets. This validates that explicitly modeling feature-wise or sample-wise correlations through our novel masking schemes can improve imputation accuracy. When all the three components are combined together as in M 3 -Impute, they work synergistically to lower MAE values, validating the efficacy of explicit consideration of both sample-wise and feature-wise correlations (in addition to the refined initialization process) for missing data imputation.Missing ratio: In practice, datasets may possess different missing ratios. To validate the model's robustness under such circumstances, we evaluate the performance of M 3 -Impute and other baseline models with varying missing ratios, i.e., 0.1, 0.3, 0.5, and 0.7. Figure 3 shows their performance. We use the MAE of HyperImpute (HI) as the reference performance and offset the performance of each model by MAE x -MAE HI , where x represents the considered model. For clarity, we here only report the results of four top-performing models. As shown in Figure 3 , M 3 -Impute outperforms other baseline models for almost all the cases, especially under YACHT, CONCRETE, ENERGY, and HOUSING datasets. It is worth noting that modeling feature correlations in these datasets is particularly challenging due to the presence of considerable amounts of weakly correlated features, along with a few strongly correlated ones. Nonetheless, FCU and SCU in M 3 -Impute were able to better capture such correlations with our efficient masking schemes, thereby resulting in a large improvement in imputation accuracy. In addition, for KIN8NM dataset, M 3 -Impute ties with the second-best model, Grape. As mentioned in Section 4.2, each feature in KIN8NM is independent of the others, so none of the observed features can help impute missing feature values. For NAVAL dataset, where each feature strongly correlates with the others, M 3 -Impute surpasses Grape but falls short of HyperImpute, due to the same reason as discussed above. Overall, M 3 -Impute is robust to various missing ratios. Comprehensive results for all the baseline models can be found in Appendix.Sampling strategy in SCU: While SCU uses a sampling strategy based on pairwise cosine similarities to construct a subset of samples P, the simplest sampling strategy to build P would be to choose samples uniformly at random without replacement (M 3 -Uniform). Intuitively, this approach cannot identify similar peer samples accurately and thus would lead to inferior performance. Nonetheless, as shown in Size of P in SCU: Intuitively, neither an excessively small nor overly large size of the sample subset P is optimal. Too few peer samples leave SCU with insufficient information to learn sample-wise correlations, while too many peer samples may include quite a few dissimilar ones, which may introduce significant noise to the computation of SCU and thus degrade the performance. Table 3 shows the performance of M 3 -Impute with varying numbers of peer samples. In general, the trends agree with our intuition. Although the optimal size varies across different datasets, we observe that having the number of peer samples to be 5 to 10 achieves the overall best imputation accuracy. Initialization parameter ϵ: We also evaluate whether a non-zero value of ϵ in the initialization process of M 3 -Impute indeed lead to an improvement in imputation accuracy. As shown in Table 3 , for YACHT and WINE datasets, the introduction of a non-zero value of ϵ results in lower MAE scores.Another insight that we have from Table 3 is that ϵ should not be set too large, as a large value of ϵ might impose incorrect weights to the features with missing values. We observe that it is an overall good choice to set ϵ to 1×10 -5 or 1×10 -4 .Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?Answer: [Yes] Justification: In Section 3, we explain the computational pipeline of the proposed model in detail and provide a pseudo-code to better outline the methodology. The experimental setup is comprehensively described in Section 4.1. In addition, supplementary materials include our complete codebase to reproduce the results presented in this paper, including the model implementation, training and testing pipeline, configuration files, and execution scripts.Guidelines:• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.• If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.• Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.• While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 
conclusion: 
 We have presented M 3 -Impute, a mask-guided representation learning for missing data imputation. In this section, we discuss further experimental details. We first give an overview of the dataset details in Section A.1, followed by the implementation of different missing types and present corresponding imputation performance under MAR and MNAR settings (Section A.2). We then provide the comprehensive results of the robustness experiments (Section A.3). Finally, we extend our evaluation of M 3 -Impute to seven additional datasets (Section A.4) and elaborate on the computational resources in Section A.5.Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] Justification: In the supplementary material, we provide the complete code for our model, including the scripts for experiments and evaluations, as well as the execution scripts used in the experiments. We have also included the data preprocessed by us, along with the download links for publicly available datasets. We will release the formatted codebase on GitHub following the conclusion of the anonymity period.Guidelines:• The answer NA means that paper does not include experiments requiring code.• Please see the NeurIPS code and data submission guidelines (https://nips.cc/ public/guides/CodeSubmissionPolicy) for more details.• While we encourage the release of code and data, we understand that this might not be possible, so No is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).• The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //nips.cc/public/guides/CodeSubmissionPolicy) for more details.• The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.• The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.• At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).• Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: Experimental setup is detailed in Section 4.1 and Appendix A.2. We also explore the hyperparameters utilized in M 3 -Impute. Results are presented in Table 3 .Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.• The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: We conduct all the experiments over five runs and report the mean MAE scores, along with the standard deviations.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)• The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean.• It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.• For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).• If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the com- Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.• The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.• The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [NA] Justification: The method proposed in this work is only applicable for missing value imputation and is unlikely to have a negative social impact.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.• The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.• If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 • Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.• The paper should discuss whether and how consent was obtained from people whose asset is used.• At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.• According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.• We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.• For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
: 
 in handling mixed data types and struggles to scale up to large datasets. Learning-based imputation methods [10, 24, 27, 29, 33, 42, 50, 51, 53] , instead, take advantage of the strong expressiveness and scalability of machine/deep learning algorithms to model data correlations. However, most of them are still built upon the raw tabular data structure as is, which greatly restricts them from jointly modeling the feature-wise and sample-wise correlations. In light of this, graph-based methods [52, 54] have been proposed to model the raw data as a bipartite graph, with samples and features being two different types of nodes. A sample node and a feature node are connected if the feature value is observed in that sample. The missing values are then predicted as the inner product between the embeddings of the corresponding sample and feature nodes. However, this simple prediction does not consider the specific missingness information as mentioned above. For instance, the target feature to impute may have different correlations with features in the samples which have different kinds of missingness; however, the same feature-node embedding is still used for their imputation.A similar issue also arises for sample-node embeddings.In this work, we address these problems by proposing M 3 -Impute, a mask-guided representation learning method for missing value imputation. The key idea behind M 3 -Impute is to explicitly utilize the data-missingness information as model input with our proposed novel masking schemes so that it can accurately learn feature-wise and sample-wise correlations in the presence of different kinds of data missingness. M 3 -Impute first builds a bipartite graph from the data as used in [52] .In the embedding initialization for graph representation learning, however, we not only use the the relationships between samples and their associated features but also the missingness information so as to initialize the embeddings of samples and features jointly and effectively. We then propose novel feature correlation unit (FCU) and sample correlation unit (SCU) in M 3 -Impute to explicitly take feature-wise and sample-wise correlations into account for imputation. FCU learns the correlations between the target missing feature and observed features within each sample, which are then further updated via a soft mask on the sample missingness information. SCU then computes the samplewise correlations with another soft mask on the missingness information for each pair of samples that have values to impute. We then integrate the output embeddings of FCU and SCU to estimate the missing values in a dataset. We carry out extensive experiments on 15 open datasets. The results show that M 3 -Impute outperforms state-of-the-art methods in 13 of the 15 datasets on average under three different settings of missing value patterns, achieving up to 11.47% improvement in MAE compared to the second-best method. 
 
 Appendix: Table 4 presents the statistics of the eight UCI datasets [11] used throughout Section 4. Figure 4 illustrates the Pearson correlation coefficients among the features. In the Kin8nm dataset, all features are linearly independent, whereas the Naval dataset exhibits strong correlations among its features.Under the MCAR setting, M 3 -Impute performs comparably to the baseline imputation methods on these two datasets (shown in Table 1 ). However, in real-world scenarios, features are not always entirely independent or strongly correlated. In the other six datasets, we observe a mix of weakly correlated features along with a few that are strongly correlated. In these cases, M 3 -Impute consistently outperforms all baseline methods.We adopt the same procedure outlined in [52, 54] to generate missing values under different settings.• MCAR: A n × m matrix is sampled from a uniform distribution. Positions with values no greater than the ratio of missingness are viewed as missing and the remaining positions are observable.• MAR: First, a subset of features is randomly selected to be fully observed. Then, these remaining features have values removed according to a logistic model with random weights, using the fully observed feature values as input. The desired rate of missingness is achieved by adjusting the bias term.• MNAR: This is done by first apply the MAR mechanism above. Then, the remaining feature values are masked out by the MCAR mechanism. In addition to the results for MCAR setting presented in Table 4 .2, Table 5 and Table 6 present the MAE scores under MAR and MNAR settings, respectively. M 3 -Impute consistently outperforms all baseline methods in both scenarios.Table 8 presents the performance of various imputation methods across different ratios of missingness. M 3 -Impute achieves the lowest MAE scores in most cases and the second-best MAE scores in the remaining ones. In this experiment, we further evaluate M 3 -Impute on seven datasets: Airfoil [3] , Blood [49] , Wine-White [7] , Ionosphere [39] , Breast Cancer [47] , Iris [13] , and Diabetes [12] . An overview of dataset details is provided in Table 7 , and feature correlations are illustrated in Figure 5 . We simulate missingness in data under MCAR, MAR, and MNAR conditions, each with a missing ratio of 0.3.Results are demonstrated in Table 9 . Across all three types of missingness, M 3 -Impute achieves five best and two second-best MAE scores on average. Justification: In the abstract and introduction sections, we clearly define the scope of this paper, focusing on missing value imputation. We propose M 3 -Impute, a mask-guided imputation method designed to compute feature-wise and sample-wise correlations based on missing data patterns. A concise summary of the experimental results is provided at the end of both sections.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper.• The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.• The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.• It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.Justification: Our paper adheres to the NeurIPS Code of Ethics in every respect. 1. We ensured fair wages for all human participants involved in our study, abiding by regional minimum hourly rates. 2. Our research methodology adhered to institutional protocols for human subjects and data privacy. 3. We obtained informed consent from all participants and minimized exposure of personally identifiable information. 4. The datasets used are publicly available and have not been deprecated, with all copyrights respected. 5. We have transparently communicated the societal impact of our research, considering potential misuse and its effects on discrimination, surveillance, and environmental impact. 6. We have also reflected on the biases in our models and datasets and taken steps to mitigate them.7. Our data and models are documented and released with appropriate licenses, and we've employed secure data storage and distribution practices. 8. We ensured legal compliance and provided all necessary elements for the reproducibility of our research.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.• The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).More details about the datasets and mask generation for missing values can be found in Appendix. 


Decision: The authors propose a new imputation algorithm based on deep learning. The starting point is to see the incomplete data matrix as a bipartite graph, with "feature" nodes and "samples" nodes, than can then be handled by a graph neural network to produce imputations.

All reviewers agreed that the method is clever and interesting, in particular in its ability to process the missingness pattern. Some argued that it was a bit overly complex, and I tend to agree (in particular, I would suggest the authors to write a more detailed caption for Fig.1, that would explain the pipeline concisely). 

Most of the reviewers's concerns were answered. In my opinion, there is a major point that remains. Two reviewers asked for more details about hyper-parameter optimisation of competing methods, and the authors's response was vague and unconvincing:

> For baselines, we followed the commonly used hyperparameter settings from the previous studies, such as GRAPE, including edge drop-out ratio during training, dropout rate, learning rate, and the number of GNN layers.

It seems clear that hyperparameter optimisation was poorly done for some of the methods. Indeed, it seems virtually impossible that methods that contain the mean imputation as a special case (like SVD or MIWAE) can do worse than the mean imputation when hyperparameters are properly optimised (SVD is often worse than the mean in the experiments, and MIWAE is almost always worse). Moreover, hyperparameter optimisation was also very problematic for MIRACLE: e.g. in Table 1, for 3 datasets, the MAE of MIRACLE is more than *ten times* the MAE of the mean imputation. For future iterations, I strongly encourage the authors to revisit their choices of hyperparameters to produce fairer comparisons. In particular, for methods based on deep nets (like MIWAE or MIRACLE), I would suggest carefully looking at how the data were normalised. Methods that produce results worse than mean imputation should be investigated and discussed, since this means exceptionally bad performance.

Another point that should be mentioned is that most of these methods were designed to minimise the MSE, not the MAE. The authors also conducted experiments on the MSE during the rebuttal, but experiments with the MAE should clearly indicate which method was designed to minimise the MSE, and which method was designed to minimise the MSE.

After an extensive discussion with one of the reviewers who recommended acceptance and the senior AC, we agreed to reject the paper on these grounds. In future iterations, I encourage the authors to focus on explaining the method in a clearer fashion, and making fairer comparisons, with clear choices of hyper-parameters and network architectures.

Reviews: review: paper_summary: paper_summary: This paper presents a novel imputation method, based on a bipartite graph constructed from the data and the missing-data patterns, and two components which allow to measure similarities between the features and samples.

The method shows very good results in terms of MAE on several datasets for MCAR, MAR and MNAR data.
strength_weakness: strength_weakness: - Although well presented, the method is complicated to understand. 

- The methods uses 8 MLPs and one GNN. The authors discuss in Appendix the computational resources, but do not compare other methods on this point.
questions: questions: General remarks: 
- How does this methodology relate to the simple concatenation of the mask to the data matrix, and the execution of an imputation method on the augmented matrix? (see Josse, Julie, et al. "On the consistency of supervised learning with missing values.")
- Is M3-Impute supposed to work well for MNAR? This should be discussed more in details, as the authors claim that the method utilizes the data-missingness information. A remark: there exists for MIWAE an extension specifically designed for MNAR data, called not-MIWAE. Ipsen, Niels Bruun, Pierre-Alexandre Mattei, and Jes Frellsen. "not-MIWAE: Deep generative modelling with missing not at random data."
It can be interesting to have a comparison of M3-Impute with this one in a final version. 

Algorithm: 
- Figure 1: maybe the authors should add numbers in the graphics to refer to them when describing the method in the text (especially in 3.1)
- In Algorithm 1, one of the input is the GNN model. How are hyperparameters of the GNN managed in practice? 

Numerical experiments: 
- In the final version, the authors should add a comparison with the missForest algorithm, which is one of the most widely used imputation methods. 
- for other methods, such as MIWAE, which hyperparameters did the authors choose? 
- in the MAR setting, how many features are selected to be observed? Did the authors take the best subset for the results?

Minor comments:
- l.15 mechanisms instead of patterns
- l.56 "the the"
- l.137 notation col_s: harmonise d and m
- l.548 "these remaining" <- "the remaining"
- l.291 mechanisms instead of patterns
limitations: limitations: Yes
paper_summary: paper_summary: This is a novel approach for imputing missing data using mask-guided representation learning. The main contributions include the development of an imputation model that leverages both feature and sample correlations. This model improves imputation accuracy and robustness compared to existing methods. The paper also provides comprehensive experiments and ablation studies to validate the effectiveness of the proposed approach across various datasets and missing data scenarios.
strength_weakness: strength_weakness: - Computational complexity 

## Minor Points

l4: "Existing imputation methods, however, fall short of considering the ‘missingness’ information in the data  during initialization and modeling the entangled feature and sample correlations  explicitly during the learning process,"
-> This is not true. Many existing methods consider missingness patterns.

The distinction between "statistical" and "learning based" methods seems off. Certainly most learning based methods are statistical, and vice versa.

l. 38 "struggles" -> struggle
questions: questions: - (How) do you do HPO for competing methods?
limitations: limitations: - Gives little insight into why it works better on some datasets than others.
- Would be interesting to understand better how robust results are under systematic changes in datasets, e.g., different types of missingness.
paper_summary: paper_summary: This paper proposes M^3-Impute, a missing value imputation method that utilizes GNNs to learn embeddings of samples and features. By incorporating feature correlation unit and sample correlation unit, M^3-Impute effectively captures correlations between features and samples for accurate imputation.
strength_weakness: strength_weakness: W1. SCU takes into account the pairwise similarity of \mathcal{P} during its construction, which subsequently determines the scalar parameter \alpha during imputation. The initialization of \mathcal{P} seems to directly impact the model's performance and remains unchanged once set. It would be beneficial for the authors to discuss this aspect and, if possible, provide some experimental evidence to support their approach.

W2. In Section 4.4, the paper mentions different sampling strategies for SCU and uses a new strategy in the ablation study (Table 2), which is different from the strategy mentioned in Section 3.4. The authors claim that this strategy leads to inferior performance compared to previous strategies, thereby highlighting the superiority of the ablation study results. This lacks experimental evidence and results in inconsistency in the experimental setup.

W3. As a crucial parameter, the size of \mathcal{P} directly affects the construction of SCU. Table 3 presents experimental results with different sizes, but the differences are not significant, which is somewhat counterintuitive. Although the authors discuss the experimental results in Section 4.4, the performance fluctuation of 0.01 to 0.02 does not clearly reflect the "decrease then increase" trend mentioned by the authors. Exploring larger peer values and providing more detailed analysis and guidance on parameter selection might be beneficial.

W4. The authors emphasize the importance of specific missingness information throughout the paper. Intuitively, different types of missing data (MAR, MNAR, MCAR) might offer varying types of missingness information, potentially impacting the model's performance. While the paper experiments with data of different missingness types, a more thorough discussion of the results could enhance the motivation and clarity of the paper.

W5. In Section 4.3, the caption of Table 2 references a concept, the uniform sampling strategy, which is introduced for the first time in a later subsection. The authors might consider adjusting the structure of the paper for better clarity.

W6. In Figure 3, in the subfigure with the missing ratio of 0.7, two bars exceed the upper boundary and need adjustment.

W7. It would be helpful to add independent labels to all subfigures, such as (a), (b), etc., to facilitate referencing.
questions: questions: Q1. In Section 3.4, the authors mention that \mathcal{P} is constructed by randomly choosing with a certain probability. Why not select directly based on the degree of similarity rather than introducing randomness? How might these different approaches impact the model's performance?

Q2. To what extent does applying different GNN models to learn embeddings impact the performance of the model proposed in this paper?
limitations: limitations: Yes.
paper_summary: paper_summary: The paper proposed a new imputations method called M3-impute. M3-impute follows the basic structure of some recent imputation methods: a undirected bipartite graph is constructed with nodes for features and samples, where edge weights correspond to observed data at the given feature-sample pair. Previous approaches use Graph Neural Networks (GNNs) to impute missing values via edge weight prediction. M3-impute improves these approaches by adding two new components on top of an initial GNN to model feature-wise and sample-wise correlations respectively. Empirical results show that M3-impute achieves competitive performance in terms of MAE for imputation across several tabular datasets.
strength_weakness: strength_weakness: - The paper does not support categorical features. This is a big weakness compared to other imputation methods that can handle categorical features such as iterative approaches like hyperimpute. 
- The paper does not discuss the impact of missing value imputation on downstream tasks. Imputation is usually a preprocessing step, and thus assessing the impact on possible downstream tasks is paramount. For example, in supervised learning, some recent evidence suggests that mean/zero imputation is as good as more complex imputations [1, 2]. 

[1] Le Morvan, Marine, et al. "What’s a good imputation to predict with missing values?." Advances in Neural Information Processing Systems 34 (2021): 11530-11540.
[2] Van Ness, Mike, and Madeleine Udell. "In defense of zero imputation for tabular deep learning." NeurIPS 2023 Second Table Representation Learning Workshop. 2023.
questions: questions: - What is the current approach for handling categorical features? Do none of the datasets in the experiments have any categorical features, or are these feature simply being one-hot encoded?
- How does the runtime of M3-impute compare to the other methods? I know this is briefly discussed in appendix 5, but more details runtimes would be appreciated. 
- Why are standard deviations not included for Table 1?
limitations: limitations: See weaknesses. In particular, not handling categorical features is not mentioned in the paper anywhere as a limitation of the method.
paper_summary: paper_summary: This paper addresses the challenge of missing values in data analysis and machine learning by proposing M3-Impute, a novel imputation method. Traditional imputation techniques often neglect 'missingness' information and fail to explicitly model feature and sample correlations, leading to suboptimal results. M3-Impute innovatively incorporates missingness information and correlations through advanced masking schemes. It represents data as a bipartite graph and utilizes a graph neural network with a refined initialization process to learn node embeddings. These embeddings are further optimized using feature correlation and sample correlation units, which explicitly consider the correlations during imputation. The method's effectiveness is demonstrated through experiments on 15 benchmark datasets with three different missing patterns.
strength_weakness: strength_weakness: M3-Impute demonstrates strong performance across many datasets but shows limitations in handling datasets with highly independent features or strong linear correlations, such as the KIN8NM and NAVAL datasets.

The model's robustness in general scenarios may degrade when confronted with datasets containing extreme correlation structures.

Future improvements could concentrate on enhancing M3-Impute's adaptability to these challenging cases to broaden its applicability and robustness across diverse datasets.
questions: questions: In ablation study (4.3), Could you provide more details on the refined initialization process of feature-node and sample-node embeddings? How does it differ specifically from the initialization used in Grape?


Can you provide more insights into the computational complexity and runtime performance of M3-Impute compared to the baselines, especially for large datasets?

In missing ratio test, why does M3-Impute perform similarly to Grape on the KIN8NM dataset, and what characteristics of KIN8NM contribute to this result?
limitations: limitations: Although M3-Impute outperforms other baselines on most datasets, two cases (KIN8NM, NAVAL dataset) highlight its limitations in handling datasets with either highly independent features or strongly linear correlations. This suggests that while M3-Impute is robust in general scenarios, its performance may degrade in datasets with extreme correlation structures.
paper_summary: paper_summary: This study proposes a missing value imputation method. The proposed method tackles the missing value imputation problem as a link prediction task on the bipartite graph. It represents a data matrix with missing values as a bipartite graph, then uses a graph neural network on the bipartite graph to learn the embeddings of samples and features. Next, a feature correlation unit and a sample correlation unit are employed to obtain feature-wise and sample-wise correlations, which are then fed into an MLP to obtain imputed values.
strength_weakness: strength_weakness: - The size of the bipartite graph may drastically increase with data size and dimensionality.
- No investigation on various missingness scenarios and missing rates.
questions: questions: - Computational and space complexity analysis is needed. Can the proposed method be used for a very large training dataset with high dimensionality?
- How does the proposed method perform under lower and higher missing rates?
- The inference phase is unclear. Please elaborate on how the proposed method makes imputations for a query instance containing missing values?
- While the authors reviewed other recent methods that leveraged graph neural networks ([40], [52], [54]), only the method in [52] was compared in the experiments. Why were the methods in [40] and [54] not compared?
limitations: limitations: -
paper_summary: paper_summary: The paper introduces M3-Impute, a mask-guided representation learning method for missing value imputation. The core idea of M3-Impute is to leverage missingness information as an explicit input to the model through masking schemes. This approach allows M3-Impute to effectively learn both feature-wise and sample-wise correlations, accommodating various types of data missingness. The model employs a variant of GraphSAGE for graph representation learning, incorporating edge embeddings via neighborhood aggregation. It outperforms traditional tabular data models in various benchmark datasets.
strength_weakness: strength_weakness: 1. The paper evaluates the sensitivity of the M3-Impute model to the initialization parameter ϵ (Table 3), demonstrating that a non-zero value of ϵ improves imputation accuracy. However, the lack of detailed sensitivity analysis for other critical hyperparameters, such as the learning rate, batch size, number of GNN layers, and the dropout rate, represents a weakness.

2. The paper also has notable limitations in its contextualization relative to prior work. While it effectively presents M3-Impute and compares it against several baseline models, it lacks a deeper analysis of how these baseline models have evolved and the specific innovations they have introduced over time. For instance, the paper mentions GRAPE and IGRM as key prior graph-based imputation methods but does not adequately explore their strengths and weaknesses or how M3-Impute directly addresses the limitations of these methods. This omission makes it challenging to understand the novelty and improvements offered by M3-Impute.

3. Relying solely on MAE to evaluate the performance of imputation models has several limitations. MAE measures the average magnitude of errors but does not account for the variance or distribution of those errors, making it insensitive to outliers and providing no insight into model bias. This can result in an incomplete understanding of a model's performance, particularly in contexts where large errors or systematic biases are important considerations. To address these limitations, incorporating RMSE alongside MAE would be beneficial. RMSE penalizes larger errors more heavily, offering additional insight into the presence and impact of significant errors in the model's predictions.
questions: questions: - The statement that most learning-based methods are``built upon the raw tabular data structure as is, which greatly restricts them from jointly modeling the feature-wise and sample-wise correlations” (line 41) is not entirely accurate for two prominent tabular generative models, MIDA and GAIN. MIDA transforms raw tabular data into a higher-dimensional space through its encoder-decoder architecture. This transformation allows MIDA to capture more complex, nonlinear relationships that are not immediately apparent in the raw data. The adversarial process of GAIN allows it to model the joint distribution of the data, thus capturing complex correlations between features and samples. 

      - Gondara, L., Wang, K. (2018). MIDA: Multiple Imputation Using Denoising Autoencoders. In: Phung, D., Tseng, V., Webb, G., Ho, B., Ganji, M., Rashidi, L. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2018. Lecture Notes in Computer Science(), vol 10939. Springer, Cham. https://doi.org/10.1007/978-3-319-93040-4_21

      - Wang, Zhenhua, et al. "Are deep learning models superior for missing data imputation in surveys? Evidence from an empirical comparison." Survey Methodology 48 (2022): 375-399.

- When discussing statistical methods (line 70), the authors should mention that FCS approaches such as MICE are flexible in imputing different types of variables. 

- Related to the previous point, MICE is generally considered a statistical method rather than a learning-based method, although a learning algorithm such as CART can be used as the imputer. See the paper by Wang et. al. (2022).

- In Sec. 4.2, it should be noted that the M3-Impute model tends to perform slightly better under MAR and MNAR settings for most datasets, indicating its effectiveness in handling missingness that depends on the observed data.

- How can interpretability techniques be incorporated into M3-Impute to help users understand the imputation decisions?
limitations: limitations: The authors should consider summarizing the limitations of their method in the conclusion to provide a comprehensive overview of their work. Specifically, in Section 4.2, the authors discuss the cases of MAE degradation for the Kin8nm and Naval datasets. They attribute this to the independence of features in Kin8nm, which prevents observed features from aiding in the imputation of missing values, and the strong linear correlations between nearly all features in the Naval dataset. Summarizing these points in the conclusion would give readers a clear understanding of the method's limitations and the contexts in which it performs best.

Comments: title: Thanks for the comments.
Thanks for the detailed response! My rating remains.
comment: We would like to thank all the reviewers again for their invaluable time and constructive comments. We will incorporate all the corresponding updates into our final manuscript. Thank you.
.
The authors have addressed the lack of empirical validation by providing additional sensitivity analyses for hyperparameters (Tables 5-7 in the rebuttal PDF) and RMSE results (Table 1 in the rebuttal) as requested.

They acknowledge the need to add a limitations section in the conclusion and summarize the cases where their method underperforms (KIN8NM and NAVAL datasets), and commit to adding this in the final manuscript.

The broader empirical validation and commitment to adding a limitations section will strengthen the revised paper.
Thank you for the response. My concerns have been addressed, and I will raise my score to 5.
I've raised my score to a 6 (weak accept) as the authors have addressed several of the concerns in my review.