M$^3$-Impute: Mask-guided Representation Learning for Missing Value Imputation

## Abstract

Abstract Missing values are a common problem that poses significant challenges to data analysis and machine learning. This problem necessitates the development of an effective imputation method to fill in the missing values accurately, thereby enhancing the overall quality and utility of the datasets. Existing imputation methods, however, fall short of explicitly considering the ‘missingness’ information in the data during the embedding initialization stage and modeling the entangled feature and sample correlations during the learning process, thus leading to inferior performance. We propose M 3 -Impute, which aims to explicitly leverage the missingness information and such correlations with novel masking schemes. M 3 -Impute first models the data as a bipartite graph and uses a graph neural network to learn node embeddings, where the refined embedding initialization process directly incorporates the missingness information. They are then optimized through M 3 -Impute’s novel feature correlation unit ( FCU ) and sample correlation unit ( SCU ) that effectively captures feature and sample correlations for imputation. Experiment results on 25 benchmark datasets under three different missingness settings show the effectiveness of M 3 -Impute by achieving 20 best and 4 second-best MAE scores on average.

## 1 Introduction

Missing values in a dataset are a pervasive issue in real-world data analysis. They arise for various reasons, ranging from the limitations of data collection methods to errors during data transmission and storage. Since many data analysis algorithms cannot directly handle missing values, the most common way to deal with them is to discard the corresponding samples or features with missing values, which would compromise the quality of data analysis. To tackle this problem, missing value imputation algorithms have been proposed to preserve all samples and features by imputing missing values with estimated ones based on the observed values in the dataset, so that the dataset can be analyzed as a complete one without losing any information.

The imputation of missing values usually requires modeling of correlations between different features and samples. Feature-wise correlations help predict missing values from other observed features in the same sample, while sample-wise correlations help predict them in one sample from other similar samples. It is thus important to jointly model the feature-wise and sample-wise correlations in the dataset. In addition, the prediction of missing values also largely depends on the ‘missingness’ of the data, i.e., whether a certain feature value is observed or not in the dataset. Specifically, the missingness information directly determines which observed feature values can be used for imputation. For example, even if two samples are closely related, it may be less effective to use them for imputation if they have missing values in exactly the same features. It still remains a challenging problem how to jointly model feature-wise and sample-wise correlations with such data missingness.

Among existing methods for missing value imputation, traditional methods [, , , , , , ] extract data correlations with statistical models, which are generally not flexible in handling mixed data types and struggle to scale up to large datasets. Recent learning-based imputation methods [, , , , , , , , ], instead, take advantage of the strong expressiveness and scalability of machine/deep learning algorithms to model data correlations. However, most of them are still built upon the raw tabular data structure as is, which greatly restricts them from jointly modeling the feature-wise and sample-wise correlations. In light of this, graph-based methods [, ] have been proposed to model the raw data as a bipartite graph, with samples and features being two different types of nodes. A sample node and a feature node are connected if the feature value is observed in that sample. The missing values are then predicted as the inner product between the embeddings of the corresponding sample and feature nodes. However, this simple prediction does not explicitly consider the specific missingness information as mentioned above.

In this work, we address these problems by proposing M^3-Impute, a mask-guided representation learning method for missing value imputation. The key idea behind M^3-Impute is to explicitly utilize the data-missingness information as model input with our proposed novel masking schemes so that it can accurately learn feature-wise and sample-wise correlations in the presence of different kinds of data missingness. M^3-Impute first builds a bipartite graph from the data as used in []. In the embedding initialization for graph representation learning, however, we not only use the the relationships between samples and their associated features but also the missingness information so as to initialize the embeddings of samples and features jointly and effectively. We then propose novel feature correlation unit (FCU) and sample correlation unit (SCU) in M^3-Impute to explicitly take feature-wise and sample-wise correlations into account for imputation. FCU learns the correlations between the target missing feature and observed features within each sample, which are then further updated via a soft mask on the sample missingness information. SCU then computes the sample-wise correlations with another soft mask on the missingness information for each pair of samples that have values to impute. We then integrate the output embeddings of FCU and SCU to estimate the missing values in a dataset. We carry out extensive experiments on 25 open datasets. The results show that M^3-Impute outperforms state-of-the-art methods in 20 of the 25 datasets on average under three different settings of missing value patterns, achieving up to $22.22\%$ improvement in MAE compared to the second-best method. Our code is available at [https://github.com/Alleinx/M3-IMPUTE](https://github.com/Alleinx/M3-IMPUTE).

## 2 Related Work

Traditional methods: These imputation approaches include joint modeling with expectation-maximization (EM) [, , ], $k$-nearest neighbors (kNN) [, ], and matrix completion [, , , ]. However, joint modeling with EM and matrix completion often lack the flexibility to handle data with mixed modalities, while kNN faces scalability issues due to its high computational complexity. In contrast, M^3-Impute is scalable and adaptive to different data distributions.

Learning-based methods: Iterative imputation frameworks [, , , , , ], such as MICE [] and HyperImpute [], have been extensively studied. These iterative frameworks apply different imputation methods for each feature and iteratively estimate missing values until convergence. In addition, for deep neural network learners, both generative models [, , , , , ] and discriminative models [, , ] have also been proposed. However, these methods are built upon raw tabular data structures, which may fall short of capturing the complex correlations in features, samples, and their combination. In contrast, M^3-Impute is based on the bipartite graph modeling of the data, which is more suitable for learning the data correlations for imputation.

Graph neural network-based methods: GNN-based methods [, ] are proposed to address the drawbacks mentioned above due to their effectiveness in modeling complex relations between entities. Among them, GRAPE [] transforms tabular data into a bipartite graph where features are one type of nodes and samples are the other. A sample node is connected to a feature node only if the corresponding feature value is present. This transformation allows the imputation task to be framed as a link prediction problem, where the inner product of the learned node embeddings is computed as the predicted values. However, these methods do not explicitly encode the missingness information of different samples and features into the imputation process, which can impair their imputation accuracy. In contrast, M^3-Impute enables explicit modeling of missingness information through FCU and SCU as well as our novel initialization unit so that feature-wise and sample-wise correlations can be accurately captured in the imputation process.

## 3 M 3 -Impute

Figure: Figure 1: Overview of the M^3-Impute model. The tabular data with missing values is first modeled as a bipartite graph with our refined initialization unit, which incorporates the missingness information in node embedding initialization. The graph is then processed with a GNN to update node embeddings. After that, we apply our novel soft masking schemes on these node embeddings to further encode correlation and missingness information in the learning process, using our novel components of feature correlation unit (FCU) and sample correlation unit (SCU). Eventually, the missing value is predicted with an MLP on the weighted sum of the outputs from FCU and SCU.
Refer to caption: x1.png

### 3.1 Overview

We here provide an overview of M^3-Impute to impute the missing value of feature $f$ for a given sample $s$, as depicted in Figure 1. Initially, the data matrix with missing values is modeled as an undirected bipartite graph, and the missing value is imputed by predicting the edge weight $\hat{e}_{sf}$ of its corresponding missing edge (Section 3.2). M^3-Impute next employs a GNN model, such as GraphSAGE [], on the bipartite graph to learn the embeddings of samples and features. These embeddings, along with the known masks of the data matrix (used to indicate which feature values are available in each sample), are then input into our novel feature correlation unit (FCU) and sample correlation unit (SCU), which shall be explained in Section 3.3 and Section 3.4, to obtain feature-wise and sample-wise correlations, respectively. Finally, M^3-Impute takes the feature-wise and sample-wise correlations into a multi-layer perceptron (MLP) to predict the missing feature value $\hat{e}_{sf}$ (Section 3.5). The whole process, including the embedding generation, is trained in an end-to-end manner.

### 3.2 Initialization Unit

Let $\mathbf{A}\in\mathbb{R}^{n\times m}$ be an $n\times m$ matrix that consists of $n$ data samples and $m$ features, where $\mathbf{A}_{ij}$ denotes the $j$-th feature value of the $i$-th data sample. We introduce an $n\times m$ mask matrix $\mathbf{M}\in\{0,1\}^{n\times m}$ for $\mathbf{A}$ to indicate that the value of $\mathbf{A}_{ij}$ is observed when $\mathbf{M}_{ij}=1$. In other words, the goal of imputation here is to predict the missing feature values $\mathbf{A}_{ij}$ for $i$ and $j$ such that $\mathbf{M}_{ij}=0$. We define the masked data matrix $\mathbf{D}$ to be $\mathbf{D}=\mathbf{A}\odot\mathbf{M}$, where $\odot$ is the Hadamard product, i.e., the element-wise multiplication of two matrices.

As used in recent studies [], we model the masked data matrix $\mathbf{D}$ as a bipartite graph and tackle the missing value imputation problem as a link prediction task on the bipartite graph. Specifically, $\mathbf{D}$ is modeled as an undirected bipartite graph $\mathcal{G}=(\mathcal{S}\cup\mathcal{F},\mathcal{E})$, where $\mathcal{S}=\{s_{1},s_{2},\ldots,s_{n}\}$ is the set of ‘sample’ nodes and $\mathcal{F}=\{f_{1},f_{2},\ldots,f_{m}\}$ is the set of ‘feature’ nodes. Also, $\mathcal{E}$ is the set of edges that only exist between sample node $s$ and feature node $f$ when $\mathbf{D}_{sf}\neq 0$, and each edge $(s,f)\in\mathcal{E}$ is associated with edge weight $e_{sf}$, which is given by $e_{sf}=\mathbf{D}_{sf}$. Then, the missing value imputation problem becomes, for any missing entries in $\mathbf{D}$ (where $\mathbf{D}_{sf}$ = 0), to predict their corresponding edge weights by developing a learnable mapping $F(\cdot)$, i.e.,

$$ $\hat{e}_{sf}=F(\mathcal{G},(s,f)\not\in\mathcal{E}).$ (1) $$

The recent studies that use the bipartite graph modeling initialize all sample node embeddings as all-one vectors and feature node embeddings as one-hot vectors, which have a value 1 in the positions representing their respective features and 0’s elsewhere. We observe, however, that such an initialization does not effectively utilize the information from the masked data matrix, which leads to inferior imputation accuracy, as shall be demonstrated in Section 4.3. Thus, in M^3-Impute, we propose to initialize each sample node embedding based on its associated (initial) feature embeddings instead of initializing them separately. While the feature embeddings are randomly initialized, the sample node embeddings are initialized in a way that reflects the embeddings of the features whose values are available in their corresponding samples.

Let $\mathbf{h}_{f}^{0}$ be the initial embedding of feature $f$, which is a randomly initialized $d$-dimensional vector, and define $\mathbf{H}_{F}^{0}=[\mathbf{h}_{f_{1}}^{0}\mathbf{h}_{f_{2}}^{0}\ldots\mathbf{
h}_{f_{m}}^{0}]\in\mathds{R}^{d\times m}$. Also, let $\mathbf{d}_{s}\in\mathds{R}^{m}$ be the $s$-th column vector of $\mathbf{D}^{\top}$, which is a vector of the feature values of sample $s$, and let $\mathbf{m}_{s}\in\mathds{R}^{m}$ be its corresponding mask vector, i.e., $\mathbf{m}_{s}=\text{col}_{s}(\mathbf{M}^{\top})$, where $\text{col}_{s}(\cdot)$ denotes the $s$-th column vector of the matrix. We then initialize the embedding $\mathbf{h}_{s}^{0}$ of each sample node $s$ as follows:

$$ $\mathbf{h}_{s}^{0}=\phi\Big{(}\mathbf{H}_{F}^{0}\big{[}\mathbf{d}_{s}+\epsilon (\mathds{1}-\mathbf{m}_{s})\big{]}\Big{)},$ (2) $$

where $\mathds{1}\in\mathds{R}^{m}$ is an all-one vector, and $\phi(\cdot)$ is an MLP. Note that the term $\mathbf{d}_{s}+\epsilon(\mathds{1}-\mathbf{m}_{s})$ indicates a vector that consists of observable feature values of $s$ and some small positive values $\epsilon$ in the places where the feature values are unavailable (masked out).

### 3.3 Feature Correlation Unit

To improve the accuracy of missing value imputation, we aim to fully exploit feature correlations which often appear in the datasets. While the feature correlations are naturally captured by GNNs, we observe that there is still room for improvement. We propose FCU as an integral component of M^3-Impute to fully exploit the feature correlations.

To impute the missing value of feature $f$ for a given sample $s$, FCU begins by computing the feature ‘context’ vector of sample $s$ in the embedding space that reflects the correlations between the target missing feature $f$ and observed features. Let $\mathbf{h}_{f}\in\mathds{R}^{d}$ be the learned embedding vector of feature $f$ from the GNN, and let $\mathbf{H}_{F}$ be the $d\times m$ matrix that consists of all the learned feature embedding vectors. We first obtain dot-product similarities between feature $f$ and all the features in the embedding space, i.e., $\mathbf{H}_{F}^{\top}\mathbf{h}_{f}$. We then mask out the similarity values with respect to *non-observed* features in sample $s$. Here, instead of applying the mask vector $\mathbf{m}_{s}$ of sample $s$ directly, we use a learnable ‘soft’ mask vector, denoted by $\mathbf{m}^{\prime}_{s}$, which is defined to be $\mathbf{m}^{\prime}_{s}=\sigma_{1}(\mathbf{m}_{s})\in\mathds{R}^{m}$, where $\sigma_{1}(\cdot)$ is an MLP with the GELU activation function []. In other words, we obtain feature-wise similarities with respect to sample $s$, denoted by $\mathbf{r}_{s}^{f}$, as follows:

$$ $\mathbf{r}_{s}^{f}=\sigma_{2}\left((\mathbf{H}_{F}^{\top}\mathbf{h}_{f})\odot \mathbf{m}^{\prime}_{s}\right)\in\mathds{R}^{d},$ (3) $$

where $\sigma_{2}(\cdot)$ denotes another MLP with the GELU activation function. FCU next obtains the Hadamard product between the learned embedding vector of sample $s$, $\mathbf{h}_{s}$, and the feature-wise similarities with respect to sample $s$, $\mathbf{r}_{s}^{f}$, to learn their joint representations in a multiplicative manner. Specifically, FCU obtains the feature context vector of sample $s$, denoted by $\mathbf{c}_{s}^{f}$, as follows:

$$ $\mathbf{c}_{s}^{f}=\sigma_{3}\left(\mathbf{h}_{s}\odot\mathbf{r}_{s}^{f}\right )\in\mathds{R}^{d},$ (4) $$

where $\sigma_{3}(\cdot)$ is also an MLP with the GELU activation function. That is, FCU fuses the representation vector of $s$ and the vector that has embedding similarity values between the target feature $f$ and the available features in $s$ through the effective use of the soft mask $\mathbf{m}_{s}^{\prime}$. From (3) and (4), the operations of FCU can be written as

$$ $\mathbf{c}_{s}^{f}=\textbf{FCU}(\mathbf{h}_{s},\mathbf{m}_{s},\mathbf{H}_{F})= \sigma_{3}\left(\mathbf{h}_{s}\odot\sigma_{2}\left((\mathbf{H}_{F}^{\top} \mathbf{h}_{f})\odot\sigma_{1}(\mathbf{m}_{s})\right)\right).$ (5) $$

### 3.4 Sample Correlation Unit

To measure similarities between $s$ and other samples, a common approach would be to use the dot product or cosine similarity between their embedding vectors. This approach, however, fails to take into account the observability or availability of each feature in a sample. It also does not capture the fact that different observed features are of different importance to the target feature to impute when it comes to measuring the similarities. We introduce SCU as another integral component of M^3-Impute to compute the sample ‘context’ vector of sample $s$ by incorporating the embedding vectors of its similar samples as well as different weights of observed features. SCU works based on the two novel masking schemes, which shall be explained shortly.

Figure: Figure 2: SCU.
Refer to caption: x2.png

Suppose we are to impute the missing value of feature $f$ for a given sample $s$. SCU aims to leverage the information from the samples that are similar to $s$. As a first step to this end, we create a subset of samples $\mathcal{P}\!\subset\!\mathcal{S}$ that are similar to $s$. Specifically, we randomly choose and put a sample into $\mathcal{P}$ with probability that is proportional to the cosine similarity between $s$ and the sample. This operation is repeated without replacement until $\mathcal{P}$ reaches a given size.

Mutual Sample Masking: Given a subset of samples $\mathcal{P}$ that include $s$, we first compute the pairwise similarities between $s$ and other samples in the subset $\mathcal{P}$. While they are computed in a similar way to FCU, we only consider the commonly observed features (or the common ones that have feature values) in both $s$ and its peer $p\!\in\!\mathcal{P}\setminus\{s\}$, to calculate their pairwise similarity in the sense that the missing value of feature $f$ is inferred. Specifically, we compute the pairwise similarity between $s$ and $p\!\in\!\mathcal{P}\setminus\{s\}$, which is denoted by $\mbox{sim}(s,p\mid f)$, as follows:

$$ $\mbox{sim}(s,p~{}|~{}f)=\textbf{FCU}(\mathbf{h}_{s},\mathbf{m}_{p},\mathbf{H}_ {F})\cdot\textbf{FCU}(\mathbf{h}_{p},\mathbf{m}_{s},\mathbf{H}_{F})\in\mathds{ R},$ (6) $$

where $\mathbf{h}_{s}$ and $\mathbf{h}_{p}$ are the learned embedding vectors of samples $s$ and $p$ from the GNN, respectively, and $\mathbf{m}_{s}$ and $\mathbf{m}_{p}$ are their respective mask vectors. Note that the multiplication in the RHS of (6) is the dot product.

Irrelevant Feature Masking: After we obtain the pairwise similarities between $s$ and other samples in $\mathcal{P}$, it would be natural to consider a weighted sum of their corresponding embedding vectors, i.e., $\sum_{p\in\mathcal{P}\setminus\{s\}}\mbox{sim}(s,p~{}|~{}f)~{}\mathbf{h}_{p}$, in imputing the value of the target feature $f$. However, we observe that $\mathbf{h}_{p}$ contains the information from the features whose values are available in $p$ as well as possibly other features as it is learned via the so-called neighborhood aggregation mechanism that is central to GNNs, but some of the features may be irrelevant in inferring the value of feature $f$. Thus, instead of using $\{\mathbf{h}_{p}\}$ directly, we introduce a $d$-dimensional mask vector $\mathbf{r}_{p}^{f}$ for $\mathbf{h}_{p}$, which is to mask out potentially irrelevant feature information in $\mathbf{h}_{p}$, when it comes to imputing the value of feature $f$. Specifically, it is defined by

$$ $\mathbf{r}_{p}^{f}=\sigma_{4}\left([\mathbf{m}_{p};\overline{\mathbf{m}}_{f}] \right)\in\mathbb{R}^{d},$ (7) $$

where $\overline{\mathbf{m}}_{f}$ is an $m$-dimensional one-hot vector that has a value 1 in the place of feature $f$ and 0’s elsewhere, $[\cdot\,;\cdot]$ denotes the vector concatenation operation, and $\sigma_{4}(\cdot)$ is an MLP with the GELU activation function. Note that the rationale behind the design of $\mathbf{r}_{p}^{f}$ is to embed the information on the features whose values are present in $p$ as well as the information on the target feature $f$ to impute. The mask $\mathbf{r}_{p}^{f}$ is then applied to $\mathbf{h}_{p}$ to obtain the masked embedding vector of $p$ as follows:

$$ $\phi_{p}(\mathbf{h}_{p},\mathbf{r}_{p}^{f})=\sigma_{5}\left(\mathbf{h}_{p} \odot\mathbf{r}_{p}^{f}\right)\in\mathbb{R}^{d},$ (8) $$

where $\sigma_{5}(\cdot)$ is also an MLP with the GELU activation function. Once we have the masked embedding vectors of samples (excluding $s$) in $\mathcal{P}$, we finally compute the sample context vector of sample $s$, denoted by $\mathbf{z}_{s}^{f}$, which is a weighted sum of the masked embedding vectors with weights being the pairwise similarity values, i.e.,

$$ $\mathbf{z}_{s}^{f}=\sigma_{6}\left(\sum_{p\in\mathcal{P}\setminus\{s\}}\mbox{ sim}(s,p~{}|~{}f)~{}\phi_{p}(\mathbf{h}_{p},\mathbf{r}_{p}^{f})\right)\in \mathbb{R}^{d},$ (9) $$

where $\sigma_{6}(\cdot)$ is again an MLP with the GELU activation function. From (6)–(9), the operations of SCU can be written as

$$ $\mathbf{z}_{s}^{f}=\textbf{SCU}{}(\mathbf{H}_{\mathcal{P}},\mathbf{M}_{ \mathcal{P}},\mathbf{H}_{F})=\sigma_{6}\left(\sum_{p\in\mathcal{P}\setminus\{s \}}\mbox{sim}(s,p~{}|~{}f)~{}\sigma_{5}\left(\mathbf{h}_{p}\odot\sigma_{4} \left([\mathbf{m}_{p};\overline{\mathbf{m}}_{f}]\right)\right)\right),$ (10) $$

where $\mathbf{H}_{\mathcal{P}}=\{\mathbf{h}_{p},p\in\mathcal{P}\}$ and $\mathbf{M}_{\mathcal{P}}=\{\mathbf{m}_{p},p\in\mathcal{P}\}$.

### 3.5 Imputation

For a given sample $s$, to impute the missing value of feature $f$, M^3-Impute obtains its feature context vector $\mathbf{c}_{s}^{f}$ and sample context vector $\mathbf{z}_{s}^{f}$ through FCU and SCU, respectively, which are then used for imputation. Specifically, it is done by predicting the corresponding edge weight $\hat{e}_{sf}$ as follows:

$$ $\hat{e}_{sf}=\phi_{\alpha}\left((1-\alpha)\mathbf{c}_{s}^{f}+\alpha\mathbf{z}_ {s}^{f}\right),$ (11) $$

where $\phi_{\alpha}(\cdot)$ denotes an MLP with a non-linear activation function (i.e., ReLU for continuous values and softmax for discrete ones), and $\alpha$ is a learnable scalar parameter. This scalar parameter $\alpha$ is introduced to strike a balance between leveraging feature-wise correlation and sample-wise correlation. It is necessary because the quality of $\mathbf{z}_{s}^{f}$ relies on the quality of the samples chosen in $\mathcal{P}$, so overly relying on $\mathbf{z}_{s}^{f}$ would backfire if their quality is not as desired. To address this problem, instead of employing a fixed weight $\alpha$, we make $\alpha$ learnable and adaptive in determining the weights for $\mathbf{c}_{s}^{f}$ and $\mathbf{z}_{s}^{f}$ . Note that this kind of learnable parameter approach has been widely adopted in natural language processing [, , , ] and computer vision [, , ], showing superior performance to its fixed counterpart. In M^3-Impute, the scalar parameter $\alpha$ is learned based on the similarity values between $s$ and its peer samples $p\in\mathcal{P}\setminus\{s\}$ as follows:

$$ $\alpha=\phi_{\gamma}\Big{(}\operatorname*{\scalerel*{\|}{\sum}}_{p\in\mathcal{ P}\setminus\{s\}}\mbox{sim}\left(s,p~{}|~{}f\right)\Big{)},$ (12) $$

where $\|$ represents the concatenation operation, and $\phi_{\gamma}(\cdot)$ is an MLP with the activation function $\gamma(x)=1-1\,/\,e^{|x|}$. The overall operation of M^3-Impute is summarized in Algorithm 1. To learn network parameters, we use cross-entropy loss and mean square error loss for imputing discrete and continuous feature values, respectively.

Figure: Algorithm 1 Forward computation of M^3-Impute to impute the value of feature $f$ for sample $s$.

## 4 Experiments

### 4.1 Experiment Setup

Datasets: We conduct experiments on 25 open datasets. These real-world datasets consist of mixed data types with both continuous and discrete values and cover different domains including civil engineering (Concrete, Energy), physics and chemistry (Yacht), thermal dynamics (Naval), etc. Since the datasets are fully observed, we introduce missing values by applying a randomly generated mask to the data matrix. Specifically, as used in prior studies [, ], we apply three masking generation schemes, namely missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR).(^1^11More details about the datasets and mask generation for missing values can be found in Appendix.) We use MCAR with a missing ratio of $30\%$, unless otherwise specified. We follow the preprocessing steps adopted in Grape [] to scale feature values to [0, 1] with a MinMax scaler []. Due to the space limit, we below present the results of eight datasets that are used in Grape and report other results in Appendix.

Baseline models: M^3-Impute is compared against popular and state-of-the-art imputation methods, including statistical methods, deep generative methods, and graph-based methods listed as follows: Mean: It imputes the missing value $\hat{e}_{sf}$ as the mean of observed values in feature $f$ from all the samples. K-nearest neighbors (kNN) []: It imputes the missing value $\hat{e}_{sf}$ using the kNNs that have observed values in feature $f$ with weights that are based on the Euclidean distance to sample $s$. Multivariate imputation by chained equations (Mice) []: This method runs multiple regressions where each missing value is modeled upon the observed non-missing values. Iterative SVD (Svd) []: It imputes missing values by solving a matrix completion problem with iterative low-rank singular value decomposition. Spectral regularization algorithm (Spectral) []: This matrix completion algorithm uses the nuclear norm as a regularizer and imputes missing values with iterative soft-thresholded SVD. Miwae []: It works based on an autoencoder generative model trained to maximize a potentially tight lower bound of the log-likelihood of the observed data and Monte Carlo techniques for imputation. Miracle []: It uses the imputation results from naive methods such as Mean and refines them iteratively by learning a missingness graph (m-graph) and regularizing an imputation function. Gain []: This method trains a data imputation generator with a generalized generative adversarial network in which the discriminator aims to distinguish between real and imputed values. Grape []: It models the data as a bipartite graph and imputes missing values by predicting the weights of the missing edges, each of which is done based on the inner product between the embeddings of its corresponding sample and feature nodes. HyperImpute []: HyperImpute is a framework that conducts an extensive search among a set of imputation methods, selecting the optimal imputation method with fine-tuned parameters for each feature in the dataset. We follow the official implementations of all the baseline models and report their hyperparameters in Appendix.

M^3-Impute configurations: Parameters of M^3-Impute are updated by the Adam optimizer with a learning rate of 0.001 for 40,000 epochs. For graph representation learning, we use a three-layer GNN model, which is a variant of GraphSAGE [] that not only learns node embeddings but also edge embeddings via the neighborhood aggregation mechanism, as similarly used in Grape []. We employ mean-pooling as the aggregation function and use ReLU as the activation function for the GNN layers. We set the embedding dimension $d$ to 128. We randomly drop $50\%$ of observable edges during training to improve the model’s generalization ability. For each experiment, we conduct five runs with different random seeds and report the average results.

### 4.2 Overall Performance

**Table 1: Imputation accuracy in MAE. MAE scores are enlarged by 10 times.**
|  | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Mean | 2.09 $\pm$ .04 | 0.98 $\pm$ .01 | 1.79 $\pm$ .01 | 1.85 $\pm$ .00 | 3.10 $\pm$ .04 | 2.31 $\pm$ .00 | 2.50 $\pm$ .00 | 1.68 $\pm$ .00 |
| Svd | 2.46 $\pm$ .16 | 0.92 $\pm$ .01 | 1.94 $\pm$ .02 | 1.53 $\pm$ .03 | 2.24 $\pm$ .06 | 0.50 $\pm$ .00 | 3.67 $\pm$ .06 | 2.33 $\pm$ .01 |
| Spectral | 2.64 $\pm$ .11 | 0.91 $\pm$ .01 | 1.98 $\pm$ .04 | 1.46 $\pm$ .03 | 2.26 $\pm$ .09 | 0.41 $\pm$ .00 | 2.80 $\pm$ .01 | 2.13 $\pm$ .01 |
| Mice | 1.68 $\pm$ .05 | 0.77 $\pm$ .00 | 1.34 $\pm$ .01 | 1.16 $\pm$ .03 | 1.53 $\pm$ .04 | 0.20 $\pm$ .01 | 2.50 $\pm$ .00 | 1.16 $\pm$ .01 |
| Knn | 1.67 $\pm$ .02 | 0.72 $\pm$ .00 | 1.16 $\pm$ .03 | 0.95 $\pm$ .01 | 1.81 $\pm$ .03 | 0.10 $\pm$ .00 | 2.77 $\pm$ .01 | 1.38 $\pm$ .01 |
| Gain | 2.26 $\pm$ .11 | 0.86 $\pm$ .00 | 1.67 $\pm$ .03 | 1.23 $\pm$ .02 | 1.99 $\pm$ .03 | 0.46 $\pm$ .02 | 2.70 $\pm$ .00 | 1.31 $\pm$ .05 |
| Miwae | 2.37 $\pm$ .01 | 1.00 $\pm$ .00 | 1.81 $\pm$ .01 | 1.74 $\pm$ .04 | 2.79 $\pm$ .04 | 2.37 $\pm$ .00 | 2.57 $\pm$ .00 | 1.72 $\pm$ .00 |
| Grape | 1.46 $\pm$ .01 | 0.60 $\pm$ .00 | 0.75 $\pm$ .01 | 0.64 $\pm$ .01 | 1.36 $\pm$ .01 | 0.07 $\pm$ .00 | 2.50 $\pm$ .00 | 1.00 $\pm$ .00 |
| Miracle | 3.84 $\pm$ .00 | 0.70 $\pm$ .00 | 1.71 $\pm$ .05 | 3.12 $\pm$ .00 | 3.94 $\pm$ .01 | 0.18 $\pm$ .00 | 2.49 $\pm$ .00 | 1.13 $\pm$ .01 |
| HyperImpute | 1.76 $\pm$ .03 | 0.67 $\pm$ .01 | 0.84 $\pm$ .02 | 0.82 $\pm$ .01 | 1.32 $\pm$ .02 | 0.04 $\pm$ .00 | 2.58 $\pm$ .05 | 1.06 $\pm$ .01 |
| M^3-Impute | 1.33 $\pm$ .04 | 0.60 $\pm$ .00 | 0.71 $\pm$ .01 | 0.59 $\pm$ .00 | 1.31 $\pm$ .01 | 0.06 $\pm$ .00 | 2.50 $\pm$ .00 | 0.99 $\pm$ .00 |

We first compare the feature imputation performance of M^3-Impute with popular and state-of-the-art imputation methods. As shown in Table 1, M^3-Impute achieves the lowest imputation MAE for six out of the eight examined datasets and the second-best MAE scores in the other two, which validates the effectiveness of M^3-Impute. For Kin8nm dataset, M^3-Impute underperforms Miracle. It is mainly because each feature in Kin8nm is independent of the others, so none of the observed features can help impute missing feature values. For Naval dataset, the only model that outperforms M^3-Impute is HyperImpute []. In the Naval dataset, nearly every feature exhibits a strong linear correlation with the other features, i.e., every pair of features has correlation coefficient close to one. This allows HyperImpute to readily select a linear model from its model pool for each feature to impute. Nonetheless, M^3-Impute exhibits overall superior performance to the baselines as it can be well adapted to datasets with different levels of correlations over features and samples. In other words, M^3-Impute benefits from explicitly incorporating the missingness information with our carefully designed masking schemes to better capture feature-wise and sample-wise correlations.

Furthermore, we evaluate the performance of M^3-Impute under MAR and MNAR settings. We observe that M^3-Impute consistently outperforms all the baselines under all the eight datasets and achieves an even larger margin in the improvement compared to the case with MCAR setting. This implies that our explicit modeling of the missingness information through our novel soft masking schemes in FCU and SCU as well as the initialization unit is effective in handling different patterns of missing values in the input data. It is worth noting that some baseline models may perform worse than simple imputers, such as Mean, on certain datasets, as similarly observed in recent studies [, ]. It may be because these datasets are of relative small size, and the number of samples and features is not sufficient to train the corresponding models. Comprehensive results on all the datasets across different missing ratios are provided in Appendix.

### 4.3 Ablation Study

To study the effectiveness of three integral components of M^3-Impute, we consider three variants of M^3-Impute, each with a subset of the components, namely initialization only (Init Only), initialization + FCU (Init + FCU), and initialization + SCU (Init + SCU). The performance of these variants are evaluated against the top-performing imputation baselines such as Grape and HyperImpute. As shown in Table 2, the three variants derived from M^3-Impute achieve lower MAE values than both baselines in most datasets, demonstrating the effectiveness of our novel components in M^3-Impute.

**Table 2: Ablation study. M^3-Uniform stands for M^3-Impute with the uniform sampling strategy.**
|  | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| HyperImpute | 1.76 $\pm$ .03 | 0.67 $\pm$ .01 | 0.84 $\pm$ .02 | 0.82 $\pm$ .01 | 1.32 $\pm$ .02 | 0.04 $\pm$ .00 | 2.58 $\pm$ .05 | 1.06 $\pm$ .01 |
| Grape | 1.46 $\pm$ .01 | 0.60 $\pm$ .00 | 0.75 $\pm$ .01 | 0.64 $\pm$ .01 | 1.36 $\pm$ .01 | 0.07 $\pm$ .00 | 2.50 $\pm$ .00 | 1.00 $\pm$ .00 |
| Architecture |  |  |  |  |  |  |  |  |
| Init Only | 1.43 $\pm$ .01 | 0.60 $\pm$ .00 | 0.74 $\pm$ .00 | 0.63 $\pm$ .01 | 1.35 $\pm$ .01 | 0.06 $\pm$ .00 | 2.50 $\pm$ .00 | 0.99 $\pm$ .00 |
| Init+FCU | 1.35 $\pm$ .01 | 0.61 $\pm$ .00 | 0.72 $\pm$ .03 | 0.61 $\pm$ .02 | 1.32 $\pm$ .00 | 0.07 $\pm$ .01 | 2.50 $\pm$ .00 | 0.99 $\pm$ .00 |
| Init+SCU | 1.37 $\pm$ .01 | 0.60 $\pm$ .00 | 0.73 $\pm$ .00 | 0.63 $\pm$ .01 | 1.30 $\pm$ .00 | 0.09 $\pm$ .01 | 2.50 $\pm$ .00 | 1.00 $\pm$ .00 |
| M^3-Impute | 1.33 $\pm$ .04 | 0.60 $\pm$ .00 | 0.71 $\pm$ .01 | 0.59 $\pm$ .00 | 1.31 $\pm$ .01 | 0.06 $\pm$ .00 | 2.50 $\pm$ .00 | 0.99 $\pm$ .00 |
| Sampling Strategy |  |  |  |  |  |  |  |  |
| M^3-Uniform | 1.34 $\pm$ .01 | 0.60 $\pm$ .00 | 0.73 $\pm$ .01 | 0.61 $\pm$ .00 | 1.31 $\pm$ .00 | 0.06 $\pm$ .00 | 2.50 $\pm$ .00 | 0.99 $\pm$ .00 |

Specifically, for initialization only, the key difference between M^3-Impute and Grape lies in our refined initialization process to explicitly leverage missingness information in node embeddings. The reduced MAE values observed by the Init Only variant demonstrate that our proposed initialization process is more effective in utilizing information between samples and their associated features, including missing ones, as compared to the basic initialization used in Grape []. In addition, we observe that when FCU or SCU is incorporated, MAE values are further reduced for most datasets. This validates that explicitly modeling of missingness information through our novel masking schemes in FCU and SCU indeed improves imputation accuracy. When all the three components are combined together as in M^3-Impute, they work synergistically to lower MAE values, validating the efficacy of incorporating the missingness information when capturing sample-wise and feature-wise correlations for missing data imputation.

### 4.4 Robustness

Missing ratio: In practice, datasets may possess different missing ratios. To validate the model’s robustness under such circumstances, we evaluate the performance of M^3-Impute and other baseline models with varying missing ratios, i.e., 0.1, 0.3, 0.5, and 0.7. Figure 3 shows their performance. We use the MAE of HyperImpute ($HI$) as the reference performance and offset the performance of each model by $\mbox{MAE}_{x}-\mbox{MAE}_{HI}$, where $x$ represents the considered model. For clarity, we here only report the results of four top-performing models. As shown in Figure 3, M^3-Impute outperforms other baseline models for almost all the cases, especially under Yacht, Concrete, Energy, and Housing datasets. It is worth noting that modeling feature correlations in these datasets is particularly challenging due to the presence of considerable amounts of weakly correlated features, along with a few strongly correlated ones. Nonetheless, FCU and SCU in M^3-Impute are able to better capture such correlations with our efficient masking schemes, thereby resulting in a large improvement in imputation accuracy. In addition, for Kin8nm dataset, M^3-Impute ties with the second-best model, Grape. As mentioned in Section 4.2, each feature in Kin8nm is independent of the others, so none of the observed features can help impute missing feature values. For Naval dataset, where each feature strongly correlates with the others, M^3-Impute surpasses Grape but falls short of HyperImpute, due to the same reason as discussed in Section 4.2. Overall, M^3-Impute is robust to various missing ratios. Comprehensive results can be found in Appendix.

Figure: Figure 3: Model performance vs. missing ratios. MAE scores are offset by HyperImpute.
Refer to caption: x3.png

Sampling strategy in SCU: While SCU uses a sampling strategy based on pairwise cosine similarities to construct a subset of samples $\mathcal{P}$, the simplest sampling strategy to build $\mathcal{P}$ would be to choose samples uniformly at random without replacement (M^3-Uniform). Intuitively, this approach cannot identify similar peer samples accurately and thus would lead to inferior performance. Nonetheless, as shown in Table 2, even with this naive uniform sampling strategy, M^3-Uniform still outperforms the two leading imputation baselines.

**Table 3: MAE scores for varying peer-sample size ($|\mathcal{P}|\!-\!1$) and different values of $\epsilon$.**
|  | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Peer $=1$ | 1.34 $\pm$ .00 | 0.60 $\pm$ .00 | 0.73 $\pm$ .00 | 0.61 $\pm$ .01 | 1.32 $\pm$ .00 | 0.06 $\pm$ .00 | 2.5 $\pm$ .00 | 0.99$\pm$ .00 |
| Peer $=2$ | 1.35 $\pm$ .01 | 0.61 $\pm$ .00 | 0.72 $\pm$ .01 | 0.59 $\pm$ .01 | 1.32 $\pm$ .00 | 0.06 $\pm$ .00 | 2.5 $\pm$ .00 | 1.00 $\pm$ .00 |
| Peer $=5$ | 1.33 $\pm$ .04 | 0.60 $\pm$ .00 | 0.71 $\pm$ .01 | 0.60 $\pm$ .00 | 1.32 $\pm$ .01 | 0.06 $\pm$ .00 | 2.5 $\pm$ .00 | 0.99$\pm$ .00 |
| Peer $=10$ | 1.33 $\pm$ .01 | 0.61 $\pm$ .00 | 0.71 $\pm$ .01 | 0.60 $\pm$ .01 | 1.31 $\pm$ .01 | 0.07 $\pm$ .00 | 2.5 $\pm$ .00 | 1.00 $\pm$ .00 |
| Peer $=15$ | 1.34 $\pm$ .00 | 0.61 $\pm$ .00 | 0.72 $\pm$ .01 | 0.60 $\pm$ .00 | 1.31 $\pm$ .00 | 0.07 $\pm$ .00 | 2.5 $\pm$ .00 | 0.99 $\pm$ .00 |
| Peer $=20$ | 1.34 $\pm$ .04 | 0.61 $\pm$ .00 | 0.72 $\pm$ .01 | 0.60 $\pm$ .01 | 1.31 $\pm$ .00 | 0.07 $\pm$ .00 | 2.5 $\pm$ .00 | 1.00 $\pm$ .00 |
| $\epsilon$ = 0 | 1.34 $\pm$ .01 | 0.61 $\pm$ .00 | 0.71 $\pm$ .01 | 0.60 $\pm$ .01 | 1.30 $\pm$ .00 | 0.06 $\pm$ .00 | 2.50 $\pm$ .00 | 0.99 $\pm$ .00 |
| $\epsilon$ = $10^{-5}$ | 1.31 $\pm$ .01 | 0.61 $\pm$ .00 | 0.71 $\pm$ .00 | 0.60 $\pm$ .01 | 1.30 $\pm$ .00 | 0.07 $\pm$ .00 | 2.50 $\pm$ .00 | 1.00 $\pm$ .00 |
| $\epsilon$ = $10^{-4}$ | 1.33 $\pm$ .04 | 0.60 $\pm$ .00 | 0.71 $\pm$ .01 | 0.60 $\pm$ .00 | 1.30 $\pm$ .00 | 0.06 $\pm$ .00 | 2.50 $\pm$ .00 | 0.99 $\pm$ .00 |
| $\epsilon$ = $10^{-3}$ | 1.33 $\pm$ .04 | 0.60 $\pm$ .00 | 0.72 $\pm$ .01 | 0.60 $\pm$ .01 | 1.30 $\pm$ .00 | 0.07 $\pm$ .01 | 2.50 $\pm$ .00 | 0.99 $\pm$ .00 |

Size of $\mathcal{P}$ in SCU: Intuitively, a proper peer size ($|\mathcal{P}|\!-\!1$) should balance high-similarity peers and potential peers that serve for regularization and generalization purposes. In general, the trend across different datasets shows that a too-small peer size may only include high-similarity peers, while a too-large peer size may include too many noisy nodes and incur higher computational overhead. As shown in Table 3, the variation in performance is small, indicating that our method is relatively robust to this parameter. From the extensive experiments on 25 datasets, we recommend a peer size of 5–10 for practical use.

Initialization parameter $\epsilon$: We also evaluate whether a non-zero value of $\epsilon$ in the initialization unit of M^3-Impute indeed leads to an improvement in imputation accuracy. As shown in Table 3, for Yacht and Wine datasets, the introduction of a non-zero value of $\epsilon$ results in lower MAE scores. Another insight that we have from Table 3 is that $\epsilon$ should not be set too large, as a large value of $\epsilon$ might impose incorrect weights to the features with missing values. We observe that it is an overall good choice to set $\epsilon$ to $1\!\times\!10^{-5}$ or $1\!\times\!10^{-4}$.

### 4.5 Running time Analysis

We present a running time comparison in Table 4. The results show that our method is both accurate and time-efficient. For example, for inference with GPU, the time taken to impute *all* the missing values for any dataset we tested is less than one second under the setting of MCAR with 30% missingness. More results can be found in Appendix.

**Table 4: Running time (in seconds) for feature imputation using different methods at test time. (C) represents CPU running time and (G) indicates GPU running time.**
| Model | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Mean (C) | 0.0006 | 0.0018 | 0.0013 | 0.0018 | 0.0011 | 0.0105 | 0.0037 | 0.0020 |
| kNN (C) | 0.03 | 0.75 | 0.27 | 0.10 | 0.17 | 47.97 | 16.36 | 17.92 |
| Mice (C) | 0.03 | 0.13 | 0.04 | 0.07 | 0.05 | 1.21 | 0.16 | 0.05 |
| Gain (C) | 3.25 | 3.95 | 3.21 | 4.24 | 3.38 | 4.01 | 3.05 | 3.21 |
| HyperImpute (C) | 21.68 | 23.98 | 36.00 | 131.83 | 42.12 | 56.88 | 28.67 | 22.61 |
| Miracle (C) | 3.94 | 12.28 | 8.65 | 6.23 | 5.23 | 75.19 | 35.03 | 32.77 |
| Miwae (C) | 7.63 | 37.14 | 23.44 | 12.13 | 17.95 | 283.64 | 164.71 | 206.09 |
| Grape (C) | 0.05 | 0.26 | 0.12 | 0.10 | 0.09 | 3.32 | 1.14 | 0.63 |
| Grape (G) | 0.02 | 0.02 | 0.02 | 0.02 | 0.02 | 0.19 | 0.07 | 0.05 |
| M^3-Impute (C) | 0.05 | 0.43 | 0.18 | 0.14 | 0.13 | 5.09 | 1.55 | 0.78 |
| M^3-Impute (G) | 0.02 | 0.02 | 0.04 | 0.04 | 0.04 | 0.56 | 0.19 | 0.12 |

### 4.6 Different GNN Variants

We also conduct experiments using different GNN variants such as GraphSAGE, GAT, and GCN. The results in Table 5 indicate that different aggregation mechanisms may introduce varying errors, but our method consistently outperforms its Grape counterpart, demonstrating its effectiveness.

**Table 5: MAE for different GNN variants under MCAR setting with 30% missingness.**
|  | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| M^3-Impute w/ different GNN variants: |  |  |  |  |  |  |  |  |
| E-GraphSage | 1.33 | 0.60 | 0.71 | 0.60 | 1.32 | 0.06 | 2.50 | 0.99 |
| GCN | 2.04 | 0.97 | 1.76 | 1.62 | 2.37 | 2.10 | 2.50 | 1.66 |
| GAT | 1.56 | 0.95 | 1.01 | 0.78 | 1.39 | 0.31 | 2.50 | 1.02 |
| GraphSage | 1.48 | 0.68 | 1.02 | 0.78 | 1.44 | 0.37 | 2.50 | 1.05 |
| Grape w/ different GNN variants: |  |  |  |  |  |  |  |  |
| E-GraphSage | 1.46 | 0.60 | 0.75 | 0.64 | 1.36 | 0.07 | 2.50 | 1.00 |
| GCN | 2.04 | 1.44 | 2.24 | 2.90 | 3.27 | 2.71 | 2.50 | 1.73 |
| GAT | 2.02 | 0.99 | 1.90 | 2.13 | 3.22 | 2.45 | 2.50 | 1.67 |
| GraphSage | 2.02 | 0.98 | 1.81 | 1.74 | 3.17 | 2.30 | 2.50 | 1.66 |

## 5 Conclusion

In this paper, we highlighted the importance of missingness information and presented M^3-Impute, a mask-guided representation learning method for missing value imputation. M^3-Impute improved the embedding initialization process by considering the relationships between samples and their associated features (including missing ones). In addition, for more effective representation learning, we introduced two novel components in M^3-Impute – FCU and SCU, which explicitly model the missingness information with our novel soft masking schemes to better capture data correlations for imputation. Extensive experiment results on 25 open datasets demonstrate the effectiveness of M^3-Impute, where it achieves overall superior performance to popular and state-of-the-art methods, with 20 best and 4 second-best MAE scores on average under three different settings of missing value patterns.

## Appendix A Appendix

**Table 6: Overview of datasets, which contains continuous features (Cont. F.) and discrete features (Disc. F.).**
|  | Concrete | Housing | Wine | Yacht | Energy | Kin8nm | Naval | Power |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| # Samples | 1030 | 506 | 1599 | 308 | 768 | 8192 | 11934 | 9568 |
| # Cont. F. | 8 | 12 | 11 | 6 | 8 | 8 | 16 | 4 |
| # Disc. F. | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 |

Figure: Figure 4: Pearson correlation coefficients of UCI datasets.
Refer to caption: x4.png

In this section, we elaborate on extensive and comprehensive experiment results. We first provide an overview of the dataset details in Section A.1, and present the performance of the imputation methods under different missingness settings, namely MAR and MNAR, in Section A.2. We then provide the comprehensive results across different missingness ratios in Section A.3. For more thorough analysis, we extend our evaluation of M^3-Impute on 17 additional datasets, totaling 25 datasets, in Section A.4, and elaborate on the computational resources used in Section A.5. We further assess the quality of imputed values generated by M^3-Impute by leveraging them in downstream tasks in Section A.6. Finally, we perform a sensitivity analysis on the hyperparameters of M^3-Impute in Section A.7, and provide the implementation details of baselines, including their hyperparameter choices, in Section A.8.

### A.1 Dataset Details

Table 6 presents the statistics of the eight UCI datasets [] used throughout Section 4. Figure 4 illustrates the Pearson correlation coefficients among the features. In the Kin8nm dataset, all features are linearly independent, whereas the Naval dataset exhibits strong correlations among its features. Under the MCAR setting, M^3-Impute performs comparably to the baseline imputation methods on these two datasets (shown in Table 1). However, in real-world scenarios, features are not always entirely independent or strongly correlated. In the other six datasets, we observe a mix of weakly correlated features along with a few that are strongly correlated. In these cases, M^3-Impute consistently outperforms all baseline methods.

### A.2 Detailed Results of Different Missingness Settings

We adopt the same procedure outlined in Grape [] to generate missing values under different settings.

- •
MCAR: An $n\times m$ matrix is sampled from a uniform distribution. Positions with values no greater than the ratio of missingness are viewed as missing and the remaining positions are observable.
- •
MAR: A subset of features is randomly selected to be fully observed. The values for the remaining features are removed according to a logistic model with random weights, using the fully observed feature values as input. The desired rate of missingness is achieved by adjusting the bias term.
- •
MNAR: This is done by first applying the MAR mechanism above. Then, the remaining feature values are masked out using the MCAR mechanism.

**Table 7: MAE scores under MAR setting with 30% missingness.**
|  | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Mean | 2.20 $\pm$ .13 | 1.09 $\pm$ .05 | 1.79 $\pm$ .21 | 2.02 $\pm$ .20 | 3.26 $\pm$ .36 | 2.75 $\pm$ .11 | 2.49 $\pm$ .01 | 1.81 $\pm$ .25 |
| Svd | 2.64 $\pm$ .22 | 1.04 $\pm$ .14 | 2.32 $\pm$ .06 | 1.71 $\pm$ .15 | 3.68 $\pm$ .16 | 0.52 $\pm$ .11 | 2.69 $\pm$ .02 | 2.37 $\pm$ .62 |
| Spectral | 3.06 $\pm$ .11 | 0.91 $\pm$ .13 | 2.12 $\pm$ .17 | 1.84 $\pm$ .28 | 2.88 $\pm$ .35 | 1.29 $\pm$ .47 | 3.56 $\pm$ .01 | 3.37 $\pm$ .04 |
| Mice | 1.79 $\pm$ .10 | 0.79 $\pm$ .01 | 1.27 $\pm$ .08 | 1.22 $\pm$ .05 | 1.12 $\pm$ .07 | 0.27 $\pm$ .01 | 2.51 $\pm$ .03 | 1.16 $\pm$ .11 |
| Knn | 1.69 $\pm$ .07 | 0.66 $\pm$ .07 | 0.89 $\pm$ .30 | 0.89 $\pm$ .12 | 1.61 $\pm$ .35 | 0.07 $\pm$ .00 | 2.94 $\pm$ .01 | 1.11 $\pm$ .04 |
| Gain | 2.07 $\pm$ .02 | 1.13 $\pm$ .20 | 1.87 $\pm$ .16 | 0.92 $\pm$ .05 | 2.26 $\pm$ .14 | 0.91 $\pm$ .07 | 2.93 $\pm$ .02 | 1.42 $\pm$ .01 |
| Miwae | 2.17 $\pm$ .02 | 0.98 $\pm$ .02 | 1.80 $\pm$ .01 | 1.53 $\pm$ .05 | 3.91 $\pm$ .04 | 2.91 $\pm$ .07 | 2.58 $\pm$ .02 | 2.05 $\pm$ .01 |
| Grape | 1.20 $\pm$ .03 | 0.60 $\pm$ .00 | 0.77 $\pm$ .02 | 0.66 $\pm$ .01 | 1.05 $\pm$ .02 | 0.07 $\pm$ .01 | 2.49 $\pm$ .00 | 1.06 $\pm$ .04 |
| Miracle | 3.75 $\pm$ .00 | 0.70 $\pm$ .00 | 1.94 $\pm$ .00 | 2.24 $\pm$ .00 | 3.89 $\pm$ .00 | 0.36 $\pm$ .00 | 2.82 $\pm$ .10 | 0.86 $\pm$ .01 |
| Hyperimpute | 2.06 $\pm$ .12 | 0.78 $\pm$ .06 | 1.30 $\pm$ .15 | 1.05 $\pm$ .21 | 1.11 $\pm$ .38 | 1.01 $\pm$ .18 | 3.07 $\pm$ .06 | 1.07 $\pm$ .14 |
| M^3-Impute | 1.09 $\pm$ .03 | 0.60 $\pm$ .00 | 0.77 $\pm$ .02 | 0.60 $\pm$ .00 | 0.98 $\pm$ .02 | 0.07 $\pm$ .00 | 2.49 $\pm$ .00 | 1.01 $\pm$ .00 |

**Table 8: MAE scores under MNAR setting with 30% missingness.**
|  | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Mean | 2.18 $\pm$ .09 | 1.04 $\pm$ .02 | 1.80 $\pm$ .09 | 1.95 $\pm$ .13 | 3.17 $\pm$ .22 | 2.60 $\pm$ .07 | 2.49 $\pm$ .01 | 1.76 $\pm$ .14 |
| Svd | 2.61 $\pm$ .13 | 1.06 $\pm$ .07 | 2.24 $\pm$ .05 | 1.58 $\pm$ .06 | 3.55 $\pm$ .09 | 0.53 $\pm$ .05 | 2.69 $\pm$ .02 | 2.27 $\pm$ .25 |
| Spectral | 2.75 $\pm$ .14 | 1.01 $\pm$ .08 | 1.86 $\pm$ .03 | 1.60 $\pm$ .22 | 2.50 $\pm$ .15 | 1.35 $\pm$ .21 | 3.34 $\pm$ .00 | 3.14 $\pm$ .41 |
| Mice | 1.91 $\pm$ .10 | 0.77 $\pm$ .07 | 1.37 $\pm$ .05 | 1.22 $\pm$ .06 | 1.57 $\pm$ .03 | 0.21 $\pm$ .07 | 2.50 $\pm$ .00 | 1.08 $\pm$ .02 |
| Knn | 1.92 $\pm$ .10 | 0.75 $\pm$ .05 | 1.15 $\pm$ .32 | 0.95 $\pm$ .11 | 1.96 $\pm$ .11 | 0.08 $\pm$ .02 | 3.06 $\pm$ .02 | 1.65 $\pm$ .07 |
| Gain | 2.34 $\pm$ .12 | 0.92 $\pm$ .05 | 1.80 $\pm$ .05 | 1.08 $\pm$ .05 | 1.92 $\pm$ .06 | 1.12 $\pm$ .03 | 2.78 $\pm$ .03 | 1.22 $\pm$ .03 |
| Miwae | 2.17 $\pm$ .00 | 0.99 $\pm$ .01 | 1.81 $\pm$ .03 | 1.60 $\pm$ .02 | 3.63 $\pm$ .00 | 2.63 $\pm$ .03 | 2.55 $\pm$ .02 | 1.95 $\pm$ .03 |
| Grape | 1.23 $\pm$ .03 | 0.61 $\pm$ .00 | 0.73 $\pm$ .01 | 0.61 $\pm$ .01 | 1.16 $\pm$ .01 | 0.08 $\pm$ .01 | 2.46 $\pm$ .01 | 1.02 $\pm$ .01 |
| Miracle | 3.85 $\pm$ .00 | 0.70 $\pm$ .00 | 1.87 $\pm$ .00 | 2.51 $\pm$ .00 | 3.86 $\pm$ .00 | 0.30 $\pm$ .00 | 2.64 $\pm$ .00 | 1.06 $\pm$ .00 |
| Hyperimpute | 1.95 $\pm$ .10 | 0.72 $\pm$ .03 | 0.88 $\pm$ .02 | 0.85 $\pm$ .03 | 1.19 $\pm$ .24 | 0.85 $\pm$ .04 | 2.71 $\pm$ .06 | 1.09 $\pm$ .06 |
| M^3-Impute | 1.15 $\pm$ .02 | 0.60 $\pm$ .00 | 0.68 $\pm$ .02 | 0.54 $\pm$ .01 | 1.09 $\pm$ .01 | 0.08 $\pm$ .00 | 2.46 $\pm$ .00 | 1.00 $\pm$ .00 |

In addition to the results for MCAR setting presented in Table 4.2, Tables 7 and 8 present the MAE scores under MAR and MNAR settings, respectively. M^3-Impute consistently outperforms all the baseline methods in both scenarios.

### A.3 Robustness against Various Missingness Settings

Tables 16, 17, and 18 present the performance of various imputation methods under different levels of missingness across MCAR, MAR, and MNAR settings, respectively. M^3-Impute achieves the lowest MAE scores in most cases and the second-best MAE scores in the remaining ones.

### A.4 Further Evaluation on 17 Additional Datasets

**Table 9: Overview of 17 additional datasets, which contains continuous features (Cont. F.) and discrete features (Disc. F.).**
|  | AIrfoil | BLood | Wine-White | IOnosphere | BReast | IRis | DIabetes | PRotein | SPam |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| # Samples | 1503 | 748 | 4899 | 351 | 569 | 150 | 442 | 45730 | 4601 |
| # Cont. F. | 5 | 4 | 12 | 34 | 30 | 4 | 10 | 9 | 57 |
| # Disc. F. | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
|  | LEtter | ABalone | Ai4i | CMC | GErman | STeel | LIbras | California-Housing |  |
| # Samples | 20000 | 4177 | 10000 | 1473 | 1000 | 1941 | 360 | 20640 |  |
| # Cont. F. | 16 | 7 | 7 | 8 | 13 | 33 | 91 | 9 |  |
| # Disc. F. | 0 | 1 | 5 | 1 | 7 | 0 | 0 | 0 |  |

Figure: Figure 5: Pearson correlation coefficient of 17 additional datasets.
Refer to caption: x5.png

In this experiment, we further evaluate M^3-Impute on 17 datasets: Airfoil [], Blood [], Wine-White [], Ionosphere [], Breast Cancer [], Iris [], Diabetes [],
Protein, Spam, Letter [], Abalone, Ai4i, CMC [], German [], Steel, Libras, and California-housing.
An overview of the dataset details is provided in Table 9, and feature correlations are illustrated in Figure 5. We conduct experiments with missingness in data under MCAR, MAR, and MNAR settings, each with missing ratios of {0.1, 0.3, 0.5, 0.7}. Results are presented in Tables 19, 20, and 21 for MCAR, MAR, and MNAR settings, respectively. Across all three types of missingness for the 17 datasets, M^3-Impute achieves 13 best and 3 second-best MAE scores on average.

### A.5 Computational Resources

All our experiments are conducted on a GPU server running Ubuntu 22.04, with PyTorch 2.1.0 and CUDA 12.1. We train and test M^3-Impute using a single NVIDIA A100 80G GPU. With the experimental setup described in Section 4.1, the total running time (including both training and testing) for each of the five repeated runs ranged from 1 to 5 hours, depending on the scale of the datasets. In Section 4.5, we report the running time of M^3-Impute on eight datasets. Here we extend our investigation to larger datasets with more samples and features. As shown in Table 10. M^3-Impute also demonstrates efficiency on these datasets. When using a GPU, the time required to impute all missing values for any dataset is less than one second under the setting of MCAR with 30% missingness.

**Table 10: Running time (in seconds) for feature imputation using different methods at test time. (C) represents CPU running time and (G) indicates GPU running time.**
| Model | Protein | Spam | Letter | Libras | California-housing |
| --- | --- | --- | --- | --- | --- |
| Mean (C) | 0.0187 | 0.0151 | 0.0162 | 0.0073 | 0.0090 |
| Knn (C) | 523.68 | 18.50 | 126.43 | 0.63 | 103.14 |
| Svd (C) | 0.61 | 5.29 | 0.61 | 0.16 | 0.48 |
| Mice (C) | 1.36 | 8.99 | 2.21 | 14.45 | 0.35 |
| Spectral (C) | 3.13 | 1.61 | 3.24 | 0.55 | 1.62 |
| Gain (C) | 5.07 | 5.71 | 4.28 | 8.94 | 3.73 |
| HyperImpute (C) | 37.89 | 183.68 | 26.25 | 26.67 | 35.03 |
| Miracle (C) | 173.69 | 91.53 | 110.45 | 34.24 | 81.48 |
| Miwae (C) | 982.18 | 147.75 | 443.49 | 17.80 | 454.16 |
| Grape (C) | 6.97 | 4.33 | 5.25 | 0.84 | 3.13 |
| Grape (G) | 0.37 | 0.24 | 0.28 | 0.05 | 0.18 |
| M^3-Impute (C) | 10.27 | 7.53 | 7.80 | 1.51 | 4.61 |
| M^3-Impute (G) | 0.99 | 0.69 | 0.76 | 0.19 | 0.51 |

### A.6 Downstream Task Performance

We further evaluate the quality of imputed values from different imputation methods by performing a downstream label prediction task. In particular, each sample in the eight examined datasets contains a continuous label, and the task is to predict the label using the feature values. Starting with an input data matrix with 30% missingness, we first impute the data using the corresponding imputation methods, and then do linear regression on the completed data matrix to predict labels. As shown in Table 11, M^3-Impute consistently achieves good performance across different datasets, with six best performance and two second best, indicating its effectiveness in the missing value imputation.

**Table 11: MAE scores of label prediction under the MCAR setting with 30% missingness.**
|  | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Mean | 9.08 | 0.54 | 10.50 | 4.44 | 4.29 | 0.0065 | 0.18 | 6.31 |
| Svd | 9.12 | 0.55 | 10.90 | 4.26 | 3.40 | 0.0058 | 0.19 | 6.93 |
| Spectral | 8.98 | 0.54 | 10.50 | 4.33 | 3.42 | 0.0057 | 0.19 | 6.67 |
| Mice | 8.95 | 0.54 | 10.20 | 3.99 | 2.84 | 0.0044 | 0.18 | 4.99 |
| Knn | 8.91 | 0.53 | 9.95 | 4.17 | 3.04 | 0.0049 | 0.18 | 5.68 |
| Gain | 9.82 | 0.53 | 10.60 | 4.20 | 2.79 | 0.0060 | 0.18 | 5.08 |
| Miwae | 9.40 | 0.54 | 10.60 | 5.43 | 3.65 | 0.0065 | 0.18 | 5.50 |
| Grape | 8.96 | 0.52 | 9.17 | 3.66 | 2.61 | 0.0038 | 0.18 | 4.83 |
| Miracle | 9.70 | 0.55 | 10.50 | 5.01 | 4.37 | 0.0039 | 0.18 | 4.93 |
| HyperImpute | 9.58 | 0.51 | 9.94 | 3.87 | 2.49 | 0.0032 | 0.18 | 4.69 |
| M^3-Impute | 8.82 | 0.51 | 9.04 | 3.60 | 2.57 | 0.0036 | 0.18 | 4.69 |

### A.7 Analysis of HyperParameters in M 3 -Impute

**Table 12: MAE of M^3-Impute with varying embedding dimensions under the MCAR setting with 30% missingness.**
|  | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| d=32 | 1.38 | 0.63 | 0.89 | 0.71 | 1.32 | 0.18 | 2.50 | 1.02 |
| d=64 | 1.34 | 0.62 | 0.78 | 0.63 | 1.31 | 0.11 | 2.50 | 1.01 |
| d=128 | 1.33 | 0.60 | 0.71 | 0.60 | 1.32 | 0.06 | 2.50 | 0.99 |
| d=256 | 1.37 | 0.61 | 0.68 | 0.60 | 1.33 | 0.06 | 2.50 | 0.99 |

**Table 13: MAE of M^3-Impute with varying numbers of GNN layers (L) under the MCAR setting with 30% missingness.**
|  | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| L = 1 | 1.43 | 0.62 | 0.82 | 0.65 | 1.30 | 0.12 | 2.50 | 1.01 |
| L = 2 | 1.36 | 0.61 | 0.77 | 0.62 | 1.31 | 0.09 | 2.50 | 1.01 |
| L = 3 | 1.33 | 0.60 | 0.71 | 0.60 | 1.32 | 0.06 | 2.50 | 0.99 |
| L = 4 | 1.35 | 0.60 | 0.71 | 0.61 | 1.32 | 0.06 | 2.50 | 0.99 |

**Table 14: MAE of M^3-Impute with varying edge dropout ratios under MCAR setting with 30% missingness.**
| Drop % | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 90% | 1.53 | 0.66 | 0.94 | 0.69 | 1.35 | 0.13 | 2.50 | 1.03 |
| 70% | 1.35 | 0.62 | 0.75 | 0.62 | 1.33 | 0.08 | 2.50 | 1.00 |
| 50% | 1.33 | 0.60 | 0.71 | 0.60 | 1.32 | 0.06 | 2.50 | 0.99 |
| 30% | 1.38 | 0.61 | 0.73 | 0.60 | 1.31 | 0.06 | 2.50 | 1.00 |

In this experiment, we evaluate the sensitivity of M^3-Impute to various hyperparameter settings. Tables 12, 13, and 14 summarize the performance of M^3-Impute across different hidden dimensions, GNN layer counts, and edge dropout ratios. Overall, the results show that M^3-Impute is robust to different hyperparameter settings across the tested datasets. Based on these observations, we recommend setting the hidden dimension to 128, the number of GNN layers to 3, and the edge dropout ratio to 50% as a general guideline.

### A.8 Baseline Configuration

For Mean, Svd, Spectral, and Knn, we follow the widely adopted implementation in Grape []. For Gain [], Miwae [], Grape [], Miracle [], and HyperImpute [], we use their official implementations. By default, we follow the optimal parameter settings provided in the original papers. However, we observe that part of the baselines do not perform well with their default parameters on certain datasets. To ensure a fair comparison, we conduct a grid search over the hyperparameters and report the best results achieved across all our experiments. The search ranges for these hyperparameters are detailed in Table 15.

**Table 15: Hyperparameter search space.**
| Model | Hyperparameters |
| --- | --- |
| Svd | rank = {3, 5, 10, 20} |
| max_iters = {200, 1000, 2000} |  |
| Spectral | max_iters = {100, 200, 500} |
| Mice | max_iter = {10, 30, 50, 100} |
| Knn | K = {3, 5, 10, 20} |
| Gain | n_epochs = {1000, 2000, 3000} |
| Miwae | n_epochs = {1000, 2000, 3000} |
| K = {5, 10, 15, 20} |  |
| Grape | hidden_dim = {64, 128, 256} |
| edge_dropout_ratio = {0.1, 0.3, 0.5} |  |
| Miracle | n_hidden = {8, 16, 32, 64} |
| reg_lambda = range(0.1, 1, 0.1) |  |
| reg_beta = range(0.1, 1, 0.1) |  |
| max_steps = {500, 1000, 2000} |  |

**Table 16: MAE scores under the MCAR setting across different levels of missingness.**
|  | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Missing $10\%$ |  |  |  |  |  |  |  |  |
| Mean | 2.22 $\pm$ 0.05 | 0.96 $\pm$ 0.02 | 1.81 $\pm$ 0.02 | 1.84 $\pm$ 0.01 | 3.09 $\pm$ 0.07 | 2.30 $\pm$ 0.01 | 2.50 $\pm$ 0.01 | 1.68 $\pm$ 0.00 |
| Svd | 1.92 $\pm$ 0.16 | 0.88 $\pm$ 0.03 | 2.04 $\pm$ 0.04 | 1.69 $\pm$ 0.11 | 1.75 $\pm$ 0.10 | 0.34 $\pm$ 0.00 | 5.04 $\pm$ 0.06 | 2.26 $\pm$ 0.04 |
| Spectral | 2.24 $\pm$ 0.12 | 0.76 $\pm$ 0.02 | 1.84 $\pm$ 0.05 | 1.28 $\pm$ 0.04 | 1.76 $\pm$ 0.08 | 0.38 $\pm$ 0.01 | 2.71 $\pm$ 0.02 | 1.77 $\pm$ 0.02 |
| Mice | 1.38 $\pm$ 0.13 | 0.62 $\pm$ 0.01 | 0.97 $\pm$ 0.04 | 0.98 $\pm$ 0.04 | 1.28 $\pm$ 0.07 | 0.13 $\pm$ 0.00 | 2.50 $\pm$ 0.01 | 1.01 $\pm$ 0.01 |
| Knn | 1.40 $\pm$ 0.17 | 0.49 $\pm$ 0.01 | 0.58 $\pm$ 0.05 | 0.74 $\pm$ 0.04 | 1.42 $\pm$ 0.05 | 0.03 $\pm$ 0.00 | 2.53 $\pm$ 0.01 | 1.26 $\pm$ 0.00 |
| Gain | 2.30 $\pm$ 0.04 | 0.83 $\pm$ 0.04 | 1.62 $\pm$ 0.05 | 1.16 $\pm$ 0.05 | 1.95 $\pm$ 0.05 | 0.45 $\pm$ 0.01 | 2.74 $\pm$ 0.02 | 1.22 $\pm$ 0.00 |
| Miwae | 2.39 $\pm$ 0.00 | 1.01 $\pm$ 0.04 | 1.93 $\pm$ 0.02 | 1.73 $\pm$ 0.01 | 3.30 $\pm$ 0.00 | 2.37 $\pm$ 0.00 | 2.57 $\pm$ 0.00 | 1.72 $\pm$ 0.00 |
| Grape | 1.00 $\pm$ 0.00 | 0.48 $\pm$ 0.00 | 0.45 $\pm$ 0.01 | 0.49 $\pm$ 0.00 | 1.19 $\pm$ 0.00 | 0.05 $\pm$ 0.00 | 2.49 $\pm$ 0.00 | 0.85 $\pm$ 0.03 |
| Miracle | 3.87 $\pm$ 0.01 | 0.62 $\pm$ 0.00 | 1.63 $\pm$ 0.01 | 3.07 $\pm$ 0.00 | 4.04 $\pm$ 0.00 | 0.12 $\pm$ 0.00 | 2.48 $\pm$ 0.01 | 0.99 $\pm$ 0.00 |
| HyperImpute | 1.50 $\pm$ 0.11 | 0.52 $\pm$ 0.00 | 0.51 $\pm$ 0.04 | 0.75 $\pm$ 0.04 | 1.18 $\pm$ 0.05 | 0.06 $\pm$ 0.04 | 2.50 $\pm$ 0.00 | 0.84$\pm$ 0.00 |
| M^3-Impute | 0.96 $\pm$ 0.00 | 0.47 $\pm$ 0.01 | 0.41 $\pm$ 0.01 | 0.45 $\pm$ 0.00 | 1.15 $\pm$ 0.00 | 0.05 $\pm$ 0.00 | 2.49 $\pm$ 0.00 | 0.84 $\pm$ 0.01 |
|  | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| Missing $30\%$ |  |  |  |  |  |  |  |  |
| Mean | 2.09 $\pm$ 0.04 | 0.98 $\pm$ 0.01 | 1.79 $\pm$ 0.01 | 1.85 $\pm$ 0.00 | 3.10 $\pm$ 0.04 | 2.31 $\pm$ 0.00 | 2.50 $\pm$ 0.00 | 1.68 $\pm$ 0.00 |
| Svd | 2.46 $\pm$ 0.16 | 0.92 $\pm$ 0.01 | 1.94 $\pm$ 0.02 | 1.53 $\pm$ 0.03 | 2.24 $\pm$ 0.06 | 0.50 $\pm$ 0.00 | 3.67 $\pm$ 0.06 | 2.33 $\pm$ 0.01 |
| Spectral | 2.64 $\pm$ 0.11 | 0.91 $\pm$ 0.01 | 1.98 $\pm$ 0.04 | 1.46 $\pm$ 0.03 | 2.26 $\pm$ 0.09 | 0.41 $\pm$ 0.00 | 2.80 $\pm$ 0.01 | 2.13 $\pm$ 0.01 |
| Mice | 1.68 $\pm$ 0.05 | 0.77 $\pm$ 0.00 | 1.34 $\pm$ 0.01 | 1.16 $\pm$ 0.03 | 1.53 $\pm$ 0.04 | 0.20 $\pm$ 0.01 | 2.50 $\pm$ 0.00 | 1.16 $\pm$ 0.01 |
| Knn | 1.67 $\pm$ 0.02 | 0.72 $\pm$ 0.00 | 1.16 $\pm$ 0.03 | 0.95 $\pm$ 0.01 | 1.81 $\pm$ 0.03 | 0.10 $\pm$ 0.00 | 2.77 $\pm$ 0.01 | 1.38 $\pm$ 0.01 |
| Gain | 2.26 $\pm$ 0.11 | 0.86 $\pm$ 0.00 | 1.67 $\pm$ 0.03 | 1.23 $\pm$ 0.02 | 1.99 $\pm$ 0.03 | 0.46 $\pm$ 0.02 | 2.70 $\pm$ 0.00 | 1.31 $\pm$ 0.05 |
| Miwae | 2.37 $\pm$ 0.01 | 1.00 $\pm$ 0.00 | 1.81 $\pm$ 0.01 | 1.74 $\pm$ 0.04 | 2.79 $\pm$ 0.04 | 2.37 $\pm$ 0.00 | 2.57 $\pm$ 0.00 | 1.72 $\pm$ 0.00 |
| Grape | 1.46 $\pm$ 0.01 | 0.60 $\pm$ 0.00 | 0.75 $\pm$ 0.01 | 0.64 $\pm$ 0.01 | 1.36 $\pm$ 0.01 | 0.07 $\pm$ 0.00 | 2.50 $\pm$ 0.00 | 1.00 $\pm$ 0.00 |
| Miracle | 3.84 $\pm$ 0.00 | 0.70 $\pm$ 0.00 | 1.71 $\pm$ 0.05 | 3.12 $\pm$ 0.00 | 3.94 $\pm$ 0.01 | 0.18 $\pm$ 0.00 | 2.49 $\pm$ 0.00 | 1.13 $\pm$ 0.01 |
| HyperImpute | 1.76 $\pm$ 0.03 | 0.67 $\pm$ 0.01 | 0.84 $\pm$ 0.02 | 0.82 $\pm$ 0.01 | 1.32 $\pm$ 0.02 | 0.04 $\pm$ 0.00 | 2.58 $\pm$ 0.05 | 1.06 $\pm$ 0.01 |
| M^3-Impute | 1.33 $\pm$ 0.04 | 0.60 $\pm$ 0.00 | 0.71 $\pm$ 0.01 | 0.59 $\pm$ 0.00 | 1.31 $\pm$ 0.01 | 0.06 $\pm$ 0.00 | 2.50 $\pm$ 0.00 | 0.99 $\pm$ 0.00 |
|  | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| Missing $50\%$ |  |  |  |  |  |  |  |  |
| Mean | 2.12 $\pm$ 0.02 | 0.98 $\pm$ 0.01 | 1.81 $\pm$ 0.01 | 1.84 $\pm$ 0.01 | 3.08 $\pm$ 0.02 | 2.31 $\pm$ 0.00 | 2.50 $\pm$ 0.00 | 1.67 $\pm$ 0.00 |
| Svd | 3.00 $\pm$ 0.11 | 1.18 $\pm$ 0.00 | 2.19 $\pm$ 0.01 | 1.88 $\pm$ 0.01 | 2.88 $\pm$ 0.04 | 0.87 $\pm$ 0.00 | 3.30 $\pm$ 0.01 | 2.92 $\pm$ 0.02 |
| Spectral | 3.17 $\pm$ 0.13 | 1.13 $\pm$ 0.00 | 2.31 $\pm$ 0.01 | 1.76 $\pm$ 0.03 | 3.03 $\pm$ 0.02 | 0.46 $\pm$ 0.00 | 3.02 $\pm$ 0.00 | 2.98 $\pm$ 0.02 |
| Mice | 1.99 $\pm$ 0.08 | 0.83 $\pm$ 0.00 | 1.59 $\pm$ 0.03 | 1.33 $\pm$ 0.02 | 2.13 $\pm$ 0.12 | 0.31 $\pm$ 0.01 | 2.50 $\pm$ 0.00 | 1.32 $\pm$ 0.01 |
| Knn | 2.08 $\pm$ 0.02 | 0.98 $\pm$ 0.01 | 1.40 $\pm$ 0.02 | 1.37 $\pm$ 0.01 | 2.21 $\pm$ 0.01 | 0.76 $\pm$ 0.01 | 2.65 $\pm$ 0.00 | 1.80 $\pm$ 0.01 |
| Gain | 2.33 $\pm$ 0.03 | 1.18 $\pm$ 0.15 | 2.20 $\pm$ 0.17 | 1.43 $\pm$ 0.09 | 2.58 $\pm$ 0.09 | 0.56 $\pm$ 0.03 | 2.86 $\pm$ 0.06 | 1.36 $\pm$ 0.00 |
| Miwae | 2.41 $\pm$ 0.01 | 1.02 $\pm$ 0.00 | 1.87 $\pm$ 0.04 | 1.76 $\pm$ 0.01 | 3.23 $\pm$ 0.00 | 2.39 $\pm$ 0.01 | 2.58 $\pm$ 0.00 | 1.73 $\pm$ 0.02 |
| Grape | 1.89 $\pm$ 0.02 | 0.75 $\pm$ 0.01 | 1.24 $\pm$ 0.00 | 0.83 $\pm$ 0.01 | 1.63 $\pm$ 0.01 | 0.09 $\pm$ 0.00 | 2.50 $\pm$ 0.00 | 1.19 $\pm$ 0.00 |
| Miracle | 3.84 $\pm$ 0.00 | 0.81 $\pm$ 0.00 | 1.80 $\pm$ 0.00 | 3.07 $\pm$ 0.00 | 3.94 $\pm$ 0.00 | 0.24 $\pm$ 0.00 | 2.76 $\pm$ 0.00 | 1.29 $\pm$ 0.00 |
| HyperImpute | 2.07 $\pm$ 0.11 | 0.85 $\pm$ 0.00 | 1.33 $\pm$ 0.08 | 1.06 $\pm$ 0.11 | 1.70 $\pm$ 0.05 | 0.07 $\pm$ 0.00 | 2.96 $\pm$ 0.04 | 1.29 $\pm$ 0.01 |
| M^3-Impute | 1.74 $\pm$ 0.01 | 0.74 $\pm$ 0.00 | 1.19 $\pm$ 0.02 | 0.79 $\pm$ 0.01 | 1.57 $\pm$ 0.00 | 0.08 $\pm$ 0.00 | 2.50 $\pm$ 0.00 | 1.19 $\pm$ 0.00 |
|  | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| Missing $70\%$ |  |  |  |  |  |  |  |  |
| Mean | 2.16 $\pm$ 0.06 | 0.99 $\pm$ 0.00 | 1.81 $\pm$ 0.01 | 1.83 $\pm$ 0.02 | 3.08 $\pm$ 0.01 | 2.31 $\pm$ 0.00 | 2.50 $\pm$ 0.00 | 1.67 $\pm$ 0.00 |
| Svd | 3.78 $\pm$ 0.06 | 1.63 $\pm$ 0.02 | 2.53 $\pm$ 0.03 | 2.58 $\pm$ 0.07 | 3.65 $\pm$ 0.09 | 1.56 $\pm$ 0.00 | 3.58 $\pm$ 0.00 | 3.88 $\pm$ 0.01 |
| Spectral | 4.17 $\pm$ 0.10 | 1.67 $\pm$ 0.02 | 2.75 $\pm$ 0.01 | 2.59 $\pm$ 0.05 | 4.00 $\pm$ 0.03 | 1.04 $\pm$ 0.00 | 3.73 $\pm$ 0.01 | 4.33 $\pm$ 0.01 |
| Mice | 2.21 $\pm$ 0.10 | 0.93 $\pm$ 0.01 | 1.72 $\pm$ 0.02 | 1.54 $\pm$ 0.04 | 2.71 $\pm$ 0.15 | 0.53 $\pm$ 0.00 | 2.62 $\pm$ 0.08 | 1.46 $\pm$ 0.00 |
| Knn | 2.62 $\pm$ 0.08 | 1.05 $\pm$ 0.00 | 1.60 $\pm$ 0.01 | 1.43 $\pm$ 0.02 | 2.54 $\pm$ 0.04 | 1.08 $\pm$ 0.00 | 2.84 $\pm$ 0.01 | 2.73 $\pm$ 0.00 |
| Gain | 3.07 $\pm$ 0.08 | 1.61 $\pm$ 0.15 | 2.84 $\pm$ 0.04 | 3.09 $\pm$ 0.04 | 3.83 $\pm$ 0.15 | 1.07 $\pm$ 0.02 | 3.31 $\pm$ 0.21 | 1.51 $\pm$ 0.05 |
| Miwae | 2.40 $\pm$ 0.01 | 1.02 $\pm$ 0.03 | 1.86 $\pm$ 0.00 | 1.75 $\pm$ 0.00 | 3.20 $\pm$ 0.01 | 2.39 $\pm$ 0.01 | 2.58 $\pm$ 0.00 | 1.73 $\pm$ 0.04 |
| Grape | 2.14 $\pm$ 0.01 | 0.88 $\pm$ 0.01 | 1.64 $\pm$ 0.02 | 1.12 $\pm$ 0.01 | 2.10 $\pm$ 0.01 | 0.17 $\pm$ 0.00 | 2.49 $\pm$ 0.00 | 1.37 $\pm$ 0.00 |
| Miracle | 3.88 $\pm$ 0.00 | 0.90 $\pm$ 0.00 | 2.23 $\pm$ 0.00 | 3.05 $\pm$ 0.00 | 3.95 $\pm$ 0.00 | 0.48 $\pm$ 0.00 | 2.88 $\pm$ 0.00 | 1.54 $\pm$ 0.00 |
| HyperImpute | 2.49 $\pm$ 0.08 | 0.92 $\pm$ 0.02 | 1.71 $\pm$ 0.01 | 1.12 $\pm$ 0.13 | 2.16 $\pm$ 0.06 | 0.15 $\pm$ 0.00 | 3.15 $\pm$ 0.03 | 1.54 $\pm$ 0.02 |
| M^3-Impute | 2.14 $\pm$ 0.00 | 0.87 $\pm$ 0.00 | 1.56 $\pm$ 0.01 | 1.08 $\pm$ 0.00 | 2.05 $\pm$ 0.00 | 0.17 $\pm$ 0.00 | 2.49 $\pm$ 0.00 | 1.37 $\pm$ 0.00 |

**Table 17: MAE scores under the MAR setting across different levels of missingness.**
|  | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Missing $10\%$ |  |  |  |  |  |  |  |  |
| Mean | 2.06 $\pm$ .16 | 0.97 $\pm$ .08 | 1.95 $\pm$ .15 | 1.46 $\pm$ .14 | 3.44 $\pm$ .23 | 2.62 $\pm$ .20 | 2.50 $\pm$ .02 | 1.82 $\pm$ .16 |
| Svd | 2.61 $\pm$ .16 | 1.18 $\pm$ .07 | 2.26 $\pm$ .20 | 1.34 $\pm$ .16 | 3.37 $\pm$ .28 | 0.65 $\pm$ .19 | 2.64 $\pm$ .02 | 2.27 $\pm$ .29 |
| Spectral | 2.36 $\pm$ .32 | 0.86 $\pm$ .11 | 1.93 $\pm$ .18 | 1.19 $\pm$ .10 | 2.24 $\pm$ .35 | 0.45 $\pm$ .22 | 3.12 $\pm$ .04 | 2.15 $\pm$ .26 |
| Mice | 1.26 $\pm$ .17 | 0.65 $\pm$ .02 | 1.03 $\pm$ .08 | 1.05 $\pm$ .08 | 0.89 $\pm$ .09 | 0.18 $\pm$ .01 | 2.48 $\pm$ .01 | 0.98 $\pm$ .09 |
| Knn | 1.62 $\pm$ .51 | 0.51 $\pm$ .02 | 0.47 $\pm$ .10 | 0.74 $\pm$ .02 | 1.50 $\pm$ .10 | 0.03 $\pm$ .01 | 2.92 $\pm$ .03 | 0.84 $\pm$ .10 |
| Gain | 2.00 $\pm$ .12 | 0.79 $\pm$ .02 | 1.72 $\pm$ .04 | 0.91 $\pm$ .13 | 1.51 $\pm$ .06 | 0.54 $\pm$ .16 | 2.77 $\pm$ .03 | 1.24 $\pm$ .08 |
| Miwae | 2.06 $\pm$ .01 | 1.02 $\pm$ .00 | 1.88 $\pm$ .00 | 1.56 $\pm$ .01 | 3.82 $\pm$ .01 | 2.92 $\pm$ .00 | 2.57 $\pm$ .01 | 2.06 $\pm$ .01 |
| Grape | 0.75 $\pm$ .00 | 0.49 $\pm$ .01 | 0.42 $\pm$ .01 | 0.51 $\pm$ .02 | 0.89 $\pm$ .01 | 0.07 $\pm$ .02 | 2.46 $\pm$ .00 | 0.70 $\pm$ .03 |
| Miracle | 3.78 $\pm$ .00 | 0.58 $\pm$ .04 | 1.65 $\pm$ .00 | 2.21 $\pm$ .00 | 3.45 $\pm$ .00 | 0.16 $\pm$ .01 | 2.52 $\pm$ .01 | 0.81 $\pm$ .00 |
| HyperImpute | 1.22 $\pm$ .13 | 0.51 $\pm$ .01 | 0.49 $\pm$ .03 | 0.82 $\pm$ .16 | 0.70 $\pm$ .14 | 0.02 $\pm$ .02 | 2.56 $\pm$ .04 | 0.77 $\pm$ .06 |
| M^3-Impute | 0.70 $\pm$ .01 | 0.49 $\pm$ .00 | 0.39 $\pm$ .01 | 0.44 $\pm$ .01 | 0.85 $\pm$ .00 | 0.07 $\pm$ .00 | 2.46 $\pm$ .00 | 0.69 $\pm$ .01 |
|  | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| Missing $30\%$ |  |  |  |  |  |  |  |  |
| Mean | 2.20 $\pm$ .13 | 1.09 $\pm$ .05 | 1.79 $\pm$ .21 | 2.02 $\pm$ .20 | 3.26 $\pm$ .36 | 2.75 $\pm$ .11 | 2.49 $\pm$ .01 | 1.81 $\pm$ .25 |
| Svd | 2.64 $\pm$ .22 | 1.04 $\pm$ .14 | 2.32 $\pm$ .06 | 1.71 $\pm$ .15 | 3.68 $\pm$ .16 | 0.52 $\pm$ .11 | 2.69 $\pm$ .02 | 2.37 $\pm$ .62 |
| Spectral | 3.06 $\pm$ .11 | 0.91 $\pm$ .13 | 2.12 $\pm$ .17 | 1.84 $\pm$ .28 | 2.88 $\pm$ .35 | 1.29 $\pm$ .47 | 3.56 $\pm$ .01 | 3.37 $\pm$ .04 |
| Mice | 1.79 $\pm$ .10 | 0.79 $\pm$ .01 | 1.27 $\pm$ .08 | 1.22 $\pm$ .05 | 1.12 $\pm$ .07 | 0.27 $\pm$ .01 | 2.51 $\pm$ .03 | 1.16 $\pm$ .11 |
| Knn | 1.69 $\pm$ .07 | 0.66 $\pm$ .07 | 0.89 $\pm$ .30 | 0.89 $\pm$ .12 | 1.61 $\pm$ .35 | 0.07 $\pm$ .00 | 2.94 $\pm$ .01 | 1.11 $\pm$ .04 |
| Gain | 2.07 $\pm$ .02 | 1.13 $\pm$ .20 | 1.87 $\pm$ .16 | 0.92 $\pm$ .05 | 2.26 $\pm$ .14 | 0.91 $\pm$ .07 | 2.93 $\pm$ .02 | 1.42 $\pm$ .01 |
| Miwae | 2.17 $\pm$ .01 | 0.98 $\pm$ .01 | 1.80 $\pm$ .04 | 1.54 $\pm$ .00 | 3.91 $\pm$ .01 | 2.80 $\pm$ .01 | 2.58 $\pm$ .01 | 2.05 $\pm$ .02 |
| Grape | 1.20 $\pm$ .03 | 0.60 $\pm$ .00 | 0.77 $\pm$ .02 | 0.66 $\pm$ .01 | 1.05 $\pm$ .02 | 0.07 $\pm$ .01 | 2.49 $\pm$ .00 | 1.06 $\pm$ .04 |
| Miracle | 3.75 $\pm$ .00 | 0.70 $\pm$ .00 | 1.94 $\pm$ .00 | 2.24 $\pm$ .00 | 3.89 $\pm$ .00 | 0.36 $\pm$ .00 | 2.82 $\pm$ .10 | 0.86 $\pm$ .01 |
| Hyperimpute | 2.06 $\pm$ .12 | 0.78 $\pm$ .06 | 1.30 $\pm$ .15 | 1.05 $\pm$ .21 | 1.11 $\pm$ .38 | 1.01 $\pm$ .18 | 3.07 $\pm$ .06 | 1.07 $\pm$ .14 |
| M^3-Impute | 1.09 $\pm$ .03 | 0.60 $\pm$ .00 | 0.77 $\pm$ .02 | 0.60 $\pm$ .00 | 0.98 $\pm$ .02 | 0.07 $\pm$ .00 | 2.49 $\pm$ .00 | 1.01 $\pm$ .00 |
|  | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| Missing $50\%$ |  |  |  |  |  |  |  |  |
| Mean | 2.20 $\pm$ .13 | 0.98 $\pm$ .06 | 1.74 $\pm$ .09 | 1.92 $\pm$ .05 | 3.30 $\pm$ .12 | 2.80 $\pm$ .05 | 2.49 $\pm$ .01 | 1.86 $\pm$ .03 |
| Svd | 2.95 $\pm$ .11 | 1.06 $\pm$ .04 | 2.63 $\pm$ .42 | 1.78 $\pm$ .14 | 3.69 $\pm$ .14 | 0.59 $\pm$ .14 | 2.83 $\pm$ .03 | 2.64 $\pm$ .27 |
| Spectral | 3.41 $\pm$ .09 | 1.35 $\pm$ .02 | 2.14 $\pm$ .02 | 2.10 $\pm$ .30 | 3.62 $\pm$ .16 | 1.98 $\pm$ .25 | 3.84 $\pm$ .01 | 4.02 $\pm$ .32 |
| Mice | 2.15 $\pm$ .09 | 0.87 $\pm$ .05 | 1.56 $\pm$ .08 | 1.45 $\pm$ .03 | 1.96 $\pm$ .04 | 0.25 $\pm$ .08 | 2.61 $\pm$ .08 | 1.35 $\pm$ .14 |
| Knn | 2.45 $\pm$ .19 | 0.90 $\pm$ .09 | 1.04 $\pm$ .11 | 1.14 $\pm$ .26 | 1.64 $\pm$ .39 | 0.07 $\pm$ .03 | 3.00 $\pm$ .01 | 1.44 $\pm$ .23 |
| Gain | 3.40 $\pm$ .08 | 1.60 $\pm$ .33 | 2.13 $\pm$ .24 | 1.95 $\pm$ .10 | 3.04 $\pm$ .35 | 1.02 $\pm$ .05 | 3.08 $\pm$ .08 | 1.69 $\pm$ .09 |
| Miwae | 2.31 $\pm$ .01 | 1.09 $\pm$ .03 | 1.74 $\pm$ .03 | 1.84 $\pm$ .03 | 3.45 $\pm$ .00 | 2.89 $\pm$ .01 | 2.57 $\pm$ .01 | 1.95 $\pm$ .01 |
| Grape | 2.07 $\pm$ .00 | 0.81 $\pm$ .00 | 1.17 $\pm$ .01 | 0.91 $\pm$ .02 | 1.73 $\pm$ .03 | 0.10 $\pm$ .01 | 2.49 $\pm$ .00 | 1.35 $\pm$ .00 |
| Miracle | 3.98 $\pm$ .00 | 0.87 $\pm$ .00 | 2.50 $\pm$ .00 | 2.41 $\pm$ .00 | 4.08 $\pm$ .00 | 0.63 $\pm$ .00 | 2.86 $\pm$ .00 | 1.55 $\pm$ .00 |
| HyperImpute | 2.47 $\pm$ .12 | 0.81 $\pm$ .06 | 1.60 $\pm$ .03 | 1.01 $\pm$ .09 | 1.50 $\pm$ .22 | 0.04 $\pm$ .02 | 3.19 $\pm$ .01 | 1.27 $\pm$ .15 |
| M^3-Impute | 2.07 $\pm$ .00 | 0.81 $\pm$ .00 | 1.17 $\pm$ .01 | 0.84 $\pm$ .00 | 1.64 $\pm$ .03 | 0.10 $\pm$ .00 | 2.49 $\pm$ .00 | 1.35 $\pm$ .00 |
|  | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| Missing $70\%$ |  |  |  |  |  |  |  |  |
| Mean | 2.13 $\pm$ .12 | 1.01 $\pm$ .03 | 1.85 $\pm$ .03 | 1.91 $\pm$ .06 | 3.19 $\pm$ .07 | 3.11 $\pm$ .48 | 2.49 $\pm$ .01 | 1.79 $\pm$ .07 |
| Svd | 3.10 $\pm$ .21 | 1.36 $\pm$ .03 | 2.47 $\pm$ .20 | 2.44 $\pm$ .21 | 4.33 $\pm$ .38 | 0.82 $\pm$ .03 | 3.13 $\pm$ .03 | 2.70 $\pm$ .22 |
| Spectral | 3.68 $\pm$ .27 | 1.49 $\pm$ .17 | 2.55 $\pm$ .22 | 2.49 $\pm$ .15 | 3.97 $\pm$ .01 | 2.82 $\pm$ .69 | 4.10 $\pm$ .01 | 4.23 $\pm$ .34 |
| Mice | 2.28 $\pm$ .12 | 0.97 $\pm$ .03 | 1.76 $\pm$ .08 | 1.81 $\pm$ .01 | 2.81 $\pm$ .18 | 0.50 $\pm$ .03 | 2.62 $\pm$ .02 | 1.38 $\pm$ .17 |
| Knn | 2.02 $\pm$ .34 | 1.15 $\pm$ .05 | 1.54 $\pm$ .06 | 1.63 $\pm$ .25 | 1.65 $\pm$ .28 | 0.19 $\pm$ .04 | 2.98 $\pm$ .01 | 1.23 $\pm$ .06 |
| Gain | 3.64 $\pm$ .27 | 1.94 $\pm$ .05 | 2.59 $\pm$ .11 | 2.74 $\pm$ .09 | 4.40 $\pm$ .04 | 0.69 $\pm$ .05 | 4.00 $\pm$ .04 | 2.41 $\pm$ .42 |
| Miwae | 2.31 $\pm$ .01 | 1.08 $\pm$ .01 | 1.89 $\pm$ .00 | 1.84 $\pm$ .02 | 3.18 $\pm$ .00 | 2.95 $\pm$ .00 | 2.58 $\pm$ .00 | 1.94 $\pm$ .01 |
| Grape | 2.06 $\pm$ .00 | 0.94 $\pm$ .01 | 1.69 $\pm$ .03 | 1.20 $\pm$ .01 | 2.23 $\pm$ .02 | 0.17 $\pm$ .00 | 2.49 $\pm$ .00 | 1.42 $\pm$ .00 |
| Miracle | 3.99 $\pm$ .00 | 0.96 $\pm$ .00 | 2.70 $\pm$ .04 | 2.83 $\pm$ .00 | 3.82 $\pm$ .00 | 0.24 $\pm$ .01 | 2.89 $\pm$ .00 | 1.61 $\pm$ .00 |
| HyperImpute | 2.56 $\pm$ .04 | 0.96 $\pm$ .02 | 1.93 $\pm$ .04 | 1.28 $\pm$ .06 | 2.43 $\pm$ .12 | 0.12 $\pm$ .09 | 3.22 $\pm$ .02 | 1.36 $\pm$ .18 |
| M^3-Impute | 2.06 $\pm$ .00 | 0.92 $\pm$ .00 | 1.68 $\pm$ .01 | 1.13 $\pm$ .01 | 2.16 $\pm$ .00 | 0.17 $\pm$ .00 | 2.49 $\pm$ .00 | 1.42 $\pm$ .00 |

**Table 18: MAE scores under the MNAR setting across different levels of missingness.**
|  | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Missing $10\%$ |  |  |  |  |  |  |  |  |
| Mean | 2.20 $\pm$ .10 | 0.97 $\pm$ .02 | 1.88 $\pm$ .01 | 1.75 $\pm$ .12 | 3.12 $\pm$ .07 | 2.48 $\pm$ .05 | 2.51 $\pm$ .01 | 1.72 $\pm$ .05 |
| Svd | 2.64 $\pm$ .01 | 1.09 $\pm$ .06 | 2.14 $\pm$ .03 | 1.31 $\pm$ .03 | 3.45 $\pm$ .23 | 0.75 $\pm$ .03 | 2.63 $\pm$ .01 | 2.08 $\pm$ .05 |
| Spectral | 2.23 $\pm$ .03 | 0.83 $\pm$ .05 | 1.97 $\pm$ .02 | 1.18 $\pm$ .05 | 1.86 $\pm$ .05 | 0.30 $\pm$ .05 | 3.00 $\pm$ .01 | 2.36 $\pm$ .22 |
| Mice | 1.41 $\pm$ .05 | 0.65 $\pm$ .01 | 1.07 $\pm$ .03 | 0.93 $\pm$ .01 | 1.33 $\pm$ .14 | 0.12 $\pm$ .00 | 2.51 $\pm$ .02 | 1.04 $\pm$ .02 |
| Knn | 1.44 $\pm$ .14 | 0.53 $\pm$ .03 | 0.54 $\pm$ .04 | 0.60 $\pm$ .04 | 1.64 $\pm$ .11 | 0.03 $\pm$ .00 | 3.00 $\pm$ .03 | 1.55 $\pm$ .02 |
| Gain | 2.39 $\pm$ .03 | 0.86 $\pm$ .01 | 1.66 $\pm$ .04 | 1.05 $\pm$ .08 | 1.94 $\pm$ .04 | 0.42 $\pm$ .01 | 2.74 $\pm$ .01 | 1.23 $\pm$ .02 |
| Miwae | 2.23 $\pm$ .01 | 1.01 $\pm$ .02 | 1.92 $\pm$ .03 | 1.50 $\pm$ .01 | 3.25 $\pm$ .01 | 2.50 $\pm$ .01 | 2.59 $\pm$ .00 | 1.83 $\pm$ .01 |
| Grape | 1.13 $\pm$ .01 | 0.49 $\pm$ .00 | 0.46 $\pm$ .01 | 0.55 $\pm$ .01 | 1.14 $\pm$ .00 | 0.04 $\pm$ .00 | 2.51 $\pm$ .00 | 0.88 $\pm$ .00 |
| Miracle | 3.87 $\pm$ .00 | 0.62 $\pm$ .00 | 1.70 $\pm$ .04 | 2.51 $\pm$ .00 | 4.05 $\pm$ .00 | 0.11 $\pm$ .01 | 2.51 $\pm$ .02 | 1.01 $\pm$ .03 |
| HyperImpute | 1.51 $\pm$ .07 | 0.55 $\pm$ .01 | 0.58 $\pm$ .02 | 0.73 $\pm$ .04 | 1.11 $\pm$ .06 | 0.03 $\pm$ .00 | 2.51 $\pm$ .01 | 0.85 $\pm$ .00 |
| M^3-Impute | 1.08 $\pm$ .00 | 0.49 $\pm$ .00 | 0.44 $\pm$ .01 | 0.50 $\pm$ .01 | 1.10 $\pm$ .00 | 0.04 $\pm$ .00 | 2.51 $\pm$ .00 | 0.84 $\pm$ .01 |
|  | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| Missing $30\%$ |  |  |  |  |  |  |  |  |
| Mean | 2.18 $\pm$ .09 | 1.04 $\pm$ .02 | 1.80 $\pm$ .09 | 1.95 $\pm$ .13 | 3.17 $\pm$ .22 | 2.60 $\pm$ .07 | 2.49 $\pm$ .01 | 1.76 $\pm$ .14 |
| Svd | 2.61 $\pm$ .13 | 1.06 $\pm$ .07 | 2.24 $\pm$ .05 | 1.58 $\pm$ .06 | 3.55 $\pm$ .09 | 0.53 $\pm$ .05 | 2.69 $\pm$ .02 | 2.27 $\pm$ .25 |
| Spectral | 2.75 $\pm$ .14 | 1.01 $\pm$ .08 | 1.86 $\pm$ .03 | 1.60 $\pm$ .22 | 2.50 $\pm$ .15 | 1.35 $\pm$ .21 | 3.34 $\pm$ .00 | 3.14 $\pm$ .41 |
| Mice | 1.91 $\pm$ .10 | 0.77 $\pm$ .07 | 1.37 $\pm$ .05 | 1.22 $\pm$ .06 | 1.57 $\pm$ .03 | 0.21 $\pm$ .07 | 2.50 $\pm$ .00 | 1.08 $\pm$ .02 |
| Knn | 1.92 $\pm$ .10 | 0.75 $\pm$ .05 | 1.15 $\pm$ .32 | 0.95 $\pm$ .11 | 1.96 $\pm$ .11 | 0.08 $\pm$ .02 | 3.06 $\pm$ .02 | 1.65 $\pm$ .07 |
| Gain | 2.34 $\pm$ .12 | 0.92 $\pm$ .05 | 1.80 $\pm$ .05 | 1.08 $\pm$ .05 | 1.92 $\pm$ .06 | 1.12 $\pm$ .03 | 2.78 $\pm$ .03 | 1.22 $\pm$ .03 |
| Miwae | 2.17 $\pm$ .00 | 0.99 $\pm$ .01 | 1.81 $\pm$ .03 | 1.60 $\pm$ .02 | 3.63 $\pm$ .00 | 2.63 $\pm$ .03 | 2.55 $\pm$ .02 | 1.95 $\pm$ .03 |
| Grape | 1.23 $\pm$ .03 | 0.61 $\pm$ .00 | 0.73 $\pm$ .01 | 0.61 $\pm$ .01 | 1.16 $\pm$ .01 | 0.08 $\pm$ .01 | 2.46 $\pm$ .01 | 1.02 $\pm$ .01 |
| Miracle | 3.85 $\pm$ .00 | 0.70 $\pm$ .00 | 1.87 $\pm$ .00 | 2.51 $\pm$ .00 | 3.86 $\pm$ .00 | 0.30 $\pm$ .00 | 2.64 $\pm$ .00 | 1.06 $\pm$ .00 |
| Hyperimpute | 1.95 $\pm$ .10 | 0.72 $\pm$ .03 | 0.88 $\pm$ .02 | 0.85 $\pm$ .03 | 1.19 $\pm$ .24 | 0.85 $\pm$ .04 | 2.71 $\pm$ .06 | 1.09 $\pm$ .06 |
| M^3-Impute | 1.15 $\pm$ .02 | 0.60 $\pm$ .00 | 0.68 $\pm$ .02 | 0.54 $\pm$ .01 | 1.09 $\pm$ .01 | 0.08 $\pm$ .00 | 2.46 $\pm$ .00 | 1.00 $\pm$ .00 |
|  | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| Missing $50\%$ |  |  |  |  |  |  |  |  |
| Mean | 2.17 $\pm$ .08 | 1.01 $\pm$ .06 | 1.85 $\pm$ .09 | 1.89 $\pm$ .13 | 3.27 $\pm$ .10 | 2.66 $\pm$ .11 | 2.49 $\pm$ .01 | 1.77 $\pm$ .06 |
| Svd | 3.08 $\pm$ .10 | 1.15 $\pm$ .12 | 2.46 $\pm$ .15 | 1.90 $\pm$ .16 | 3.56 $\pm$ .25 | 0.58 $\pm$ .09 | 2.83 $\pm$ .02 | 2.62 $\pm$ .12 |
| Spectral | 3.38 $\pm$ .04 | 1.31 $\pm$ .02 | 2.36 $\pm$ .15 | 1.91 $\pm$ .11 | 3.49 $\pm$ .29 | 1.99 $\pm$ .08 | 3.82 $\pm$ .01 | 4.19 $\pm$ .35 |
| Mice | 2.17 $\pm$ .07 | 0.90 $\pm$ .07 | 1.59 $\pm$ .06 | 1.50 $\pm$ .14 | 2.39 $\pm$ .35 | 0.27 $\pm$ .02 | 2.62 $\pm$ .02 | 1.35 $\pm$ .09 |
| Knn | 2.31 $\pm$ .10 | 1.02 $\pm$ .07 | 1.35 $\pm$ .10 | 1.55 $\pm$ .33 | 2.51 $\pm$ .19 | 0.39 $\pm$ .18 | 3.03 $\pm$ .01 | 1.69 $\pm$ .26 |
| Gain | 3.22 $\pm$ .07 | 1.26 $\pm$ .12 | 1.96 $\pm$ .08 | 1.76 $\pm$ .07 | 3.16 $\pm$ .13 | 0.74 $\pm$ .09 | 3.12 $\pm$ .13 | 1.72 $\pm$ .24 |
| Miwae | 2.32 $\pm$ .00 | 1.08 $\pm$ .02 | 1.78 $\pm$ .02 | 1.87 $\pm$ .02 | 3.40 $\pm$ .00 | 2.80 $\pm$ .02 | 2.58 $\pm$ .00 | 1.91 $\pm$ .03 |
| Grape | 2.09 $\pm$ .00 | 0.82 $\pm$ .00 | 1.23 $\pm$ .00 | 0.90 $\pm$ .01 | 1.69 $\pm$ .01 | 0.10 $\pm$ .00 | 2.49 $\pm$ .00 | 1.33 $\pm$ .00 |
| Miracle | 3.97 $\pm$ .00 | 0.88 $\pm$ .00 | 2.43 $\pm$ .00 | 2.53 $\pm$ .00 | 3.81 $\pm$ .00 | 0.59 $\pm$ .00 | 2.84 $\pm$ .05 | 1.55 $\pm$ .00 |
| HyperImpute | 2.39 $\pm$ .10 | 0.83 $\pm$ .06 | 1.56 $\pm$ .04 | 0.99 $\pm$ .06 | 1.62 $\pm$ .18 | 0.05 $\pm$ .02 | 3.17 $\pm$ .05 | 1.29 $\pm$ .12 |
| M^3-Impute | 2.09 $\pm$ .00 | 0.81 $\pm$ .00 | 1.21 $\pm$ .01 | 0.87 $\pm$ .01 | 1.62 $\pm$ .01 | 0.10 $\pm$ .00 | 2.48 $\pm$ .00 | 1.32 $\pm$ .00 |
|  | Yacht | Wine | Concrete | Housing | Energy | Naval | Kin8nm | Power |
| Missing $70\%$ |  |  |  |  |  |  |  |  |
| Mean | 2.25 $\pm$ .21 | 1.01 $\pm$ .02 | 1.83 $\pm$ .08 | 1.96 $\pm$ .10 | 3.22 $\pm$ .13 | 2.81 $\pm$ .08 | 2.49 $\pm$ .00 | 1.78 $\pm$ .05 |
| Svd | 3.11 $\pm$ .05 | 1.38 $\pm$ .03 | 2.68 $\pm$ .21 | 2.32 $\pm$ .15 | 4.19 $\pm$ .30 | 1.79 $\pm$ .39 | 3.16 $\pm$ .03 | 2.87 $\pm$ .22 |
| Spectral | 3.63 $\pm$ .10 | 1.68 $\pm$ .11 | 2.73 $\pm$ .04 | 2.29 $\pm$ .26 | 3.68 $\pm$ .18 | 2.83 $\pm$ .66 | 4.15 $\pm$ .01 | 4.35 $\pm$ .30 |
| Mice | 2.22 $\pm$ .17 | 0.93 $\pm$ .03 | 1.80 $\pm$ .03 | 1.75 $\pm$ .15 | 2.66 $\pm$ .29 | 0.56 $\pm$ .04 | 2.64 $\pm$ .02 | 1.41 $\pm$ .13 |
| Knn | 2.09 $\pm$ .31 | 1.15 $\pm$ .03 | 1.92 $\pm$ .13 | 1.80 $\pm$ .23 | 2.36 $\pm$ .39 | 0.83 $\pm$ .10 | 3.02 $\pm$ .00 | 1.60 $\pm$ .07 |
| Gain | 3.70 $\pm$ .31 | 1.66 $\pm$ .19 | 2.57 $\pm$ .09 | 2.77 $\pm$ .14 | 4.27 $\pm$ .08 | 0.42 $\pm$ .01 | 3.56 $\pm$ .27 | 1.89 $\pm$ .28 |
| Miwae | 2.32 $\pm$ .01 | 1.07 $\pm$ .02 | 1.90 $\pm$ .03 | 1.83 $\pm$ .02 | 3.17 $\pm$ .02 | 2.92 $\pm$ .03 | 2.57 $\pm$ .00 | 1.90 $\pm$ .00 |
| Grape | 2.08 $\pm$ .00 | 0.94 $\pm$ .01 | 1.73 $\pm$ .03 | 1.22 $\pm$ .01 | 2.27 $\pm$ .01 | 0.18 $\pm$ .00 | 2.49 $\pm$ .00 | 1.43 $\pm$ .00 |
| Miracle | 3.99 $\pm$ .00 | 0.95 $\pm$ .00 | 2.66 $\pm$ .02 | 2.88 $\pm$ .00 | 2.85 $\pm$ .00 | 0.91 $\pm$ .01 | 2.82 $\pm$ .00 | 1.61 $\pm$ .00 |
| HyperImpute | 2.42 $\pm$ .16 | 0.94 $\pm$ .03 | 1.86 $\pm$ .05 | 1.22 $\pm$ .02 | 2.41 $\pm$ .06 | 0.14 $\pm$ .00 | 3.19 $\pm$ .02 | 1.36 $\pm$ .13 |
| M^3-Impute | 2.08 $\pm$ .00 | 0.92 $\pm$ .00 | 1.70 $\pm$ .02 | 1.15 $\pm$ .02 | 2.19 $\pm$ .02 | 0.18 $\pm$ .00 | 2.49 $\pm$ .00 | 1.41 $\pm$ .00 |

**Table 19: MAE scores under the MCAR setting across different levels of missingness on the extra 17 datasets. AI is short for AIrfoil. Please refer to Table 9 for dataset names.**
| Dataset | Mean | Knn | Svd | Mice | Spectral | HI | Gain | Miracle | Miwae | Grape | M^3-Impute |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Missing 10% |  |  |  |  |  |  |  |  |  |  |  |
| AI | 2.32 $\pm$ .03 | 1.59 $\pm$ .04 | 2.67 $\pm$ .08 | 1.83 $\pm$ .01 | 2.16 $\pm$ .03 | 0.59 $\pm$ .03 | 2.18 $\pm$ .02 | 1.81 $\pm$ .01 | 2.30 $\pm$ .02 | 0.61 $\pm$ .00 | 0.55 $\pm$ .01 |
| BL | 1.16 $\pm$ .02 | 0.96 $\pm$ .03 | 0.90 $\pm$ .02 | 0.61 $\pm$ .02 | 0.90 $\pm$ .05 | 0.53 $\pm$ .02 | 1.26 $\pm$ .07 | 1.48 $\pm$ .05 | 1.98 $\pm$ .05 | 0.51 $\pm$ .00 | 0.51 $\pm$ .00 |
| WW | 0.76 $\pm$ .01 | 0.38 $\pm$ .00 | 0.86 $\pm$ .01 | 0.50 $\pm$ .00 | 0.69 $\pm$ .01 | 0.44 $\pm$ .01 | 0.70 $\pm$ .02 | 0.49 $\pm$ .00 | 0.78 $\pm$ .01 | 0.43 $\pm$ .00 | 0.42 $\pm$ .00 |
| IO | 2.00 $\pm$ .02 | 1.02 $\pm$ .03 | 1.21 $\pm$ .05 | 1.20 $\pm$ .06 | 1.30 $\pm$ .03 | 1.14 $\pm$ .04 | 1.39 $\pm$ .06 | 5.66 $\pm$ .00 | 5.12 $\pm$ .02 | 1.04 $\pm$ .01 | 0.96 $\pm$ .02 |
| BR | 1.08 $\pm$ .02 | 0.50 $\pm$ .01 | 0.57 $\pm$ .00 | 0.25$\pm$ .01 | 0.31 $\pm$ .00 | 0.28 $\pm$ .00 | 0.47 $\pm$ .02 | 1.40 $\pm$ .00 | 1.90 $\pm$ .02 | 0.31 $\pm$ .00 | 0.30 $\pm$ .01 |
| IR | 2.18 $\pm$ .09 | 1.25 $\pm$ .22 | 1.67 $\pm$ .25 | 0.94 $\pm$ .13 | 1.43 $\pm$ .22 | 0.97 $\pm$ .12 | 1.27 $\pm$ .17 | 3.20 $\pm$ .00 | 4.80 $\pm$ .25 | 0.71 $\pm$ .01 | 0.69 $\pm$ .02 |
| DI | 1.80 $\pm$ .11 | 1.51 $\pm$ .07 | 1.68 $\pm$ .04 | 1.17 $\pm$ .08 | 1.38 $\pm$ .07 | 1.09 $\pm$ .04 | 1.47 $\pm$ .12 | 2.82 $\pm$ .00 | 5.23 $\pm$ .16 | 1.12 $\pm$ .02 | 1.08 $\pm$ .02 |
| PR | 0.91 $\pm$ .01 | 0.26 $\pm$ .00 | 0.98 $\pm$ .00 | 0.28 $\pm$ .00 | 0.80 $\pm$ .00 | 0.20 $\pm$ .00 | 0.51 $\pm$ .00 | 0.27 $\pm$ .00 | 0.94 $\pm$ .01 | 0.22 $\pm$ .02 | 0.20 $\pm$ .00 |
| SP | 0.23 $\pm$ .00 | 0.14 $\pm$ .01 | 0.30 $\pm$ .01 | 0.19 $\pm$ .00 | 0.16 $\pm$ .00 | 0.17 $\pm$ .02 | 0.22 $\pm$ .01 | 0.19 $\pm$ .01 | 0.16 $\pm$ .00 | 0.17 $\pm$ .00 | 0.16 $\pm$ .01 |
| LE | 1.28 $\pm$ .01 | 0.32 $\pm$ .00 | 1.25 $\pm$ .01 | 0.88 $\pm$ .00 | 1.26 $\pm$ .01 | 0.49 $\pm$ .00 | 1.07 $\pm$ .01 | 0.76 $\pm$ .01 | 1.37 $\pm$ .01 | 0.42 $\pm$ .00 | 0.41 $\pm$ .01 |
| AB | 2.50 $\pm$ .03 | 2.06 $\pm$ .06 | 2.41 $\pm$ .04 | 2.01 $\pm$ .02 | 2.29 $\pm$ .02 | 1.71 $\pm$ .14 | 2.23 $\pm$ .05 | 3.68 $\pm$ .00 | 2.57 $\pm$ .03 | 1.84 $\pm$ .03 | 1.71 $\pm$ .02 |
| A4 | 1.06 $\pm$ .02 | 1.08 $\pm$ .02 | 1.14 $\pm$ .02 | 0.78 $\pm$ .02 | 1.22 $\pm$ .02 | 0.67 $\pm$ .01 | 1.23 $\pm$ .02 | 0.81 $\pm$ .04 | 1.09 $\pm$ .04 | 0.71 $\pm$ .01 | 0.70 $\pm$ .00 |
| CM | 2.31 $\pm$ .07 | 2.19 $\pm$ .11 | 2.43 $\pm$ .10 | 1.97 $\pm$ .08 | 2.62 $\pm$ .05 | 1.98 $\pm$ .15 | 2.35 $\pm$ .17 | 1.86 $\pm$ .00 | 2.35 $\pm$ .08 | 1.75 $\pm$ .01 | 1.68 $\pm$ .02 |
| GE | 2.50 $\pm$ .03 | 2.06 $\pm$ .06 | 2.41 $\pm$ .04 | 2.01 $\pm$ .02 | 2.29 $\pm$ .02 | 1.71 $\pm$ .14 | 2.23 $\pm$ .05 | 3.68 $\pm$ .00 | 2.57 $\pm$ .03 | 1.85 $\pm$ .02 | 1.68 $\pm$ .02 |
| ST | 1.78 $\pm$ .02 | 0.55 $\pm$ .00 | 1.27 $\pm$ .01 | 0.70 $\pm$ .02 | 0.87 $\pm$ .01 | 0.59 $\pm$ .00 | 0.96 $\pm$ .02 | 0.74 $\pm$ .00 | 1.68 $\pm$ .02 | 0.34 $\pm$ .01 | 0.29 $\pm$ .01 |
| LI | 1.82 $\pm$ .01 | 0.19 $\pm$ .01 | 0.36 $\pm$ .00 | 0.05 $\pm$ .00 | 0.14 $\pm$ .00 | 0.06 $\pm$ .00 | 0.37 $\pm$ .00 | 6.05 $\pm$ .00 | 2.16 $\pm$ .01 | 0.10 $\pm$ .00 | 0.10 $\pm$ .00 |
| CH | 1.13 $\pm$ .00 | 0.78 $\pm$ .01 | 1.30 $\pm$ .00 | 0.56 $\pm$ .00 | 1.26 $\pm$ .01 | 0.42 $\pm$ .00 | 1.07 $\pm$ .03 | 0.53 $\pm$ .00 | 1.15 $\pm$ .01 | 0.40 $\pm$ .00 | 0.40 $\pm$ .00 |
| Missing 30% |  |  |  |  |  |  |  |  |  |  |  |
| AI | 2.32 $\pm$ .05 | 2.18 $\pm$ .04 | 2.76 $\pm$ .05 | 1.97 $\pm$ .04 | 2.30 $\pm$ .07 | 1.09 $\pm$ .02 | 2.22 $\pm$ .06 | 1.97 $\pm$ .00 | 2.36 $\pm$ .06 | 1.16 $\pm$ .02 | 1.09 $\pm$ .03 |
| BL | 1.14 $\pm$ .01 | 0.93 $\pm$ .01 | 0.97 $\pm$ .04 | 0.69 $\pm$ .01 | 0.94 $\pm$ .03 | 0.63 $\pm$ .02 | 1.26 $\pm$ .04 | 1.50 $\pm$ .00 | 2.03 $\pm$ .05 | 0.68 $\pm$ .00 | 0.67 $\pm$ .00 |
| WW | 0.76 $\pm$ .00 | 0.64 $\pm$ .01 | 0.87 $\pm$ .00 | 0.61 $\pm$ .01 | 0.78 $\pm$ .01 | 0.55 $\pm$ .00 | 0.73 $\pm$ .01 | 0.58 $\pm$ .00 | 0.77 $\pm$ .00 | 0.52 $\pm$ .00 | 0.52 $\pm$ .00 |
| IO | 2.01 $\pm$ .03 | 1.07 $\pm$ .03 | 1.26 $\pm$ .03 | 1.37 $\pm$ .03 | 1.38 $\pm$ .02 | 1.18 $\pm$ .04 | 1.50 $\pm$ .01 | 5.56 $\pm$ .00 | 5.14 $\pm$ .06 | 1.08 $\pm$ .01 | 1.01 $\pm$ .01 |
| BR | 1.06 $\pm$ .00 | 0.53 $\pm$ .01 | 0.58 $\pm$ .00 | 0.34 $\pm$ .01 | 0.38 $\pm$ .00 | 0.33 $\pm$ .01 | 0.51 $\pm$ .01 | 1.42 $\pm$ .00 | 1.89 $\pm$ .02 | 0.37 $\pm$ .00 | 0.36 $\pm$ .01 |
| IR | 2.15 $\pm$ .09 | 1.54 $\pm$ .22 | 1.70 $\pm$ .07 | 1.07 $\pm$ .09 | 1.48 $\pm$ .13 | 1.04 $\pm$ .11 | 1.29 $\pm$ .07 | 3.22 $\pm$ .00 | 4.60 $\pm$ .17 | 0.82 $\pm$ .00 | 0.82 $\pm$ .00 |
| DI | 1.78 $\pm$ .03 | 1.71 $\pm$ .04 | 1.76 $\pm$ .02 | 1.29 $\pm$ .05 | 1.48 $\pm$ .03 | 1.17 $\pm$ .02 | 1.47 $\pm$ .06 | 2.69 $\pm$ .00 | 5.05 $\pm$ .04 | 1.31 $\pm$ .00 | 1.29 $\pm$ .01 |
| PR | 0.91 $\pm$ .00 | 0.58 $\pm$ .01 | 1.00 $\pm$ .00 | 0.33 $\pm$ .00 | 1.14 $\pm$ .00 | 0.25 $\pm$ .00 | 0.72 $\pm$ .06 | 0.32 $\pm$ .00 | 0.94 $\pm$ .00 | 0.25 $\pm$ .02 | 0.24 $\pm$ .00 |
| SP | 0.23 $\pm$ .00 | 0.17 $\pm$ .00 | 0.31 $\pm$ .00 | 0.22 $\pm$ .00 | 0.16 $\pm$ .00 | 0.18 $\pm$ .01 | 0.21 $\pm$ .00 | 0.19 $\pm$ .00 | 0.16 $\pm$ .00 | 0.17 $\pm$ .01 | 0.16 $\pm$ .00 |
| LE | 1.28 $\pm$ .00 | 0.89 $\pm$ .01 | 1.29 $\pm$ .00 | 1.00 $\pm$ .00 | 1.75 $\pm$ .01 | 0.61 $\pm$ .01 | 1.09 $\pm$ .01 | 1.06 $\pm$ .01 | 1.33 $\pm$ .04 | 0.53 $\pm$ .00 | 0.52 $\pm$ .00 |
| AB | 2.52 $\pm$ .02 | 2.34 $\pm$ .02 | 2.60 $\pm$ .04 | 2.26 $\pm$ .04 | 2.45 $\pm$ .02 | 2.05 $\pm$ .25 | 2.27 $\pm$ .09 | 3.67 $\pm$ .00 | 2.59 $\pm$ .01 | 2.01 $\pm$ .00 | 1.84 $\pm$ .02 |
| A4 | 1.07 $\pm$ .00 | 1.17 $\pm$ .00 | 1.18 $\pm$ .01 | 0.87 $\pm$ .01 | 1.58 $\pm$ .01 | 0.75 $\pm$ .02 | 1.03 $\pm$ .02 | 0.84 $\pm$ .02 | 1.12 $\pm$ .01 | 0.79 $\pm$ .00 | 0.76 $\pm$ .00 |
| CM | 2.35 $\pm$ .03 | 2.32 $\pm$ .04 | 2.52 $\pm$ .05 | 2.06 $\pm$ .02 | 2.96 $\pm$ .01 | 1.91 $\pm$ .03 | 2.33 $\pm$ .15 | 2.01 $\pm$ .00 | 2.37 $\pm$ .04 | 1.87 $\pm$ .00 | 1.81 $\pm$ .01 |
| GE | 2.52 $\pm$ .02 | 2.34 $\pm$ .02 | 2.60 $\pm$ .04 | 2.26 $\pm$ .04 | 2.45 $\pm$ .02 | 2.05 $\pm$ .25 | 2.27 $\pm$ .09 | 3.67 $\pm$ .00 | 2.59 $\pm$ .01 | 2.01 $\pm$ .01 | 1.87 $\pm$ .02 |
| ST | 1.80 $\pm$ .01 | 0.78 $\pm$ .02 | 1.37 $\pm$ .01 | 0.95 $\pm$ .02 | 1.10 $\pm$ .01 | 0.72 $\pm$ .01 | 1.03 $\pm$ .02 | 0.95 $\pm$ .03 | 1.70 $\pm$ .01 | 0.45 $\pm$ .00 | 0.39 $\pm$ .00 |
| LI | 1.82 $\pm$ .01 | 0.25 $\pm$ .00 | 0.37 $\pm$ .00 | 0.11 $\pm$ .00 | 0.18 $\pm$ .00 | 0.11 $\pm$ .00 | 0.46 $\pm$ .00 | 5.61 $\pm$ .00 | 2.13 $\pm$ .02 | 0.10 $\pm$ .00 | 0.10 $\pm$ .00 |
| CH | 1.13 $\pm$ .00 | 1.17 $\pm$ .00 | 1.35 $\pm$ .01 | 0.69 $\pm$ .00 | 1.50 $\pm$ .00 | 0.57 $\pm$ .01 | 1.07 $\pm$ .03 | 0.67 $\pm$ .00 | 1.16 $\pm$ .00 | 0.54 $\pm$ .00 | 0.54 $\pm$ .01 |
| Missing 50% |  |  |  |  |  |  |  |  |  |  |  |
| AI | 2.32 $\pm$ .02 | 2.30 $\pm$ .04 | 2.93 $\pm$ .01 | 2.17 $\pm$ .04 | 2.45 $\pm$ .06 | 1.59 $\pm$ .04 | 2.30 $\pm$ .04 | 2.17 $\pm$ .05 | 2.34 $\pm$ .04 | 1.68 $\pm$ .01 | 1.66 $\pm$ .02 |
| BL | 1.15 $\pm$ .02 | 1.14 $\pm$ .06 | 1.12 $\pm$ .02 | 0.86 $\pm$ .04 | 1.11 $\pm$ .02 | 0.82 $\pm$ .02 | 1.31 $\pm$ .07 | 1.48 $\pm$ .09 | 1.98 $\pm$ .01 | 0.78 $\pm$ .00 | 0.78 $\pm$ .00 |
| WW | 0.76 $\pm$ .00 | 0.85 $\pm$ .00 | 0.91 $\pm$ .00 | 0.67 $\pm$ .00 | 0.96 $\pm$ .00 | 0.69 $\pm$ .01 | 0.99 $\pm$ .01 | 0.68 $\pm$ .01 | 0.76 $\pm$ .00 | 0.61 $\pm$ .00 | 0.61 $\pm$ .00 |
| IO | 2.02 $\pm$ .02 | 1.33 $\pm$ .03 | 1.40 $\pm$ .01 | 1.58 $\pm$ .01 | 1.54 $\pm$ .01 | 1.37 $\pm$ .02 | 2.44 $\pm$ .38 | 5.46 $\pm$ .00 | 5.10 $\pm$ .04 | 1.18 $\pm$ .00 | 1.11 $\pm$ .01 |
| BR | 1.07 $\pm$ .00 | 0.77 $\pm$ .00 | 0.62 $\pm$ .00 | 0.47 $\pm$ .01 | 0.48 $\pm$ .00 | 0.41 $\pm$ .00 | 0.87 $\pm$ .07 | 1.43 $\pm$ .00 | 1.89 $\pm$ .01 | 0.44 $\pm$ .00 | 0.44 $\pm$ .00 |
| IR | 2.19 $\pm$ .06 | 1.75 $\pm$ .07 | 2.19 $\pm$ .03 | 1.44 $\pm$ .12 | 2.13 $\pm$ .03 | 1.34 $\pm$ .12 | 1.49 $\pm$ .08 | 3.22 $\pm$ .00 | 4.78 $\pm$ .08 | 1.05 $\pm$ .01 | 1.05 $\pm$ .00 |
| DI | 1.78 $\pm$ .02 | 1.96 $\pm$ .02 | 1.86 $\pm$ .00 | 1.55 $\pm$ .03 | 1.69 $\pm$ .00 | 1.38 $\pm$ .01 | 1.58 $\pm$ .01 | 2.74 $\pm$ .00 | 5.01 $\pm$ .02 | 1.44 $\pm$ .01 | 1.44 $\pm$ .00 |
| PR | 0.91 $\pm$ .00 | 0.64 $\pm$ .00 | 1.04 $\pm$ .00 | 0.42 $\pm$ .00 | 1.56 $\pm$ .00 | 0.33 $\pm$ .00 | 1.00 $\pm$ .02 | 0.39 $\pm$ .01 | 0.94 $\pm$ .00 | 0.30 $\pm$ .00 | 0.30 $\pm$ .00 |
| SP | 0.23 $\pm$ .00 | 0.21 $\pm$ .00 | 0.32 $\pm$ .00 | 0.23 $\pm$ .00 | 0.17 $\pm$ .00 | 0.19 $\pm$ .00 | 0.21 $\pm$ .00 | 0.21 $\pm$ .00 | 0.16 $\pm$ .00 | 0.17 $\pm$ .01 | 0.16 $\pm$ .00 |
| LE | 1.28 $\pm$ .00 | 1.39 $\pm$ .00 | 1.37 $\pm$ .00 | 1.07 $\pm$ .00 | 2.35 $\pm$ .00 | 0.81 $\pm$ .00 | 2.20 $\pm$ .04 | 1.17 $\pm$ .01 | 1.32 $\pm$ .02 | 0.69 $\pm$ .00 | 0.69 $\pm$ .01 |
| AB | 2.52 $\pm$ .01 | 2.56 $\pm$ .03 | 2.90 $\pm$ .05 | 2.39 $\pm$ .03 | 2.64 $\pm$ .01 | 2.15 $\pm$ .20 | 2.83 $\pm$ .07 | 3.63 $\pm$ .00 | 2.65 $\pm$ .01 | 2.19 $\pm$ .00 | 2.00 $\pm$ .03 |
| A4 | 1.07 $\pm$ .00 | 1.20 $\pm$ .02 | 1.28 $\pm$ .01 | 0.96 $\pm$ .01 | 1.95 $\pm$ .00 | 0.86 $\pm$ .02 | 1.38 $\pm$ .04 | 1.00 $\pm$ .00 | 1.12 $\pm$ .01 | 0.87 $\pm$ .00 | 0.85 $\pm$ .00 |
| CM | 2.36 $\pm$ .01 | 2.43 $\pm$ .06 | 2.65 $\pm$ .01 | 2.33 $\pm$ .03 | 3.50 $\pm$ .04 | 2.18 $\pm$ .02 | 2.50 $\pm$ .16 | 2.24 $\pm$ .03 | 2.40 $\pm$ .02 | 2.01 $\pm$ .02 | 1.94 $\pm$ .02 |
| GE | 2.52 $\pm$ .01 | 2.56 $\pm$ .03 | 2.90 $\pm$ .05 | 2.38 $\pm$ .03 | 2.64 $\pm$ .01 | 2.15 $\pm$ .20 | 2.83 $\pm$ .07 | 3.63 $\pm$ .00 | 2.65 $\pm$ .01 | 2.17 $\pm$ .01 | 2.03 $\pm$ .01 |
| ST | 1.80 $\pm$ .00 | 1.41 $\pm$ .01 | 1.54 $\pm$ .01 | 1.19 $\pm$ .01 | 1.42 $\pm$ .00 | 0.89 $\pm$ .01 | 1.67 $\pm$ .01 | 1.22 $\pm$ .05 | 1.70 $\pm$ .01 | 0.61 $\pm$ .01 | 0.56 $\pm$ .01 |
| LI | 1.82 $\pm$ .01 | 0.35 $\pm$ .00 | 0.39 $\pm$ .00 | 0.26 $\pm$ .01 | 0.27 $\pm$ .00 | 0.17 $\pm$ .00 | 0.97 $\pm$ .12 | 5.30 $\pm$ .00 | 2.08 $\pm$ .00 | 0.14 $\pm$ .01 | 0.13 $\pm$ .01 |
| CH | 1.13 $\pm$ .00 | 1.21 $\pm$ .00 | 1.45 $\pm$ .00 | 0.84 $\pm$ .00 | 1.74 $\pm$ .00 | 0.74 $\pm$ .00 | 1.33 $\pm$ .07 | 0.83 $\pm$ .00 | 1.16 $\pm$ .00 | 0.70 $\pm$ .00 | 0.70 $\pm$ .00 |
| Missing 70% |  |  |  |  |  |  |  |  |  |  |  |
| AI | 2.32 $\pm$ .01 | 2.45 $\pm$ .05 | 3.02 $\pm$ .03 | 2.26 $\pm$ .02 | 2.67 $\pm$ .03 | 2.26 $\pm$ .03 | 2.36 $\pm$ .03 | 2.31 $\pm$ .02 | 2.37 $\pm$ .01 | 2.06 $\pm$ .01 | 2.07 $\pm$ .01 |
| BL | 1.15 $\pm$ .01 | 1.18 $\pm$ .02 | 1.27 $\pm$ .02 | 0.99 $\pm$ .04 | 1.30 $\pm$ .01 | 0.99 $\pm$ .03 | 1.32 $\pm$ .05 | 1.47 $\pm$ .01 | 1.97 $\pm$ .01 | 0.93 $\pm$ .00 | 0.92 $\pm$ .00 |
| WW | 0.76 $\pm$ .00 | 0.92 $\pm$ .01 | 1.02 $\pm$ .00 | 0.72 $\pm$ .00 | 1.31 $\pm$ .00 | 0.82 $\pm$ .02 | 1.49 $\pm$ .11 | 0.72 $\pm$ .05 | 0.79 $\pm$ .01 | 0.68$\pm$ .00 | 0.68 $\pm$ .00 |
| IO | 2.02 $\pm$ .01 | 2.21 $\pm$ .04 | 1.90 $\pm$ .05 | 1.83 $\pm$ .04 | 1.94 $\pm$ .04 | 1.54 $\pm$ .01 | 3.47 $\pm$ .47 | 5.31 $\pm$ .00 | 5.12 $\pm$ .02 | 1.29 $\pm$ .00 | 1.27 $\pm$ .01 |
| BR | 1.07 $\pm$ .00 | 1.07 $\pm$ .01 | 0.73 $\pm$ .00 | 0.79 $\pm$ .07 | 0.65 $\pm$ .01 | 0.52 $\pm$ .00 | 1.10 $\pm$ .07 | 1.45 $\pm$ .00 | 1.88 $\pm$ .01 | 0.55 $\pm$ .00 | 0.55 $\pm$ .00 |
| IR | 2.22 $\pm$ .03 | 2.48 $\pm$ .14 | 2.87 $\pm$ .03 | 1.79 $\pm$ .07 | 2.91 $\pm$ .05 | 1.57 $\pm$ .13 | 1.66 $\pm$ .07 | 3.20 $\pm$ .00 | 4.71 $\pm$ .10 | 1.34 $\pm$ .00 | 1.33 $\pm$ .02 |
| DI | 1.81 $\pm$ .00 | 2.05 $\pm$ .03 | 2.05 $\pm$ .02 | 1.69 $\pm$ .03 | 2.06 $\pm$ .01 | 1.65 $\pm$ .01 | 2.44 $\pm$ .20 | 2.76 $\pm$ .00 | 5.02 $\pm$ .03 | 1.61 $\pm$ .01 | 1.58 $\pm$ .00 |
| PR | 0.91 $\pm$ .00 | 0.75 $\pm$ .00 | 1.16 $\pm$ .00 | 0.61 $\pm$ .03 | 1.94 $\pm$ .00 | 0.47 $\pm$ .00 | 1.48 $\pm$ .12 | 0.54 $\pm$ .01 | 0.94 $\pm$ .00 | 0.43 $\pm$ .01 | 0.42 $\pm$ .00 |
| SP | 0.23 $\pm$ .00 | 0.24 $\pm$ .00 | 0.35 $\pm$ .01 | 0.23 $\pm$ .01 | 0.17 $\pm$ .00 | 0.20 $\pm$ .00 | 0.17 $\pm$ .01 | 0.22 $\pm$ .00 | 0.16 $\pm$ .00 | 0.17 $\pm$ .01 | 0.16 $\pm$ .00 |
| LE | 1.28 $\pm$ .00 | 1.47 $\pm$ .00 | 1.58 $\pm$ .00 | 1.19 $\pm$ .01 | 2.95 $\pm$ .00 | 1.12 $\pm$ .02 | 1.61 $\pm$ .07 | 1.31 $\pm$ .02 | 1.32 $\pm$ .01 | 0.94 $\pm$ .00 | 0.94 $\pm$ .01 |
| AB | 2.53 $\pm$ .01 | 2.61 $\pm$ .03 | 3.31 $\pm$ .07 | 2.51 $\pm$ .06 | 2.96 $\pm$ .01 | 2.31 $\pm$ .29 | 3.21 $\pm$ .15 | 3.63 $\pm$ .00 | 2.69 $\pm$ .01 | 2.36 $\pm$ .02 | 2.24 $\pm$ .01 |
| A4 | 1.06 $\pm$ .00 | 1.27 $\pm$ .04 | 1.53 $\pm$ .01 | 1.02 $\pm$ .01 | 2.30 $\pm$ .00 | 0.99 $\pm$ .01 | 2.31 $\pm$ .03 | 1.04 $\pm$ .02 | 1.22 $\pm$ .01 | 0.96 $\pm$ .00 | 0.92 $\pm$ .00 |
| CM | 2.38 $\pm$ .01 | 2.68 $\pm$ .06 | 3.11 $\pm$ .04 | 2.37 $\pm$ .04 | 4.24 $\pm$ .01 | 1.99 $\pm$ .03 | 4.06 $\pm$ .07 | 2.40 $\pm$ .01 | 2.40 $\pm$ .01 | 2.17 $\pm$ .00 | 2.13 $\pm$ .02 |
| GE | 2.53 $\pm$ .01 | 2.61 $\pm$ .03 | 3.31 $\pm$ .07 | 2.51 $\pm$ .06 | 2.96 $\pm$ .01 | 2.31 $\pm$ .29 | 3.21 $\pm$ .15 | 3.63 $\pm$ .00 | 2.69 $\pm$ .01 | 2.35 $\pm$ .01 | 2.22 $\pm$ .02 |
| ST | 1.80 $\pm$ .00 | 1.75 $\pm$ .01 | 1.85 $\pm$ .02 | 1.56 $\pm$ .08 | 1.88 $\pm$ .00 | 1.14 $\pm$ .00 | 2.42 $\pm$ .16 | 1.50 $\pm$ .00 | 1.70 $\pm$ .01 | 0.93 $\pm$ .01 | 0.88 $\pm$ .01 |
| LI | 1.82 $\pm$ .00 | 0.74 $\pm$ .00 | 0.50 $\pm$ .01 | 0.71 $\pm$ .01 | 0.55 $\pm$ .01 | 0.33 $\pm$ .01 | 3.84 $\pm$ .27 | 4.90 $\pm$ .00 | 2.05 $\pm$ .01 | 0.22 $\pm$ .01 | 0.21 $\pm$ .01 |
| CH | 1.13 $\pm$ .00 | 1.27 $\pm$ .00 | 1.61 $\pm$ .00 | 0.99 $\pm$ .00 | 2.02 $\pm$ .00 | 0.94 $\pm$ .01 | 2.27 $\pm$ .02 | 0.98 $\pm$ .02 | 1.16 $\pm$ .00 | 0.89 $\pm$ .00 | 0.88 $\pm$ .00 |

**Table 20: MAE scores under the MAR setting across different levels of missingness on the extra 17 datasets. Please refer to Table 9 for dataset names.**
| Dataset | Mean | Knn | Svd | Mice | Spectral | HI | Gain | Miracle | Miwae | Grape | M^3-Impute |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Missing 10% |  |  |  |  |  |  |  |  |  |  |  |
| AI | 2.40 $\pm$ .19 | 1.87 $\pm$ .24 | 2.37 $\pm$ .34 | 1.85 $\pm$ .18 | 2.01 $\pm$ .28 | 0.56 $\pm$ .16 | 2.17 $\pm$ .09 | 1.22 $\pm$ .05 | 2.57 $\pm$ .05 | 0.75 $\pm$ .02 | 0.68 $\pm$ .02 |
| BL | 1.12 $\pm$ .43 | 0.55 $\pm$ .30 | 1.05 $\pm$ .35 | 0.63 $\pm$ .52 | 0.79 $\pm$ .31 | 0.59 $\pm$ .18 | 1.07 $\pm$ .10 | 1.14 $\pm$ .00 | 1.99 $\pm$ .10 | 0.21 $\pm$ .00 | 0.20 $\pm$ .00 |
| WW | 0.74 $\pm$ .12 | 0.36 $\pm$ .01 | 0.86 $\pm$ .04 | 0.45 $\pm$ .06 | 0.71 $\pm$ .11 | 0.39 $\pm$ .01 | 0.60 $\pm$ .02 | 0.44 $\pm$ .01 | 0.69 $\pm$ .01 | 0.38 $\pm$ .00 | 0.37 $\pm$ .00 |
| IO | 2.11 $\pm$ .02 | 1.19 $\pm$ .04 | 1.41 $\pm$ .01 | 1.44 $\pm$ .14 | 1.41 $\pm$ .06 | 1.32 $\pm$ .03 | 1.36 $\pm$ .02 | 5.41 $\pm$ .00 | 6.09 $\pm$ .09 | 1.19 $\pm$ .03 | 1.04 $\pm$ .02 |
| BR | 1.18 $\pm$ .03 | 0.53 $\pm$ .04 | 0.59 $\pm$ .02 | 0.29 $\pm$ .03 | 0.34 $\pm$ .01 | 0.30 $\pm$ .01 | 0.50 $\pm$ .03 | 1.40 $\pm$ .00 | 2.18 $\pm$ .03 | 0.34 $\pm$ .01 | 0.34 $\pm$ .00 |
| IR | 2.33 $\pm$ .56 | 0.97 $\pm$ .10 | 1.31 $\pm$ .18 | 0.88 $\pm$ .14 | 0.99 $\pm$ .29 | 0.89 $\pm$ .06 | 1.23 $\pm$ .14 | 2.19 $\pm$ .00 | 3.41 $\pm$ .36 | 0.87 $\pm$ .01 | 0.86 $\pm$ .01 |
| DI | 1.78 $\pm$ .38 | 1.11 $\pm$ .03 | 1.84 $\pm$ .42 | 1.26 $\pm$ .40 | 1.69 $\pm$ .41 | 0.93 $\pm$ .02 | 1.16 $\pm$ .03 | 2.41 $\pm$ .00 | 4.20 $\pm$ .09 | 0.87 $\pm$ .01 | 0.87 $\pm$ .02 |
| PR | 0.92 $\pm$ .09 | 0.23 $\pm$ .04 | 1.47 $\pm$ .15 | 0.30 $\pm$ .08 | 0.59 $\pm$ .32 | 0.19 $\pm$ .01 | 0.58 $\pm$ .13 | 0.17 $\pm$ .02 | 0.99 $\pm$ .05 | 0.20 $\pm$ .01 | 0.19 $\pm$ .00 |
| SP | 0.23 $\pm$ .01 | 0.14 $\pm$ .02 | 0.50 $\pm$ .15 | 0.20 $\pm$ .01 | 0.18 $\pm$ .00 | 0.15 $\pm$ .01 | 0.22 $\pm$ .00 | 0.18 $\pm$ .00 | 0.15 $\pm$ .01 | 0.14 $\pm$ .00 | 0.14 $\pm$ .00 |
| LE | 1.35 $\pm$ .08 | 0.31 $\pm$ .03 | 1.36 $\pm$ .09 | 0.88 $\pm$ .12 | 1.43 $\pm$ .20 | 0.45 $\pm$ .07 | 0.96 $\pm$ .06 | 0.66 $\pm$ .03 | 1.56 $\pm$ .01 | 0.44 $\pm$ .01 | 0.40 $\pm$ .00 |
| AB | 2.63 $\pm$ .28 | 2.21 $\pm$ .24 | 2.37 $\pm$ .24 | 1.93 $\pm$ .13 | 2.45 $\pm$ .08 | 2.16 $\pm$ .29 | 2.06 $\pm$ .06 | 3.09 $\pm$ .00 | 2.54 $\pm$ .04 | 1.91 $\pm$ .02 | 1.79 $\pm$ .02 |
| A4 | 1.23 $\pm$ .20 | 0.61 $\pm$ .23 | 1.38 $\pm$ .15 | 0.93 $\pm$ .39 | 1.12 $\pm$ .34 | 0.48 $\pm$ .42 | 1.27 $\pm$ .22 | 0.40 $\pm$ .02 | 1.53 $\pm$ .06 | 1.14 $\pm$ .01 | 1.10 $\pm$ .01 |
| CM | 2.17 $\pm$ .07 | 2.33 $\pm$ .11 | 2.37 $\pm$ .30 | 2.19 $\pm$ .14 | 2.97 $\pm$ .09 | 1.64 $\pm$ .19 | 2.04 $\pm$ .15 | 1.84 $\pm$ .00 | 1.97 $\pm$ .07 | 1.63 $\pm$ .01 | 1.56 $\pm$ .01 |
| GE | 2.63 $\pm$ .28 | 2.21 $\pm$ .24 | 2.37 $\pm$ .24 | 1.93 $\pm$ .13 | 2.45 $\pm$ .08 | 2.16 $\pm$ .29 | 2.06 $\pm$ .06 | 3.09 $\pm$ .00 | 2.54 $\pm$ .04 | 1.92 $\pm$ .00 | 1.83 $\pm$ .01 |
| ST | 1.76 $\pm$ .18 | 0.52 $\pm$ .07 | 1.30 $\pm$ .16 | 0.64 $\pm$ .10 | 1.07 $\pm$ .17 | 0.25 $\pm$ .06 | 0.76 $\pm$ .03 | 0.60 $\pm$ .03 | 1.48 $\pm$ .02 | 0.28 $\pm$ .01 | 0.21 $\pm$ .00 |
| LI | 1.88 $\pm$ .02 | 0.26 $\pm$ .00 | 0.36 $\pm$ .01 | 0.04 $\pm$ .00 | 0.15 $\pm$ .00 | 0.10 $\pm$ .00 | 0.34 $\pm$ .01 | 5.88 $\pm$ .00 | 2.26 $\pm$ .04 | 0.11 $\pm$ .00 | 0.10 $\pm$ .01 |
| CH | 1.18 $\pm$ .41 | 0.53 $\pm$ .13 | 1.53 $\pm$ .26 | 0.67 $\pm$ .12 | 1.21 $\pm$ .12 | 0.40 $\pm$ .09 | 1.11 $\pm$ .23 | 0.37 $\pm$ .01 | 1.02 $\pm$ .04 | 0.27 $\pm$ .00 | 0.27 $\pm$ .00 |
| Missing 30% |  |  |  |  |  |  |  |  |  |  |  |
| AI | 2.33 $\pm$ .14 | 1.59 $\pm$ .70 | 2.99 $\pm$ .83 | 2.16 $\pm$ .28 | 2.01 $\pm$ .60 | 1.21 $\pm$ .21 | 2.29 $\pm$ .09 | 2.23 $\pm$ .00 | 2.56 $\pm$ .01 | 1.57 $\pm$ .02 | 1.54 $\pm$ .02 |
| BL | 0.91 $\pm$ .02 | 0.90 $\pm$ .25 | 0.91 $\pm$ .07 | 1.00 $\pm$ .40 | 1.22 $\pm$ .36 | 0.88 $\pm$ .33 | 1.01 $\pm$ .15 | 1.14 $\pm$ .00 | 2.03 $\pm$ .03 | 0.29 $\pm$ .01 | 0.28 $\pm$ .01 |
| WW | 0.87 $\pm$ .01 | 0.53 $\pm$ .02 | 0.78 $\pm$ .05 | 0.63 $\pm$ .04 | 0.99 $\pm$ .23 | 0.57 $\pm$ .08 | 0.65 $\pm$ .11 | 0.60 $\pm$ .05 | 0.69 $\pm$ .01 | 0.48 $\pm$ .00 | 0.48 $\pm$ .00 |
| IO | 2.02 $\pm$ .08 | 1.09 $\pm$ .03 | 1.40 $\pm$ .08 | 1.43 $\pm$ .08 | 1.50 $\pm$ .02 | 1.30 $\pm$ .03 | 1.71 $\pm$ .10 | 5.28 $\pm$ .00 | 6.10 $\pm$ .04 | 1.17 $\pm$ .03 | 1.07 $\pm$ .01 |
| BR | 1.13 $\pm$ .03 | 0.53 $\pm$ .03 | 0.61 $\pm$ .03 | 0.32 $\pm$ .07 | 0.46 $\pm$ .04 | 0.34 $\pm$ .02 | 0.69 $\pm$ .05 | 1.33 $\pm$ .00 | 2.17 $\pm$ .03 | 0.39 $\pm$ .00 | 0.37 $\pm$ .01 |
| IR | 1.99 $\pm$ .25 | 0.91 $\pm$ .08 | 1.85 $\pm$ .42 | 0.85 $\pm$ .09 | 1.62 $\pm$ .13 | 1.05 $\pm$ .11 | 1.25 $\pm$ .04 | 2.38 $\pm$ .00 | 3.46 $\pm$ .13 | 0.86 $\pm$ .02 | 0.82 $\pm$ .03 |
| DI | 1.74 $\pm$ .33 | 1.43 $\pm$ .23 | 2.09 $\pm$ .02 | 1.33 $\pm$ .23 | 1.32 $\pm$ .20 | 1.46 $\pm$ .10 | 1.34 $\pm$ .04 | 2.56 $\pm$ .00 | 4.26 $\pm$ .06 | 1.12 $\pm$ .01 | 1.07 $\pm$ .00 |
| PR | 0.91 $\pm$ .03 | 0.22 $\pm$ .02 | 1.32 $\pm$ .20 | 0.35 $\pm$ .06 | 1.77 $\pm$ .26 | 0.23 $\pm$ .01 | 0.88 $\pm$ .02 | 0.27 $\pm$ .04 | 0.97 $\pm$ .01 | 0.20 $\pm$ .01 | 0.20 $\pm$ .00 |
| SP | 0.23 $\pm$ .01 | 0.16 $\pm$ .01 | 0.40 $\pm$ .06 | 0.24 $\pm$ .01 | 0.19 $\pm$ .02 | 0.18 $\pm$ .00 | 0.16 $\pm$ .01 | 0.20 $\pm$ .00 | 0.15 $\pm$ .00 | 0.15 $\pm$ .01 | 0.14 $\pm$ .00 |
| LE | 1.36 $\pm$ .02 | 0.42 $\pm$ .05 | 1.56 $\pm$ .29 | 1.07 $\pm$ .02 | 2.42 $\pm$ .21 | 0.58 $\pm$ .11 | 1.28 $\pm$ .13 | 0.97 $\pm$ .04 | 1.56 $\pm$ .01 | 0.57 $\pm$ .01 | 0.54 $\pm$ .00 |
| AB | 2.53 $\pm$ .08 | 2.43 $\pm$ .19 | 2.65 $\pm$ .16 | 2.26 $\pm$ .27 | 2.72 $\pm$ .35 | 2.25 $\pm$ .10 | 2.21 $\pm$ .08 | 3.06 $\pm$ .00 | 2.59 $\pm$ .02 | 2.08 $\pm$ .03 | 1.91 $\pm$ .02 |
| A4 | 1.35 $\pm$ .19 | 0.79 $\pm$ .04 | 1.12 $\pm$ .19 | 0.94 $\pm$ .20 | 1.72 $\pm$ .76 | 0.88 $\pm$ .27 | 0.77 $\pm$ .12 | 0.90 $\pm$ .05 | 1.44 $\pm$ .02 | 1.11 $\pm$ .00 | 1.08 $\pm$ .00 |
| CM | 2.41 $\pm$ .21 | 2.11 $\pm$ .09 | 2.60 $\pm$ .10 | 2.02 $\pm$ .21 | 3.71 $\pm$ .74 | 2.02 $\pm$ .03 | 2.18 $\pm$ .11 | 2.21 $\pm$ .00 | 2.08 $\pm$ .02 | 1.83 $\pm$ .01 | 1.74 $\pm$ .01 |
| GE | 2.53 $\pm$ .08 | 2.43 $\pm$ .19 | 2.65 $\pm$ .16 | 2.26 $\pm$ .27 | 2.72 $\pm$ .35 | 2.25 $\pm$ .10 | 2.21 $\pm$ .08 | 3.07 $\pm$ .00 | 2.59 $\pm$ .02 | 2.06 $\pm$ .02 | 1.91 $\pm$ .02 |
| ST | 1.79 $\pm$ .17 | 0.77 $\pm$ .10 | 1.58 $\pm$ .42 | 0.91 $\pm$ .12 | 1.32 $\pm$ .35 | 0.48 $\pm$ .03 | 1.25 $\pm$ .09 | 0.84 $\pm$ .02 | 1.48 $\pm$ .01 | 0.45 $\pm$ .01 | 0.42 $\pm$ .02 |
| LI | 1.91 $\pm$ .01 | 0.48 $\pm$ .01 | 0.40 $\pm$ .00 | 0.07 $\pm$ .00 | 0.30 $\pm$ .01 | 0.15 $\pm$ .00 | 0.96 $\pm$ .01 | 5.07 $\pm$ .00 | 2.24 $\pm$ .03 | 0.15 $\pm$ .01 | 0.11 $\pm$ .01 |
| CH | 1.04 $\pm$ .23 | 0.83 $\pm$ .07 | 1.65 $\pm$ .20 | 0.71 $\pm$ .25 | 2.61 $\pm$ .24 | 0.47 $\pm$ .10 | 1.29 $\pm$ .15 | 0.42 $\pm$ .03 | 1.02 $\pm$ .03 | 0.32 $\pm$ .00 | 0.32 $\pm$ .00 |
| Missing 50% |  |  |  |  |  |  |  |  |  |  |  |
| AI | 2.47 $\pm$ .17 | 2.13 $\pm$ .79 | 3.42 $\pm$ .64 | 2.18 $\pm$ .14 | 2.52 $\pm$ .34 | 2.03 $\pm$ .11 | 2.18 $\pm$ .16 | 2.16 $\pm$ .00 | 2.27 $\pm$ .01 | 1.86 $\pm$ .01 | 1.86 $\pm$ .01 |
| BL | 1.22 $\pm$ .20 | 1.26 $\pm$ .18 | 0.78 $\pm$ .05 | 0.80 $\pm$ .12 | 1.11 $\pm$ .45 | 0.96 $\pm$ .08 | 1.24 $\pm$ .05 | 1.39 $\pm$ .00 | 2.31 $\pm$ .04 | 0.73 $\pm$ .00 | 0.73 $\pm$ .00 |
| WW | 0.77 $\pm$ .04 | 0.68 $\pm$ .09 | 0.88 $\pm$ .08 | 0.69 $\pm$ .00 | 1.08 $\pm$ .15 | 0.75 $\pm$ .03 | 1.40 $\pm$ .05 | 0.74 $\pm$ .00 | 0.82 $\pm$ .02 | 0.65 $\pm$ .00 | 0.65 $\pm$ .00 |
| IO | 2.04 $\pm$ .03 | 1.06 $\pm$ .01 | 1.55 $\pm$ .09 | 1.70 $\pm$ .10 | 1.79 $\pm$ .08 | 1.43 $\pm$ .04 | 2.54 $\pm$ .10 | 5.43 $\pm$ .00 | 5.31 $\pm$ .03 | 1.20 $\pm$ .00 | 1.13 $\pm$ .02 |
| BR | 1.13 $\pm$ .03 | 0.62 $\pm$ .01 | 0.65 $\pm$ .04 | 0.42 $\pm$ .02 | 0.61 $\pm$ .08 | 0.43 $\pm$ .02 | 0.93 $\pm$ .11 | 1.37 $\pm$ .00 | 2.09 $\pm$ .01 | 0.42 $\pm$ .00 | 0.42 $\pm$ .00 |
| IR | 2.50 $\pm$ .23 | 1.62 $\pm$ .54 | 2.34 $\pm$ .53 | 1.53 $\pm$ .28 | 2.62 $\pm$ .27 | 1.36 $\pm$ .35 | 1.74 $\pm$ .29 | 3.07 $\pm$ .00 | 4.03 $\pm$ .30 | 0.84 $\pm$ .01 | 0.84 $\pm$ .01 |
| DI | 1.82 $\pm$ .25 | 1.76 $\pm$ .22 | 1.82 $\pm$ .17 | 1.53 $\pm$ .24 | 1.84 $\pm$ .21 | 1.37 $\pm$ .16 | 2.20 $\pm$ .05 | 2.88 $\pm$ .00 | 5.79 $\pm$ .02 | 1.65 $\pm$ .01 | 1.60 $\pm$ .02 |
| PR | 1.03 $\pm$ .06 | 0.38 $\pm$ .03 | 1.11 $\pm$ .07 | 0.41 $\pm$ .04 | 1.72 $\pm$ .31 | 0.27 $\pm$ .02 | 1.09 $\pm$ .18 | 0.38 $\pm$ .00 | 0.95 $\pm$ .01 | 0.23 $\pm$ .00 | 0.23 $\pm$ .00 |
| SP | 0.23 $\pm$ .01 | 0.17 $\pm$ .00 | 0.37 $\pm$ .02 | 0.22 $\pm$ .01 | 0.16 $\pm$ .02 | 0.19 $\pm$ .01 | 0.16 $\pm$ .00 | 0.20 $\pm$ .00 | 0.16 $\pm$ .00 | 0.16 $\pm$ .01 | 0.15 $\pm$ .00 |
| LE | 1.28 $\pm$ .07 | 0.86 $\pm$ .03 | 1.61 $\pm$ .06 | 1.05 $\pm$ .04 | 2.61 $\pm$ .11 | 0.92 $\pm$ .03 | 1.90 $\pm$ .10 | 1.27 $\pm$ .00 | 1.36 $\pm$ .02 | 0.71 $\pm$ .02 | 0.69 $\pm$ .01 |
| AB | 2.39 $\pm$ .16 | 2.44 $\pm$ .26 | 3.19 $\pm$ .30 | 2.51 $\pm$ .04 | 2.83 $\pm$ .15 | 2.27 $\pm$ .14 | 2.66 $\pm$ .23 | 3.65 $\pm$ .00 | 2.58 $\pm$ .04 | 2.19 $\pm$ .02 | 2.04 $\pm$ .01 |
| A4 | 1.17 $\pm$ .11 | 1.11 $\pm$ .37 | 1.29 $\pm$ .02 | 0.87 $\pm$ .29 | 1.90 $\pm$ .34 | 0.87 $\pm$ .04 | 1.50 $\pm$ .16 | 1.10 $\pm$ .04 | 1.17 $\pm$ .01 | 0.87 $\pm$ .01 | 0.84 $\pm$ .00 |
| CM | 2.37 $\pm$ .06 | 2.28 $\pm$ .18 | 2.67 $\pm$ .11 | 2.38 $\pm$ .05 | 4.27 $\pm$ .47 | 1.86 $\pm$ .08 | 3.39 $\pm$ .01 | 2.54 $\pm$ .00 | 2.43 $\pm$ .01 | 2.17 $\pm$ .01 | 2.04 $\pm$ .02 |
| GE | 2.39 $\pm$ .16 | 2.44 $\pm$ .26 | 3.19 $\pm$ .30 | 2.51 $\pm$ .04 | 2.83 $\pm$ .15 | 2.27 $\pm$ .14 | 2.66 $\pm$ .23 | 3.65 $\pm$ .00 | 2.58 $\pm$ .04 | 2.23 $\pm$ .03 | 2.05 $\pm$ .03 |
| ST | 1.72 $\pm$ .11 | 0.83 $\pm$ .03 | 1.89 $\pm$ .23 | 1.18 $\pm$ .14 | 1.59 $\pm$ .21 | 0.83 $\pm$ .05 | 2.04 $\pm$ .24 | 1.13 $\pm$ .00 | 1.52 $\pm$ .02 | 0.64 $\pm$ .01 | 0.57 $\pm$ .01 |
| LI | 1.89 $\pm$ .02 | 0.58 $\pm$ .00 | 0.44 $\pm$ .01 | 0.16 $\pm$ .04 | 0.43 $\pm$ .01 | 0.21 $\pm$ .01 | 2.11 $\pm$ .08 | 4.25 $\pm$ .00 | 2.31 $\pm$ .04 | 0.22 $\pm$ .02 | 0.13 $\pm$ .01 |
| CH | 1.09 $\pm$ .18 | 0.88 $\pm$ .12 | 1.51 $\pm$ .02 | 1.03 $\pm$ .13 | 1.78 $\pm$ .42 | 0.81 $\pm$ .08 | 1.74 $\pm$ .19 | 0.78 $\pm$ .00 | 1.06 $\pm$ .02 | 0.67 $\pm$ .00 | 0.66 $\pm$ .00 |
| Missing 70% |  |  |  |  |  |  |  |  |  |  |  |
| AI | 2.50 $\pm$ .19 | 2.31 $\pm$ .55 | 3.21 $\pm$ .96 | 2.24 $\pm$ .16 | 2.47 $\pm$ .22 | 2.06 $\pm$ .11 | 2.32 $\pm$ .22 | 2.25 $\pm$ .00 | 2.26 $\pm$ .01 | 1.96 $\pm$ .00 | 1.95 $\pm$ .00 |
| BL | 1.27 $\pm$ .31 | 1.13 $\pm$ .17 | 1.00 $\pm$ .29 | 0.93 $\pm$ .15 | 0.94 $\pm$ .51 | 1.01 $\pm$ .08 | 1.18 $\pm$ .09 | 1.40 $\pm$ .00 | 2.33 $\pm$ .03 | 0.80 $\pm$ .01 | 0.80 $\pm$ .01 |
| WW | 0.81 $\pm$ .02 | 0.88 $\pm$ .02 | 0.92 $\pm$ .01 | 0.71 $\pm$ .03 | 1.35 $\pm$ .21 | 0.85 $\pm$ .02 | 1.56 $\pm$ .06 | 0.81 $\pm$ .00 | 0.84 $\pm$ .00 | 0.72 $\pm$ .00 | 0.71 $\pm$ .00 |
| IO | 2.08 $\pm$ .03 | 1.48 $\pm$ .08 | 2.03 $\pm$ .10 | 1.90 $\pm$ .05 | 2.37 $\pm$ .13 | 1.60 $\pm$ .01 | 3.52 $\pm$ .32 | 5.44 $\pm$ .00 | 5.28 $\pm$ .05 | 1.29 $\pm$ .00 | 1.29 $\pm$ .01 |
| BR | 1.13 $\pm$ .04 | 0.68 $\pm$ .01 | 0.73 $\pm$ .02 | 0.74 $\pm$ .07 | 0.73 $\pm$ .03 | 0.54 $\pm$ .02 | 1.13 $\pm$ .07 | 1.40 $\pm$ .00 | 2.00 $\pm$ .01 | 0.55 $\pm$ .00 | 0.54 $\pm$ .01 |
| IR | 2.48 $\pm$ .13 | 1.56 $\pm$ .13 | 1.89 $\pm$ .46 | 1.89 $\pm$ .60 | 2.94 $\pm$ .52 | 1.39 $\pm$ .20 | 1.86 $\pm$ .24 | 3.07 $\pm$ .00 | 4.03 $\pm$ .29 | 0.85 $\pm$ .01 | 0.85 $\pm$ .01 |
| DI | 1.91 $\pm$ .04 | 1.96 $\pm$ .07 | 2.01 $\pm$ .09 | 1.77 $\pm$ .05 | 2.26 $\pm$ .08 | 1.52 $\pm$ .10 | 2.90 $\pm$ .13 | 2.96 $\pm$ .00 | 5.03 $\pm$ .05 | 1.68 $\pm$ .00 | 1.68 $\pm$ .01 |
| PR | 1.05 $\pm$ .05 | 0.53 $\pm$ .02 | 1.18 $\pm$ .07 | 0.57 $\pm$ .17 | 2.02 $\pm$ .18 | 0.44 $\pm$ .00 | 1.32 $\pm$ .27 | 0.60 $\pm$ .08 | 1.01 $\pm$ .01 | 0.40 $\pm$ .00 | 0.40 $\pm$ .00 |
| SP | 0.23 $\pm$ .01 | 0.20 $\pm$ .01 | 0.38 $\pm$ .01 | 0.24 $\pm$ .00 | 0.17 $\pm$ .00 | 0.21 $\pm$ .00 | 0.16 $\pm$ .01 | 0.22 $\pm$ .00 | 0.16 $\pm$ .00 | 0.16 $\pm$ .01 | 0.16 $\pm$ .00 |
| LE | 1.30 $\pm$ .08 | 1.30 $\pm$ .14 | 1.72 $\pm$ .13 | 1.25 $\pm$ .04 | 3.16 $\pm$ .10 | 1.21 $\pm$ .03 | 1.67 $\pm$ .14 | 1.30 $\pm$ .02 | 1.32 $\pm$ .05 | 0.98 $\pm$ .00 | 0.98 $\pm$ .01 |
| AB | 2.59 $\pm$ .09 | 2.47 $\pm$ .06 | 3.53 $\pm$ .23 | 2.63 $\pm$ .05 | 3.12 $\pm$ .17 | 2.93 $\pm$ .16 | 3.08 $\pm$ .07 | 3.54 $\pm$ .00 | 2.68 $\pm$ .01 | 2.46 $\pm$ .01 | 2.33 $\pm$ .03 |
| A4 | 1.14 $\pm$ .02 | 1.13 $\pm$ .11 | 1.17 $\pm$ .02 | 0.98 $\pm$ .11 | 2.45 $\pm$ .29 | 0.86 $\pm$ .04 | 1.67 $\pm$ .18 | 1.16 $\pm$ .00 | 1.21 $\pm$ .09 | 0.94 $\pm$ .01 | 0.90 $\pm$ .01 |
| CM | 2.43 $\pm$ .05 | 2.46 $\pm$ .20 | 3.13 $\pm$ .10 | 2.53 $\pm$ .06 | 4.32 $\pm$ .41 | 2.14 $\pm$ .03 | 4.24 $\pm$ .29 | 2.58 $\pm$ .00 | 2.40 $\pm$ .02 | 2.10 $\pm$ .00 | 2.05 $\pm$ .03 |
| GE | 2.59 $\pm$ .09 | 2.47 $\pm$ .06 | 3.53 $\pm$ .23 | 2.63 $\pm$ .05 | 3.12 $\pm$ .17 | 2.93 $\pm$ .16 | 3.08 $\pm$ .07 | 3.54 $\pm$ .00 | 2.68 $\pm$ .01 | 2.43 $\pm$ .00 | 2.34 $\pm$ .03 |
| ST | 1.75 $\pm$ .12 | 1.07 $\pm$ .05 | 1.88 $\pm$ .08 | 1.61 $\pm$ .10 | 2.10 $\pm$ .12 | 1.12 $\pm$ .04 | 2.36 $\pm$ .06 | 1.31 $\pm$ .00 | 1.55 $\pm$ .03 | 0.82 $\pm$ .01 | 0.76 $\pm$ .00 |
| LI | 1.90 $\pm$ .01 | 0.71 $\pm$ .03 | 0.57 $\pm$ .01 | 0.55 $\pm$ .02 | 0.89 $\pm$ .02 | 0.36 $\pm$ .01 | 3.18 $\pm$ .09 | 4.92 $\pm$ .00 | 2.10 $\pm$ .01 | 0.28 $\pm$ .01 | 0.23 $\pm$ .01 |
| CH | 1.12 $\pm$ .06 | 1.23 $\pm$ .06 | 1.75 $\pm$ .08 | 1.01 $\pm$ .08 | 1.74 $\pm$ .08 | 0.91 $\pm$ .06 | 2.18 $\pm$ .07 | 1.02 $\pm$ .01 | 1.21 $\pm$ .00 | 0.88 $\pm$ .00 | 0.87 $\pm$ .00 |

**Table 21: MAE scores under the MNAR setting across different levels of missingness on the extra 17 datasets. Please refer to Table 9 for dataset names.**
| Dataset | Mean | Knn | Svd | Mice | Spectral | HI | Gain | Miracle | Miwae | Grape | M^3-Impute |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Missing 10% |  |  |  |  |  |  |  |  |  |  |  |
| AI | 2.46 $\pm$ .12 | 1.93 $\pm$ .10 | 2.81 $\pm$ .19 | 1.83 $\pm$ .13 | 2.39 $\pm$ .33 | 0.68 $\pm$ .03 | 2.24 $\pm$ .03 | 1.70 $\pm$ .04 | 2.38 $\pm$ .03 | 0.75 $\pm$ .01 | 0.69 $\pm$ .02 |
| BL | 1.05 $\pm$ .05 | 1.07 $\pm$ .13 | 1.01 $\pm$ .18 | 0.62 $\pm$ .12 | 1.01 $\pm$ .12 | 0.56 $\pm$ .09 | 1.22 $\pm$ .05 | 1.34 $\pm$ .00 | 1.98 $\pm$ .06 | 0.39 $\pm$ .00 | 0.39 $\pm$ .00 |
| WW | 0.78 $\pm$ .04 | 0.41 $\pm$ .01 | 0.88 $\pm$ .04 | 0.51 $\pm$ .02 | 0.67 $\pm$ .05 | 0.45 $\pm$ .01 | 0.68 $\pm$ .00 | 0.49 $\pm$ .00 | 0.77 $\pm$ .00 | 0.44 $\pm$ .00 | 0.44 $\pm$ .00 |
| IO | 2.07 $\pm$ .05 | 1.10 $\pm$ .02 | 1.29 $\pm$ .04 | 1.41 $\pm$ .09 | 1.38 $\pm$ .01 | 1.24 $\pm$ .04 | 1.46 $\pm$ .04 | 5.45 $\pm$ .00 | 5.49 $\pm$ .12 | 1.17 $\pm$ .01 | 1.08 $\pm$ .00 |
| BR | 1.10 $\pm$ .02 | 0.50 $\pm$ .01 | 0.58 $\pm$ .01 | 0.26 $\pm$ .01 | 0.33 $\pm$ .01 | 0.28 $\pm$ .01 | 0.46 $\pm$ .01 | 1.34 $\pm$ .00 | 1.92 $\pm$ .01 | 0.31 $\pm$ .00 | 0.31 $\pm$ .00 |
| IR | 2.20 $\pm$ .12 | 1.21 $\pm$ .16 | 1.67 $\pm$ .16 | 0.89 $\pm$ .04 | 1.32 $\pm$ .03 | 0.91 $\pm$ .02 | 1.30 $\pm$ .05 | 2.79 $\pm$ .00 | 4.18 $\pm$ .09 | 0.75 $\pm$ .01 | 0.73 $\pm$ .01 |
| DI | 1.79 $\pm$ .13 | 1.51 $\pm$ .05 | 1.72 $\pm$ .12 | 1.13 $\pm$ .16 | 1.44 $\pm$ .09 | 1.02 $\pm$ .07 | 1.39 $\pm$ .02 | 2.74 $\pm$ .00 | 4.59 $\pm$ .09 | 1.20 $\pm$ .03 | 1.18 $\pm$ .01 |
| PR | 0.91 $\pm$ .03 | 0.32 $\pm$ .02 | 1.03 $\pm$ .09 | 0.27 $\pm$ .01 | 0.80 $\pm$ .08 | 0.21 $\pm$ .02 | 0.57 $\pm$ .02 | 0.25 $\pm$ .01 | 0.99 $\pm$ .04 | 0.19 $\pm$ .00 | 0.19 $\pm$ .00 |
| SP | 0.23 $\pm$ .01 | 0.15 $\pm$ .00 | 0.37 $\pm$ .02 | 0.20 $\pm$ .01 | 0.16 $\pm$ .00 | 0.18 $\pm$ .00 | 0.21 $\pm$ .01 | 0.18 $\pm$ .00 | 0.16 $\pm$ .00 | 0.16 $\pm$ .00 | 0.15 $\pm$ .00 |
| LE | 1.31 $\pm$ .02 | 0.34 $\pm$ .01 | 1.32 $\pm$ .02 | 0.90 $\pm$ .03 | 1.29 $\pm$ .04 | 0.50 $\pm$ .01 | 1.07 $\pm$ .01 | 0.74 $\pm$ .00 | 1.40 $\pm$ .01 | 0.44 $\pm$ .00 | 0.43 $\pm$ .00 |
| AB | 2.53 $\pm$ .12 | 2.06 $\pm$ .12 | 2.63 $\pm$ .11 | 2.04 $\pm$ .12 | 2.31 $\pm$ .03 | 1.95 $\pm$ .09 | 2.23 $\pm$ .05 | 3.43 $\pm$ .00 | 2.56 $\pm$ .03 | 1.83 $\pm$ .00 | 1.66 $\pm$ .02 |
| A4 | 1.07 $\pm$ .08 | 1.05 $\pm$ .07 | 1.21 $\pm$ .09 | 0.84 $\pm$ .15 | 1.23 $\pm$ .15 | 0.65 $\pm$ .11 | 0.65 $\pm$ .11 | 0.79 $\pm$ .02 | 1.24 $\pm$ .02 | 0.86 $\pm$ .00 | 0.84 $\pm$ .00 |
| CM | 2.45 $\pm$ .08 | 2.19 $\pm$ .12 | 2.49 $\pm$ .06 | 2.06 $\pm$ .04 | 2.68 $\pm$ .10 | 1.74 $\pm$ .11 | 2.49 $\pm$ .04 | 2.07 $\pm$ .04 | 2.38 $\pm$ .00 | 1.91 $\pm$ .00 | 1.83 $\pm$ .00 |
| GE | 2.53 $\pm$ .12 | 2.06 $\pm$ .12 | 2.63 $\pm$ .11 | 2.04 $\pm$ .12 | 2.31 $\pm$ .03 | 1.95 $\pm$ .09 | 2.23 $\pm$ .05 | 3.43 $\pm$ .00 | 2.56 $\pm$ .03 | 1.84 $\pm$ .02 | 1.67 $\pm$ .00 |
| ST | 1.71 $\pm$ .05 | 0.54 $\pm$ .03 | 1.30 $\pm$ .02 | 0.71 $\pm$ .02 | 0.90 $\pm$ .00 | 0.60 $\pm$ .03 | 0.93 $\pm$ .04 | 0.71 $\pm$ .02 | 1.60 $\pm$ .01 | 0.35 $\pm$ .01 | 0.30 $\pm$ .01 |
| LI | 1.84 $\pm$ .01 | 0.20 $\pm$ .00 | 0.37 $\pm$ .00 | 0.05 $\pm$ .00 | 0.14 $\pm$ .00 | 0.06 $\pm$ .00 | 0.38 $\pm$ .00 | 5.62 $\pm$ .00 | 2.16 $\pm$ .02 | 0.11 $\pm$ .01 | 0.09 $\pm$ .00 |
| CH | 1.07 $\pm$ .04 | 0.87 $\pm$ .06 | 1.24 $\pm$ .05 | 0.60 $\pm$ .05 | 1.42 $\pm$ .04 | 0.45 $\pm$ .04 | 1.22 $\pm$ .16 | 0.49 $\pm$ .01 | 1.09 $\pm$ .01 | 0.37 $\pm$ .00 | 0.37 $\pm$ .00 |
| Missing 30% |  |  |  |  |  |  |  |  |  |  |  |
| AI | 2.36 $\pm$ .11 | 2.11 $\pm$ .27 | 2.98 $\pm$ .52 | 2.07 $\pm$ .14 | 2.64 $\pm$ .18 | 1.23 $\pm$ .04 | 2.21 $\pm$ .05 | 1.72 $\pm$ .00 | 2.47 $\pm$ .03 | 1.46 $\pm$ .03 | 1.46 $\pm$ .01 |
| BL | 0.98 $\pm$ .05 | 1.04 $\pm$ .12 | 0.98 $\pm$ .09 | 0.76 $\pm$ .17 | 1.40 $\pm$ .18 | 0.82 $\pm$ .18 | 1.09 $\pm$ .06 | 1.24 $\pm$ .00 | 1.99 $\pm$ .04 | 0.42 $\pm$ .00 | 0.41 $\pm$ .00 |
| WW | 0.82 $\pm$ .01 | 0.60 $\pm$ .02 | 0.82 $\pm$ .04 | 0.62 $\pm$ .02 | 0.88 $\pm$ .13 | 0.58 $\pm$ .05 | 0.69 $\pm$ .01 | 0.59 $\pm$ .01 | 0.72 $\pm$ .00 | 0.49 $\pm$ .00 | 0.49 $\pm$ .00 |
| IO | 2.04 $\pm$ .06 | 1.12 $\pm$ .03 | 1.36 $\pm$ .07 | 1.44 $\pm$ .07 | 1.46 $\pm$ .02 | 1.28 $\pm$ .02 | 1.55 $\pm$ .03 | 5.39 $\pm$ .00 | 5.66 $\pm$ .02 | 1.15 $\pm$ .01 | 1.06 $\pm$ .02 |
| BR | 1.11 $\pm$ .02 | 0.55 $\pm$ .02 | 0.60 $\pm$ .03 | 0.33 $\pm$ .02 | 0.41 $\pm$ .03 | 0.36 $\pm$ .03 | 0.62 $\pm$ .02 | 1.35 $\pm$ .00 | 2.05 $\pm$ .00 | 0.38 $\pm$ .00 | 0.36 $\pm$ .01 |
| IR | 2.06 $\pm$ .09 | 1.53 $\pm$ .52 | 1.66 $\pm$ .20 | 0.99 $\pm$ .11 | 1.35 $\pm$ .11 | 1.07 $\pm$ .07 | 1.26 $\pm$ .04 | 2.67 $\pm$ .00 | 3.98 $\pm$ .32 | 0.89 $\pm$ .02 | 0.87 $\pm$ .00 |
| DI | 1.77 $\pm$ .20 | 1.60 $\pm$ .17 | 1.93 $\pm$ .02 | 1.27 $\pm$ .16 | 1.51 $\pm$ .13 | 1.30 $\pm$ .19 | 1.43 $\pm$ .06 | 2.60 $\pm$ .00 | 4.62 $\pm$ .08 | 1.21 $\pm$ .01 | 1.19 $\pm$ .00 |
| PR | 0.93 $\pm$ .02 | 0.55 $\pm$ .05 | 1.17 $\pm$ .08 | 0.38 $\pm$ .05 | 1.66 $\pm$ .12 | 0.23 $\pm$ .01 | 0.60 $\pm$ .19 | 0.28 $\pm$ .06 | 0.96 $\pm$ .01 | 0.21 $\pm$ .01 | 0.21 $\pm$ .00 |
| SP | 0.23 $\pm$ .01 | 0.18 $\pm$ .01 | 0.40 $\pm$ .07 | 0.26 $\pm$ .02 | 0.16 $\pm$ .01 | 0.17 $\pm$ .00 | 0.18 $\pm$ .00 | 0.19 $\pm$ .00 | 0.15 $\pm$ .00 | 0.15 $\pm$ .00 | 0.14 $\pm$ .00 |
| LE | 1.32 $\pm$ .08 | 0.59 $\pm$ .05 | 1.56 $\pm$ .29 | 0.94 $\pm$ .01 | 1.95 $\pm$ .14 | 0.58 $\pm$ .07 | 1.30 $\pm$ .08 | 0.96 $\pm$ .01 | 1.48 $\pm$ .01 | 0.54 $\pm$ .01 | 0.52 $\pm$ .00 |
| AB | 2.49 $\pm$ .14 | 2.46 $\pm$ .11 | 2.67 $\pm$ .07 | 2.25 $\pm$ .06 | 2.55 $\pm$ .10 | 2.36 $\pm$ .13 | 2.37 $\pm$ .03 | 3.27 $\pm$ .00 | 2.60 $\pm$ .04 | 2.03 $\pm$ .01 | 1.89 $\pm$ .03 |
| A4 | 1.01 $\pm$ .26 | 1.27 $\pm$ .13 | 0.99 $\pm$ .18 | 0.88 $\pm$ .15 | 1.98 $\pm$ .03 | 0.84 $\pm$ .22 | 0.81 $\pm$ .07 | 1.12 $\pm$ .06 | 1.35 $\pm$ .03 | 1.04 $\pm$ .00 | 1.02 $\pm$ .01 |
| CM | 2.37 $\pm$ .10 | 2.23 $\pm$ .03 | 2.54 $\pm$ .07 | 2.36 $\pm$ .13 | 2.95 $\pm$ .05 | 1.91 $\pm$ .07 | 2.13 $\pm$ .03 | 2.06 $\pm$ .00 | 2.15 $\pm$ .05 | 1.82 $\pm$ .01 | 1.71 $\pm$ .02 |
| GE | 2.49 $\pm$ .14 | 2.46 $\pm$ .11 | 2.67 $\pm$ .07 | 2.25 $\pm$ .06 | 2.55 $\pm$ .10 | 2.36 $\pm$ .13 | 2.37 $\pm$ .03 | 3.27 $\pm$ .00 | 2.60 $\pm$ .04 | 2.04 $\pm$ .02 | 1.88 $\pm$ .02 |
| ST | 1.75 $\pm$ .08 | 0.79 $\pm$ .13 | 1.58 $\pm$ .14 | 0.91 $\pm$ .11 | 1.24 $\pm$ .11 | 0.69 $\pm$ .04 | 1.27 $\pm$ .09 | 0.89 $\pm$ .04 | 1.56 $\pm$ .01 | 0.42 $\pm$ .00 | 0.39 $\pm$ .01 |
| LI | 1.85 $\pm$ .03 | 0.37 $\pm$ .00 | 0.38 $\pm$ .01 | 0.09 $\pm$ .00 | 0.22 $\pm$ .00 | 0.12 $\pm$ .01 | 0.62 $\pm$ .03 | 5.29 $\pm$ .00 | 2.21 $\pm$ .02 | 0.12 $\pm$ .01 | 0.12 $\pm$ .01 |
| CH | 0.95 $\pm$ .10 | 1.19 $\pm$ .14 | 1.52 $\pm$ .15 | 0.62 $\pm$ .14 | 1.69 $\pm$ .42 | 0.53 $\pm$ .06 | 1.09 $\pm$ .17 | 0.54 $\pm$ .02 | 1.06 $\pm$ .01 | 0.42 $\pm$ .00 | 0.42 $\pm$ .00 |
| Missing 50% |  |  |  |  |  |  |  |  |  |  |  |
| AI | 2.39 $\pm$ .11 | 2.37 $\pm$ .68 | 3.04 $\pm$ .23 | 2.35 $\pm$ .09 | 2.82 $\pm$ .08 | 2.02 $\pm$ .02 | 2.13 $\pm$ .11 | 2.17 $\pm$ .00 | 2.27 $\pm$ .03 | 1.86 $\pm$ .00 | 1.86 $\pm$ .00 |
| BL | 1.22 $\pm$ .23 | 1.24 $\pm$ .09 | 1.06 $\pm$ .31 | 0.87 $\pm$ .13 | 1.19 $\pm$ .41 | 0.94 $\pm$ .07 | 1.24 $\pm$ .04 | 1.39 $\pm$ .00 | 2.26 $\pm$ .03 | 0.77 $\pm$ .00 | 0.77 $\pm$ .00 |
| WW | 0.70 $\pm$ .07 | 0.78 $\pm$ .04 | 0.96 $\pm$ .08 | 0.72 $\pm$ .04 | 1.13 $\pm$ .10 | 0.74 $\pm$ .03 | 1.24 $\pm$ .04 | 0.73 $\pm$ .00 | 0.83 $\pm$ .02 | 0.65 $\pm$ .00 | 0.65 $\pm$ .00 |
| IO | 2.06 $\pm$ .03 | 1.20 $\pm$ .02 | 1.54 $\pm$ .02 | 1.61 $\pm$ .07 | 1.74 $\pm$ .04 | 1.43 $\pm$ .07 | 2.59 $\pm$ .15 | 5.43 $\pm$ .00 | 5.27 $\pm$ .01 | 1.18 $\pm$ .00 | 1.15 $\pm$ .00 |
| BR | 1.12 $\pm$ .03 | 0.66 $\pm$ .01 | 0.67 $\pm$ .03 | 0.46 $\pm$ .02 | 0.55 $\pm$ .03 | 0.43 $\pm$ .01 | 0.87 $\pm$ .09 | 1.40 $\pm$ .00 | 2.05 $\pm$ .02 | 0.44 $\pm$ .01 | 0.43 $\pm$ .01 |
| IR | 2.42 $\pm$ .09 | 1.58 $\pm$ .10 | 2.19 $\pm$ .26 | 1.30 $\pm$ .17 | 2.27 $\pm$ .05 | 1.38 $\pm$ .32 | 1.61 $\pm$ .23 | 3.10 $\pm$ .00 | 4.16 $\pm$ .30 | 0.96 $\pm$ .01 | 0.96 $\pm$ .01 |
| DI | 1.99 $\pm$ .04 | 1.70 $\pm$ .03 | 1.92 $\pm$ .07 | 1.69 $\pm$ .07 | 1.80 $\pm$ .19 | 1.39 $\pm$ .15 | 1.99 $\pm$ .07 | 2.86 $\pm$ .00 | 5.64 $\pm$ .04 | 1.60 $\pm$ .02 | 1.60 $\pm$ .00 |
| PR | 1.04 $\pm$ .10 | 0.66 $\pm$ .09 | 1.06 $\pm$ .03 | 0.45 $\pm$ .07 | 1.75 $\pm$ .17 | 0.30 $\pm$ .01 | 1.26 $\pm$ .15 | 0.40 $\pm$ .00 | 0.95 $\pm$ .01 | 0.25 $\pm$ .02 | 0.25 $\pm$ .00 |
| SP | 0.23 $\pm$ .00 | 0.19 $\pm$ .00 | 0.38 $\pm$ .03 | 0.23 $\pm$ .02 | 0.16 $\pm$ .02 | 0.19 $\pm$ .00 | 0.19 $\pm$ .01 | 0.20 $\pm$ .00 | 0.16 $\pm$ .00 | 0.17 $\pm$ .00 | 0.15 $\pm$ .00 |
| LE | 1.32 $\pm$ .02 | 1.26 $\pm$ .11 | 1.61 $\pm$ .04 | 1.08 $\pm$ .03 | 2.62 $\pm$ .10 | 0.92 $\pm$ .03 | 1.62 $\pm$ .15 | 1.24 $\pm$ .00 | 1.34 $\pm$ .02 | 0.72 $\pm$ .00 | 0.70 $\pm$ .01 |
| AB | 2.41 $\pm$ .06 | 2.47 $\pm$ .04 | 3.14 $\pm$ .23 | 2.34 $\pm$ .11 | 2.65 $\pm$ .19 | 2.37 $\pm$ .13 | 2.92 $\pm$ .22 | 3.63 $\pm$ .00 | 2.60 $\pm$ .01 | 2.23 $\pm$ .03 | 2.03 $\pm$ .02 |
| A4 | 1.17 $\pm$ .09 | 1.25 $\pm$ .22 | 1.37 $\pm$ .26 | 0.85 $\pm$ .11 | 2.44 $\pm$ .19 | 0.75 $\pm$ .14 | 1.46 $\pm$ .10 | 1.17 $\pm$ .00 | 1.17 $\pm$ .02 | 0.88 $\pm$ .00 | 0.84 $\pm$ .00 |
| CM | 2.40 $\pm$ .07 | 2.34 $\pm$ .07 | 2.82 $\pm$ .22 | 2.30 $\pm$ .10 | 4.13 $\pm$ .24 | 1.90 $\pm$ .09 | 3.29 $\pm$ .27 | 2.51 $\pm$ .00 | 2.43 $\pm$ .02 | 2.10 $\pm$ .00 | 1.98 $\pm$ .03 |
| GE | 2.41 $\pm$ .06 | 2.47 $\pm$ .04 | 3.14 $\pm$ .23 | 2.34 $\pm$ .11 | 2.65 $\pm$ .19 | 2.37 $\pm$ .13 | 2.92 $\pm$ .22 | 3.63 $\pm$ .00 | 2.60 $\pm$ .01 | 2.19 $\pm$ .03 | 2.01 $\pm$ .05 |
| ST | 1.76 $\pm$ .09 | 0.96 $\pm$ .06 | 1.71 $\pm$ .10 | 1.27 $\pm$ .09 | 1.67 $\pm$ .07 | 0.85 $\pm$ .01 | 1.88 $\pm$ .07 | 1.17 $\pm$ .00 | 1.56 $\pm$ .02 | 0.63 $\pm$ .01 | 0.59 $\pm$ .01 |
| LI | 1.87 $\pm$ .01 | 0.52 $\pm$ .01 | 0.44 $\pm$ .01 | 0.18 $\pm$ .01 | 0.40 $\pm$ .01 | 0.21 $\pm$ .00 | 1.68 $\pm$ .08 | 4.39 $\pm$ .00 | 2.26 $\pm$ .02 | 0.20 $\pm$ .02 | 0.14 $\pm$ .01 |
| CH | 1.12 $\pm$ .12 | 1.11 $\pm$ .11 | 1.66 $\pm$ .11 | 0.88 $\pm$ .13 | 1.82 $\pm$ .23 | 0.80 $\pm$ .07 | 1.94 $\pm$ .26 | 0.78 $\pm$ .00 | 1.08 $\pm$ .02 | 0.68 $\pm$ .00 | 0.67 $\pm$ .00 |
| Missing 70% |  |  |  |  |  |  |  |  |  |  |  |
| AI | 2.36 $\pm$ .18 | 2.33 $\pm$ .43 | 3.26 $\pm$ .67 | 2.33 $\pm$ .09 | 2.84 $\pm$ .13 | 2.11 $\pm$ .12 | 2.25 $\pm$ .14 | 2.29 $\pm$ .00 | 2.28 $\pm$ .01 | 2.01 $\pm$ .02 | 1.99 $\pm$ .02 |
| BL | 1.22 $\pm$ .19 | 1.05 $\pm$ .23 | 1.25 $\pm$ .29 | 0.93 $\pm$ .13 | 1.36 $\pm$ .19 | 1.01 $\pm$ .08 | 1.16 $\pm$ .05 | 1.40 $\pm$ .00 | 2.29 $\pm$ .03 | 0.82 $\pm$ .01 | 0.82 $\pm$ .00 |
| WW | 0.79 $\pm$ .01 | 0.90 $\pm$ .02 | 0.96 $\pm$ .00 | 0.75 $\pm$ .01 | 1.47 $\pm$ .12 | 0.87 $\pm$ .02 | 1.53 $\pm$ .11 | 0.80 $\pm$ .00 | 0.84 $\pm$ .00 | 0.72 $\pm$ .00 | 0.71 $\pm$ .00 |
| IO | 2.07 $\pm$ .01 | 1.67 $\pm$ .08 | 2.11 $\pm$ .01 | 1.92 $\pm$ .05 | 2.29 $\pm$ .07 | 1.57 $\pm$ .02 | 4.02 $\pm$ .46 | 5.43 $\pm$ .00 | 5.27 $\pm$ .04 | 1.31 $\pm$ .01 | 1.28 $\pm$ .02 |
| BR | 1.15 $\pm$ .03 | 0.93 $\pm$ .04 | 0.79 $\pm$ .06 | 0.83 $\pm$ .09 | 0.75 $\pm$ .05 | 0.55 $\pm$ .01 | 1.16 $\pm$ .06 | 1.40 $\pm$ .00 | 1.97 $\pm$ .01 | 0.55 $\pm$ .00 | 0.55 $\pm$ .01 |
| IR | 2.46 $\pm$ .10 | 1.57 $\pm$ .18 | 2.21 $\pm$ .30 | 2.58 $\pm$ .99 | 2.78 $\pm$ .52 | 1.28 $\pm$ .16 | 1.69 $\pm$ .13 | 3.07 $\pm$ .00 | 4.13 $\pm$ .29 | 1.01 $\pm$ .01 | 1.01 $\pm$ .01 |
| DI | 1.86 $\pm$ .03 | 1.96 $\pm$ .13 | 2.06 $\pm$ .05 | 1.67 $\pm$ .17 | 2.45 $\pm$ .05 | 1.58 $\pm$ .10 | 2.41 $\pm$ .19 | 2.95 $\pm$ .00 | 5.01 $\pm$ .03 | 1.67 $\pm$ .02 | 1.66 $\pm$ .02 |
| PR | 1.04 $\pm$ .05 | 0.63 $\pm$ .02 | 1.19 $\pm$ .06 | 0.59 $\pm$ .17 | 1.17 $\pm$ .09 | 0.46 $\pm$ .01 | 1.44 $\pm$ .14 | 0.62 $\pm$ .05 | 1.00 $\pm$ .01 | 0.41 $\pm$ .00 | 0.41 $\pm$ .00 |
| SP | 0.23 $\pm$ .01 | 0.23 $\pm$ .00 | 0.38 $\pm$ .00 | 0.23 $\pm$ .00 | 0.17 $\pm$ .00 | 0.21 $\pm$ .00 | 0.19 $\pm$ .03 | 0.22 $\pm$ .00 | 0.16 $\pm$ .00 | 0.17 $\pm$ .01 | 0.16 $\pm$ .00 |
| LE | 1.33 $\pm$ .04 | 1.48 $\pm$ .03 | 1.75 $\pm$ .11 | 1.25 $\pm$ .01 | 3.05 $\pm$ .01 | 1.21 $\pm$ .02 | 1.63 $\pm$ .08 | 1.24 $\pm$ .04 | 1.34 $\pm$ .02 | 0.98 $\pm$ .00 | 0.98 $\pm$ .00 |
| AB | 2.61 $\pm$ .07 | 2.51 $\pm$ .07 | 3.52 $\pm$ .17 | 2.56 $\pm$ .04 | 2.99 $\pm$ .10 | 2.36 $\pm$ .35 | 3.13 $\pm$ .06 | 3.55 $\pm$ .00 | 2.66 $\pm$ .02 | 2.47 $\pm$ .02 | 2.34 $\pm$ .03 |
| A4 | 1.13 $\pm$ .01 | 1.20 $\pm$ .10 | 1.53 $\pm$ .30 | 1.04 $\pm$ .13 | 2.55 $\pm$ .09 | 1.09 $\pm$ .13 | 1.60 $\pm$ .19 | 1.18 $\pm$ .00 | 1.22 $\pm$ .09 | 0.95 $\pm$ .00 | 0.91 $\pm$ .01 |
| CM | 2.38 $\pm$ .13 | 2.52 $\pm$ .07 | 3.16 $\pm$ .17 | 2.38 $\pm$ .11 | 4.62 $\pm$ .24 | 2.15 $\pm$ .06 | 4.00 $\pm$ .23 | 2.55 $\pm$ .00 | 2.40 $\pm$ .02 | 2.12 $\pm$ .04 | 2.02 $\pm$ .02 |
| GE | 2.61 $\pm$ .07 | 2.51 $\pm$ .07 | 3.52 $\pm$ .17 | 2.56 $\pm$ .04 | 2.99 $\pm$ .10 | 2.36 $\pm$ .35 | 3.13 $\pm$ .06 | 3.55 $\pm$ .00 | 2.66 $\pm$ .02 | 2.46 $\pm$ .02 | 2.33 $\pm$ .05 |
| ST | 1.74 $\pm$ .11 | 1.45 $\pm$ .03 | 2.05 $\pm$ .12 | 1.55 $\pm$ .12 | 2.16 $\pm$ .05 | 1.25 $\pm$ .09 | 2.35 $\pm$ .15 | 1.31 $\pm$ .00 | 1.58 $\pm$ .03 | 0.83 $\pm$ .01 | 0.78 $\pm$ .01 |
| LI | 1.89 $\pm$ .01 | 0.71 $\pm$ .01 | 0.60 $\pm$ .05 | 0.62 $\pm$ .03 | 0.92 $\pm$ .01 | 0.37 $\pm$ .01 | 3.67 $\pm$ .29 | 4.93 $\pm$ .00 | 2.09 $\pm$ .01 | 0.27 $\pm$ .02 | 0.24 $\pm$ .00 |
| CH | 1.22 $\pm$ .05 | 1.27 $\pm$ .07 | 1.81 $\pm$ .06 | 0.99 $\pm$ .06 | 1.95 $\pm$ .11 | 0.91 $\pm$ .03 | 2.02 $\pm$ .10 | 1.02 $\pm$ .03 | 1.20 $\pm$ .00 | 0.89 $\pm$ .00 | 0.88 $\pm$ .00 |

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 6
presentation: 2
questions: 1. Is there any additional analysis regarding the computational complexity during the training phase?

2. Is there any analysis when the proposed method significantly outperforms and when does not. In practicality, it is helpful to know when the proposed method is powerful since it entails more complexity. If the improvement is marginal, GRAPE can be a good alternative.

3. How did you tune the proposed method? Information about baselines are attached in Appendix but I cannot see about the proposed method. Also, some ablation study only presents marginal changes, so it is difficult to get insights from them. Is there any different results about them?
soundness: 2
strengths: - Clear statement and approach: This paper clearly states the limitations of existing methods and thereby why the proposed method is necessary.

- Novel approach: Combination of feature correlation, sample correlation, and missingness information is fair separately and concurrently.

- Comprehensive experiments: This paper conducted comprehensive experiments with 25 benchmark datasets and 3 missingness patterns. This is the most broad experimental result among studies on missing value imputation.

- Detailed description: Authors provided detailed explanation for each module in the proposed method.
summary: The paper proposes a novel grpah neural network-based approach to address the problem of missing value imputation. The proposed method explicitly uses the missingness information with correlations between features and samples. With 25 benchmark datasets, authors demonstrated its superiority compared to recent missing value imputation methods.
weaknesses: Complexity issue: The proposed method is much more complex than other methods, including its predecessor GRAPE. Although the authors already provided the computing time at test phase, but I am curious about how long the training phase takes.

Limitations in comparative analysis: While the authors conducted comprehensive experiments, but the improvements are mostly marginal, making it difficult to assess whether the improvements are significant or not. The authors should consider ways to present in a more compelling manner to highlight their significance.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: 1.The experiments, although extensive, are primarily conducted on a set of datasets that may not fully represent the diversity of real-world data. It would be beneficial to include more datasets with varying characteristics, such as different domains and levels of missingness (eg: a certain type of severe deficiency), to more comprehensively evaluate the robustness and generalizability of $M^3$-Impute.

2.The paper notes that $M^3$-Impute struggles with datasets where data points have low dependencies, as FCU and SCU rely on data similarity. This can lead to sparse or inadequate graph formation and poor model performance. Can we employ strategies such as enhancing data representation and refining graph construction techniques to tackle these challenges?

3.Reference [1] also engages in graph construction and employs graph representation learning for missing value imputation. However, the approach to graph construction varies between the two methods. Has the author conducted a comparative analysis? Is there a correlation between the efficiency of missing value imputation and the method of graph construction? Which approach proves to be more effective?

[1] Jiaxuan You, Xiaobai Ma, Daisy Yi Ding, Mykel J. Kochenderfer, and Jure Leskovec. Handling missing data with graph representation learning. Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020.
soundness: 3
strengths: 1.$M^3$-Impute demonstrates a high level of originality, with significant contributions including an innovative embedding initialization process that incorporates missingness information, which is uncommon in previous studies and offers a new perspective for the field of missing data imputation. 

2.The introduction of feature and sample correlation unit (FCU and SCU) is innovative as they explicitly model feature and sample correlations through a soft masking mechanism, marking a novel approach in existing missing data imputation methods. 

3.Extensive experiments on 25 datasets demonstrating $M^3$-Impute's superior performance over existing methods under different missing data patterns.
summary: The paper introduces $M^3$-Impute, a novel method for missing value imputation that leverages mask-guided representation learning to address the challenge of accurately imputing missing data. M3-Impute incorporates missingness information and captures feature and sample correlations through innovative masking schemes, offering a significant advancement in the field. The method's key contributions include a refined embedding initialization process that integrates missingness information, the introduction of feature correlation unit (FCU) and sample correlation unit (SCU) to effectively model feature-wise and sample-wise correlations, and extensive experiments on 25 datasets demonstrating M3-Impute's superior performance over existing methods under various missing data patterns. By jointly modeling feature and sample correlations with missingness information, M3-Impute leads to improved accuracy in imputing missing values.
weaknesses: 1.To strengthen the paper, it would benefit from comparing $M^3$-Impute against the most recent state-of-the-art methods in missing value imputation, highlighting its unique advantages over the latest techniques.

2.The paper should include a detailed examination of $M^3$-Impute's performance with large-scale datasets to assess its scalability and efficiency as data volumes expand. 

3.Adding a section on the model's interpretability would enhance the paper, as it is vital to understand the model's predictions for trust and application in critical fields like healthcare and finance.

4.The paper should delve into $M^3$-Impute's computational efficiency, including memory usage and optimization, to address resource limitations in practical applications.

---

confidence: 5
contribution: 2
initial_rating: 3
presentation: 3
questions: While Graph Neural Networks (GNNs) are valuable for capturing the intrinsic structure of feature and sample correlations, the M3 method, along with similar approaches, falls short in modeling the underlying missing data mechanisms. Additionally, most machine learning tasks increasingly emphasize causal inference to ensure robust and interpretable outcomes. However, your current method overlooks causal inference considerations, which could limit its application to settings where understanding causal relationships is critical.
soundness: 2
strengths: Originality: M3-Impute introduces a novel approach to missing value imputation by leveraging a mask-guided representation learning method. Unlike conventional methods, it incorporates both sample-feature relationships and missingness information in embedding initialization. The development of the feature correlation unit (FCU) and sample correlation unit (SCU) further distinguishes this method, providing unique mechanisms to explicitly model feature- and sample-wise correlations.

Quality: The proposed methodology demonstrates a well-structured integration of graph representation learning with innovative imputation techniques. M3-Impute’s use of bipartite graph modeling, along with the FCU and SCU units, reflects a robust design that balances theory and practical application, enhancing the overall quality and rigor of the imputation process.

Clarity: The paper clearly outlines each component of M3-Impute, detailing the step-by-step process of embedding initialization, feature- and sample-wise correlation modeling, and final imputation integration. This level of clarity provides readers with a clear understanding of the method’s architecture and functionality.

Significance: By directly addressing gaps in current imputation methods, M3-Impute holds significant potential to advance the field. Its effective incorporation of missingness information and explicit correlation modeling not only improves imputation accuracy but also sets a foundation for more sophisticated data analysis in complex datasets, impacting fields reliant on high-quality data imputation.
summary: Missing values challenge data analysis and machine learning, requiring effective imputation to enhance dataset quality. The authors claim that current methods often ignore missingness information and correlations among features and samples, reducing accuracy. M3-Impute addresses this by modeling data as a bipartite graph and using a graph neural network that incorporates missingness in initialization. Its feature and sample correlation units (FCU and SCU) capture essential correlations for improved imputation. Testing on 25 benchmark datasets under three missingness settings shows M3-Impute’s superior performance, achieving top MAE scores in most cases.
weaknesses: There are several major limitations.

First, the discussion of missing data mechanisms is incomplete. The literature on different missing data mechanisms—such as Missing Completely at Random (MCAR), Missing at Random (MAR), and Missing Not at Random (MNAR)—is extensive. Please refer to:

Ibrahim, J. G., Chen, M. H., Lipsitz, S. R., & Herring, A. H. (2005). Missing-data methods for generalized linear models: A comparative review. Journal of the American Statistical Association, 100(469), 332-346.
Kim, J.K. and Shao, J. (2021). Statistical methods for handling incomplete data. Chapman and Hall/CRC.
This paper, however, is limited to addressing only MCAR.

Second, the graph-based missing data method proposed by You et al. (2020) faces significant issues. In particular, it does not scale well to large datasets, as its efficiency depends heavily on both sample size and the number of variables.

Third, some concepts similar to the feature correlation unit (FCU) and sample correlation unit (SCU) have been previously explored in the following references:

Zhong, J., Gui, N., & Ye, W. (2023). "Data Imputation with Iterative Graph Reconstruction." Proceedings of the AAAI Conference on Artificial Intelligence, 37(9), 11399-11407.
 Jarrett, Daniel, et al. "HyperImpute: Generalized Iterative Imputation with Automatic Model Selection" ICML 2022.
 Gupta, Shubham, et al. "GRAFENNE: learning on graphs with heterogeneous and dynamic feature sets." International Conference on Machine Learning. PMLR, 2023.
 Qiao, Lishan, et al. "Estimating functional brain networks by incorporating a modularity prior." Neuroimage 141 (2016): 399-407.
 Um, Daeho, et al. "Confidence-Based Feature Imputation for Graphs with Partially Known Features." The Eleventh International Conference on Learning Representations.

Meta-review: The current paper proposed a new graph neural network for missing value imputation, with novel modules explicitly considering missing patterns. All reviewers acknowledged that the empirical study is extensive and the paper is well written. After the rebuttal, the reviewers have remaining concerns on how the method tackles different missing mechanisms, including the highly challenging MNAR, which should be analyzed in more detail, and modest claims should be made as only one specific MNAR setting is tested. The method's scalability needs more clarification on dataset with a large number features and on the computational overhead of FCU and SCU modules. The effectiveness analysis of the imputation should incorporate both feature correlations and sample correlations.

Decision: Reject