Original: Missing Data Imputation and Acquisition with Deep Hierarchical Models and Hamiltonian Monte Carlo
Revised: Missing Data Imputation and Acquisition with Deep Hierarchical Models and Hamiltonian Monte Carlo

Original: Abstract
Revised: Abstract

Original: Variational Autoencoders (VAEs) have recently been highly successful at imputing and acquiring heterogeneous missing data.
Revised: Variational Autoencoders (VAEs) have recently been highly successful at imputing and acquiring heterogeneous missing data.

Original: However, within this specific application domain, existing VAE methods are restricted by using only one layer of latent variables and strictly Gaussian posterior approximations.
Revised: However, within this specific application domain, existing VAE methods are restricted by using only one layer of latent variables and strictly Gaussian posterior approximations.

Original: To address these limitations, we present HH-VAEM, a Hierarchical VAE model for mixed-type incomplete data that uses Hamiltonian Monte Carlo with automatic hyper-parameter tuning for improved approximate inference.
Revised: To address these limitations, we present HH-VAEM, a Hierarchical VAE model for mixed-type incomplete data that uses Hamiltonian Monte Carlo with automatic hyper-parameter tuning for improved approximate inference.

Original: Our experiments show that HH-VAEM outperforms existing baselines in the tasks of missing data imputation and supervised learning with missing features.
Revised: Our experiments show that HH-VAEM outperforms existing baselines in the tasks of missing data imputation and supervised learning with missing features.

Original: Finally, we also present a sampling-based approach for efficiently computing the information gain when missing features are to be acquired with HH-VAEM.
Revised: Finally, we also present a sampling-based approach for efficiently computing the information gain when missing features are to be acquired with HH-VAEM.

Original: Our experiments show that this sampling-based approach is superior to alternatives based on Gaussian approximations.
Revised: Our experiments show that this sampling-based approach is superior to alternatives based on Gaussian approximations.

Original:  Introduction
Revised:  Introduction

Original: Many real-world unsupervised learning tasks require dealing with complicated datasets with mixed types (real, positive-valued, continuous, or discrete) and missing values.
Revised: Many real-world unsupervised learning tasks require dealing with complicated datasets with mixed types (real, positive-valued, continuous, or discrete) and missing values.

Original: For this purpose, variational autoencoders [24, 25, 41] stand out in the recent literature as robust generative models that efficiently handle high-dimensional data.
Revised: For this purpose, variational autoencoders [24, 25, 41] stand out in the recent literature as robust generative models that efficiently handle high-dimensional data.

Original: However, in their naive configuration, every data dimension is assumed to have similar statistical properties (i e., homogeneity), and all dimensions are considered to be completely observed.
Revised: However, in their naive configuration, every data dimension is assumed to have similar statistical properties (i e., homogeneity), and all dimensions are considered to be completely observed.

Original: Both assumptions won’t hold in many real-world scenarios.
Revised: Both assumptions won’t hold in many real-world scenarios.

Original: Recent works have adapted VAEs to handle incomplete [9, 14, 34, 36] and mixed-type data [18, 33, 38] and demonstrated improved performance in downstream tasks such as missing data imputation and active information acquisition.
Revised: Recent works have adapted VAEs to handle incomplete [9, 14, 34, 36] and mixed-type data [18, 33, 38] and demonstrated improved performance in downstream tasks such as missing data imputation and active information acquisition.

Original: Despite these advances, existing approaches are far from optimal as they are based on restrictive design choices: 1), only one layer of latent variables are considered; 2), Gaussian posterior approximations are usually adopted.
Revised: Despite these advances, existing approaches are far from optimal as they are based on restrictive design choices: 1), only one layer of latent variables are considered; 2), Gaussian posterior approximations are usually adopted.

Original: These will lead to limited flexibility and additional bias, especially under real-world settings with complex mixed-type incomplete data.
Revised: These will lead to limited flexibility and additional bias, especially under real-world settings with complex mixed-type incomplete data.

Original: In the literature, the issue of model flexibility and inference bias are often addressed separately.
Revised: In the literature, the issue of model flexibility and inference bias are often addressed separately.

Original: For example, approximate inference bias can be reduced by using Monte Carlo sampling [46, 51].
Revised: For example, approximate inference bias can be reduced by using Monte Carlo sampling [46, 50].

Original: More specifically, Hamiltonian Monte Carlo (HMC) [5, 12] stands out among MCMC methods in machine learning due to its superior efficiency for exploring the target density.
Revised: More specifically, Hamiltonian Monte Carlo (HMC) [5, 12] stands out among MCMC methods in machine learning due to its superior efficiency for exploring the target density.

Original: In the context of VAE, HMC has also been combined with stochastic variational inference [7] for improving the training of VAEs.
Revised: In the context of VAE, HMC has also been combined with stochastic variational inference [7] for improving the training of VAEs.

Original: On the other hand, the flexibility of VAEs can be improved by considering hierarchical VAEs with multiple layers of hidden variables [8, 35, 48, 53].
Revised: On the other hand, the flexibility of VAEs can be improved by considering hierarchical VAEs with multiple layers of hidden variables [8, 35, 48, 52].

Original: By using a hierarchical structure in the latent space, they enforce the information to flow from high-level representations to more specific observable factors, imitating the way information is often organized in the real world.
Revised: By using a hierarchical structure in the latent space, they enforce the information to flow from high-level representations to more specific observable factors, imitating the way information is often organized in the real world.

Original: However, the issue of modeling flexibility and approximate inference bias are often heavily intertwined, and addressing them simultaneously in a joint manner is highly non-trivial.
Revised: However, the issue of modeling flexibility and approximate inference bias are often heavily intertwined, and addressing them simultaneously in a joint manner is highly non-trivial.

Original: The hierarchical organization of the latent variables creates complicated posterior dependencies that are not straightforward to deal with and require special consideration.
Revised: The hierarchical organization of the latent variables creates complicated posterior dependencies that are not straightforward to deal with and require special consideration.

Original: To improve Gaussian approximate inference, most works opt by defining shared paths between the recognition and generative networks.
Revised: To improve Gaussian approximate inference, most works opt by defining shared paths between the recognition and generative networks.

Original: While this makes hierarchical VAEs practical, the bias introduced by the Gaussian approximations is still present.
Revised: While this makes hierarchical VAEs practical, the bias introduced by the Gaussian approximations is still present.

Original: To the best of our knowledge, none of the aforementioned hierarchical VAEs has been previously combined with Monte Carlo algorithms for improving over standard Gaussian approximate inference.
Revised: To the best of our knowledge, none of the aforementioned hierarchical VAEs has been previously combined with Monte Carlo algorithms for improving over standard Gaussian approximate inference.

Original: To overcome such limitations, we focus on training new hierarchical VAE models for heterogeneous mixed-type data with HMC.
Revised: To overcome such limitations, we focus on training new hierarchical VAE models for heterogeneous mixed-type data with HMC.

Original: Our models can be used for missing data imputation and for supervised learning with missing data.
Revised: Our models can be used for missing data imputation and for supervised learning with missing data.

Original: We also present a sampling-based framework that allows our models to perform accurate sequential active information acquisition.
Revised: We also present a sampling-based framework that allows our models to perform accurate sequential active information acquisition.

Original: Our main contributions are as follows:
Revised: Our main contributions are as follows:

Original: • We present HH-VAEM, a deep hierarchical model for handling mixed-type incomplete data that uses HMC with automatic hyper-parameter tuning for outperforming amortized variational inference by generating low bias samples from the true posterior.
Revised: • We present HH-VAEM, a deep hierarchical model for handling mixed-type incomplete data that uses HMC with automatic hyper-parameter tuning for outperforming amortized variational inference by generating low bias samples from the true posterior.

Original: • We propose a sampling-based strategy for missing feature acquisition that benefits from the improved inference of HH-VAEM.
Revised: • We propose a sampling-based strategy for missing feature acquisition that benefits from the improved inference of HH-VAEM.

Original: By using histograms to estimate the mutual information, this strategy achieves lower bias than other Gaussian-based alternatives.
Revised: By using histograms to estimate the mutual information, this strategy achieves lower bias than other Gaussian-based alternatives.

Original: • We exhaustively evaluate HH-VAEM in the tasks of 1) missing data imputation, 2) supervised learning with missing data and 3) information acquisition with our sampling-based strategy.
Revised: • We exhaustively evaluate HH-VAEM in the tasks of 1) missing data imputation, 2) supervised learning with missing data and 3) information acquisition with our sampling-based strategy.

Original: In all cases we report significant gains with respect to baselines.
Revised: In all cases we report significant gains with respect to baselines.

Original:  Related work
Revised:  Related work

Original: VAEs for mixed-type incomplete data
Revised: VAEs for mixed-type incomplete data

Original: Variational Autoencoders [24, 41] are deep generative models that make use of encoder and decoder networks for mapping data into a latent Gaussian distribution, and reconstructing the latent codes into the original observational space, respectively.
Revised: Variational Autoencoders [24, 41] are deep generative models that make use of encoder and decoder networks for mapping data into a latent Gaussian distribution, and reconstructing the latent codes into the original observational space, respectively.

Original: The parameters of these networks are trained using amortized Variational Inference [10, 56] optimizing a lower bound (ELBO) on the log evidence:
Revised: The parameters of these networks are trained using amortized Variational Inference [10, 55] optimizing a lower bound (ELBO) on the log evidence:

Original: E q z ( z | x ) log p θ ( x , z ) q ψ ( z | x ) , where the generative model p θ ( x , z ) can be expressed in terms of the likelihood p θ ( x | z ) and the prior p ( z ) .
Revised: E q z ( z | x ) log p θ ( x , z ) q ψ ( z | x ) , where the generative model p θ ( x , z ) can be expressed in terms of the likelihood p θ ( x | z ) and the prior p ( z ) .

Original: In mixed-type data, the vector x is composed by data from different types: real, positive real, categorical, binary, etc.
Revised: In mixed-type data, the vector x is composed by data from different types: real, positive real, categorical, binary, etc.

Original: A naive approach is to consider a factorized decoder using different likelihood contributions p θ ( x | z ) = (cid:81)
Revised: A naive approach is to consider a factorized decoder using different likelihood contributions p θ ( x | z ) = (cid:81)

Original: d p θ ( x d | z ) [1, 38].
Revised: d p θ ( x d | z ) [1, 38].

Original: Nonetheless, the problem of handling unbalanced likelihoods leads to the domination of some dimensions during the optimization process.
Revised: Nonetheless, the problem of handling unbalanced likelihoods leads to the domination of some dimensions during the optimization process.

Original: In [33], authors propose a solution using a set of marginal VAEs that encode each feature into a Gaussian uni-dimensional space, and a dependency VAE that captures the inter-dimensional dependencies more effectively using balanced Gaussian likelihoods.
Revised: In [33], authors propose a solution using a set of marginal VAEs that encode each feature into a Gaussian uni-dimensional space, and a dependency VAE that captures the inter-dimensional dependencies more effectively using balanced Gaussian likelihoods.

Original: By marginalizing each dimension of the decoder, incomplete data can be easily handled by dividing the vector x into the observed x O and unobserved x U parts.
Revised: By marginalizing each dimension of the decoder, incomplete data can be easily handled by dividing the vector x into the observed x O and unobserved x U parts.

Original: This methodology is completely valid when using the missing-at-random (MAR) assumption [30], i.e. assuming the missing mechanism is independent of the missing values.
Revised: This methodology is completely valid when using the missing-at-random (MAR) assumption [30], i.e. assuming the missing mechanism is independent of the missing values.

Original: In this our work, we use the same assumption.
Revised: In this our work, we use the same assumption.

Original: As proposed in [38] and [36], the ELBO objective is transformed into a lower bound on the observed data, and the unobserved data is replaced with zeros.
Revised: As proposed in [38] and [36], the ELBO objective is transformed into a lower bound on the observed data, and the unobserved data is replaced with zeros.

Original: 2.2 Hierarchical VAEs
Revised: 2.2 Hierarchical VAEs

Original: Hierarchical models have been successfully employed in deep generative modeling, [2, 44, 45].
Revised: Hierarchical models have been successfully employed in deep generative modeling, [2, 44, 45].

Original: In
Revised: In

Original: VAEs, defining a hierarchical latent space for VAEs can be straightforward.
Revised: VAEs, defining a hierarchical latent space for VAEs can be straightforward.

Original: Nevertheless, potential pitfalls require special attention.
Revised: Nevertheless, potential pitfalls require special attention.

Original: Concretely, if the decoder is powerful enough, the model tends to uniquely use the shallowest layers, ignoring the deepest ones and falling into the well-known problem of posterior collapse [35, 40, 54].
Revised: Concretely, if the decoder is powerful enough, the model tends to uniquely use the shallowest layers, ignoring the deepest ones and falling into the well-known problem of posterior collapse [35, 40, 53].

Original: In the last few years, several works have investigated possible hierarchical structures for VAEs.
Revised: In the last few years, several works have investigated possible hierarchical structures for VAEs.

Original: In [48], a bottom-up deterministic path is used along with a top-down inference path that shares parameters with the top-down structure of the generative model.
Revised: In [48], a bottom-up deterministic path is used along with a top-down inference path that shares parameters with the top-down structure of the generative model.

Original: In [35], the authors use a bidirectional stochastic inference path.
Revised: In [35], the authors use a bidirectional stochastic inference path.

Original: More recently, [53] or [8] have adapted these architectures to complex datasets and high quality images.
Revised: More recently, [52] or [8] have adapted these architectures to complex datasets and high quality images.

Original: Possibly motivated by the residual connections in [23], all these works use a shared path between recognition and generative models that helps in tying the divergences between approximations and priors in the ELBO.
Revised: Possibly motivated by the residual connections in [23], all these works use a shared path between recognition and generative models that helps in tying the divergences between approximations and priors in the ELBO.

Original: 2.3 Hamiltonian Monte Carlo
Revised: 2.3 Hamiltonian Monte Carlo

Original: HMC [4, 12, 39] is a particularly effective MCMC algorithm for sampling from a target distribution p ( z ) =
Revised: HMC [4, 12, 39] is a particularly effective MCMC algorithm for sampling from a target distribution p ( z ) =

Original: 1 Z p ∗ ( z ) where Z is the normalization constant and z is a d -dimensional vector.
Revised: 1 Z p ∗ ( z ) where Z is the normalization constant and z is a d -dimensional vector.

Original: By augmenting this model to p ( r , z ) =
Revised: By augmenting this model to p ( r , z ) =

Original: N ( r ; 0 , M ) p ( z ) , and denoting r as the momentum variable with diagonal covariance matrix M , with the same dimensionality as z , HMC samples are obtained from the distribution by simulating the time-evolution of a fictitious physical system.
Revised: N ( r ; 0 , M ) p ( z ) , and denoting r as the momentum variable with diagonal covariance matrix M , with the same dimensionality as z , HMC samples are obtained from the distribution by simulating the time-evolution of a fictitious physical system.

Original: The algorithm starts by firstly sampling z and r from an initial proposal and the momentum distribution, respectively.
Revised: The algorithm starts by firstly sampling z and r from an initial proposal and the momentum distribution, respectively.

Original: Chains with length T are built by recurrently proposing and accepting new states.
Revised: Chains with length T are built by recurrently proposing and accepting new states.

Original: To propose a new state, the Hamiltonian dynamics are simulated using a symplectic integrator,
Revised: To propose a new state, the Hamiltonian dynamics are simulated using a symplectic integrator,

Original: Leapfrog being the most common choice.
Revised: Leapfrog being the most common choice.

Original: The following updates are repeated for l = 1 : LF steps:
Revised: The following updates are repeated for l = 1 : LF steps:

Original: [Equation]
Revised: [Equation]

Original: (1) where ⊙ refers to the Hadamard product, and ϕ is the step size hyperparameter.
Revised: (1) where ⊙ refers to the Hadamard product, and ϕ is the step size hyperparameter.

Original: Although it is typically defined as a scalar for simplicity, a d -dimensional vector can be considered to apply different step sizes per dimension, or further, as considered in this work, a T × d matrix to apply different steps per each proposal of the chain.
Revised: Although it is typically defined as a scalar for simplicity, a d -dimensional vector can be considered to apply different step sizes per dimension, or further, as considered in this work, a T × d matrix to apply different steps per each proposal of the chain.

Original: The new proposal ( z ′ , r ′ ) is accepted with probability min [1 , exp( − H ( z ′ , r ′ )
Revised: The new proposal ( z ′ , r ′ ) is accepted with probability min [1 , exp( − H ( z ′ , r ′ )

Original: + H ( z , r ))] , where
Revised: + H ( z , r ))] , where

Original: [Equation]
Revised: [Equation]

Original: For the consecutive T proposals, a new momentum r is resampled and the updates of eq.
Revised: For the consecutive T proposals, a new momentum r is resampled and the updates of eq.

Original: (1) are repeated for LF steps to update the state if ( z ′ , r ′ ) is accepted.
Revised: (1) are repeated for LF steps to update the state if ( z ′ , r ′ ) is accepted.

Original: 2.4 Active Feature Acquisition
Revised: 2.4 Active Feature Acquisition

Original: Among all the Active Learning techniques, Active Feature Acquisition [20, 37, 43, 50] is of special interest in cost-sensitive applications for modeling a trade-off between the improvement of predictions and the cost of acquiring new data at the feature level.
Revised: Among all the Active Learning techniques, Active Feature Acquisition [20, 37, 43, 49] is of special interest in cost-sensitive applications for modeling a trade-off between the improvement of predictions and the cost of acquiring new data at the feature level.

Original: Several works in the recent literature have studied methods for performing the task of sequentially acquiring high-value information by selecting features that maximize an information theoretical reward function and enhance the accuracy of the predictions.
Revised: Several works in the recent literature have studied methods for performing the task of sequentially acquiring high-value information by selecting features that maximize an information theoretical reward function and enhance the accuracy of the predictions.

Original: This task is denoted by SAIA (Sequential Active Information Acquisition).
Revised: This task is denoted by SAIA (Sequential Active Information Acquisition).

Original: In [34], an efficient method is proposed for approximating a non-tractable reward by using the encoder of a VAE that handles missing data.
Revised: In [34], an efficient method is proposed for approximating a non-tractable reward by using the encoder of a VAE that handles missing data.

Original: In [33] they extend this method for handling mixed-type data.
Revised: In [33] they extend this method for handling mixed-type data.

Original: Both works estimate the reward by relying on Gaussian approximations given by the encoder networks.
Revised: Both works estimate the reward by relying on Gaussian approximations given by the encoder networks.

Original:  Hamiltonian hierarchical VAE for mixed-type incomplete data (HH-VAEM)
Revised:  Hamiltonian hierarchical VAE for mixed-type incomplete data (HH-VAEM)

Original: The HH-VAEM model (Figure 1 (a)) is a Hierarchical VAE for mixed-type, incomplete data that incorporates HMC with automatic hyper-parameter optimization for sampling from the posterior of the latent variables.
Revised: The HH-VAEM model (Figure 1 (a)) is a Hierarchical VAE for mixed-type, incomplete data that incorporates HMC with automatic hyper-parameter optimization for sampling from the posterior of the latent variables.

Original: In a first stage, the mixed-type data is encoded into marginal Gaussian posterior distributions as given by univariate VAEs fitted to each data dimension.
Revised: In a first stage, the mixed-type data is encoded into marginal Gaussian posterior distributions as given by univariate VAEs fitted to each data dimension.

Original: In a second stage, a hierarchical structure captures the dependencies among the standarized, homogeneous dimensions with well-balanced Gaussian likelihoods.
Revised: In a second stage, a hierarchical structure captures the dependencies among the standarized, homogeneous dimensions with well-balanced Gaussian likelihoods.

Original: The model is trained using samples from the posterior of the
Revised: The model is trained using samples from the posterior of the

Original: [Figure]
Revised: [Figure]

Original: hierarchical latent variables by means of HMC, whilst the HMC hyper-parameters are automatically tuned.
Revised: hierarchical latent variables by means of HMC, whilst the HMC hyper-parameters are automatically tuned.

Original: A more detailed description is provided in the following subsections.
Revised: A more detailed description is provided in the following subsections.

Original: 3.1 Notation
Revised: 3.1 Notation

Original: The model generates both data x ∈ R D and output y ∈ R P , where each of these variables is divided into observed parts x O , y O and unobserved parts x U , y U .
Revised: The model generates both data x ∈ R D and output y ∈ R P , where each of these variables is divided into observed parts x O , y O and unobserved parts x U , y U .

Original: Each dimension of x is denoted by x d .
Revised: Each dimension of x is denoted by x d .

Original: A training set is composed of N observations as tuples ( x ( n ) O , y ( n ) O ) .
Revised: A training set is composed of N observations as tuples ( x ( n ) O , y ( n ) O ) .

Original: For ease of notation, we omit the observation index n , and the objectives are presented for a single observation.
Revised: For ease of notation, we omit the observation index n , and the objectives are presented for a single observation.

Original: The dependency latent space is composed of L latent variables [ ϵ 1 , ..., ϵ L ] , with ϵ l ∈ R m l .
Revised: The dependency latent space is composed of L latent variables [ ϵ 1 , ..., ϵ L ] , with ϵ l ∈ R m l .

Original: The dimension of the joint latent distribution is (cid:80) l m l = M .
Revised: The dimension of the joint latent distribution is (cid:80) l m l = M .

Original: In the marginal VAEs, the latent variables z d are unidimensional.
Revised: In the marginal VAEs, the latent variables z d are unidimensional.

Original: 3.2 Handling heterogeneous incomplete data
Revised: 3.2 Handling heterogeneous incomplete data

Original: Following the strategy proposed by [33], we perform a two-staged approach for handling heterogeneous data.
Revised: Following the strategy proposed by [33], we perform a two-staged approach for handling heterogeneous data.

Original: The marginal distribution of each feature p θ d ( x d ) is modeled by a one-dimensional
Revised: The marginal distribution of each feature p θ d ( x d ) is modeled by a one-dimensional

Original: VAE.
Revised: VAE.

Original: First, the D marginal VAEs are trained independently by maximizing the marginal ELBO over observed points
Revised: First, the D marginal VAEs are trained independently by maximizing the marginal ELBO over observed points

Original: [Equation]
Revised: [Equation]

Original: where 1 ( x d ∈ x O ) is an indicator function that activates the ELBO when the feature is observed.
Revised: where 1 ( x d ∈ x O ) is an indicator function that activates the ELBO when the feature is observed.

Original: Under the missing-at-random assumption, which is the one considered in this work, Equation (3) leads to a lower bound on observed data likelihoods.
Revised: Under the missing-at-random assumption, which is the one considered in this work, Equation (3) leads to a lower bound on observed data likelihoods.

Original: Second, a dependency VAE encodes a vector z with the concatenated samples from the marginal posteriors q γ d ( z d | x d ) into the global latent variable h , using zero-filling for the unobserved variables.
Revised: Second, a dependency VAE encodes a vector z with the concatenated samples from the marginal posteriors q γ d ( z d | x d ) into the global latent variable h , using zero-filling for the unobserved variables.

Original: By using this approach, z is now homogeneous and can then be easily modeled using a standard Gaussian decoder.
Revised: By using this approach, z is now homogeneous and can then be easily modeled using a standard Gaussian decoder.

Original: By contrast, other works [13, 38] directly operate with different decoding likelihoods per data type.
Revised: By contrast, other works [13, 38] directly operate with different decoding likelihoods per data type.

Original: This approach often leads to having very different magnitudes in the ELBO and may reduce learning efficiency.
Revised: This approach often leads to having very different magnitudes in the ELBO and may reduce learning efficiency.

Original: The ELBO for the second stage dependency VAE is
Revised: The ELBO for the second stage dependency VAE is

Original: [Equation]
Revised: [Equation]

Original: where ϵ = { ϵ 1 , ..., ϵ L } is a set of reparameterized hierarchical latent variables.
Revised: where ϵ = { ϵ 1 , ..., ϵ L } is a set of reparameterized hierarchical latent variables.

Original: Further details on the design of the hierarchical dependency VAE are provided below.
Revised: Further details on the design of the hierarchical dependency VAE are provided below.

Original: 3.3 Predictive enhancement
Revised: 3.3 Predictive enhancement

Original: The combination of generative and discriminative models is an effective well-studied strategy for dealing with predictive models under missing data [15, 52].
Revised: The combination of generative and discriminative models is an effective well-studied strategy for dealing with predictive models under missing data [15, 51].

Original: In [15], they model p ( x ) for imputing missing data using a Gaussian Mixture Model.
Revised: In [15], they model p ( x ) for imputing missing data using a Gaussian Mixture Model.

Original: In a deep learning context, recent supervised VAE models have revisited this combination [21, 47] or used factorisations of type p ( z ) p ( x | z ) p ( y | z ) [22] to learn meaningful representations.
Revised: In a deep learning context, recent supervised VAE models have revisited this combination [21, 47] or used factorisations of type p ( z ) p ( x | z ) p ( y | z ) [22] to learn meaningful representations.

Original: In [28] the authors propose a deep generative model with factorisation p ( z ) p ( x | z ) p ( y | x , z ) for detecting adversarial attacks.
Revised: In [28] the authors propose a deep generative model with factorisation p ( z ) p ( x | z ) p ( y | x , z ) for detecting adversarial attacks.

Original: With the aim at reinforcing the prediction of the variable of interest, we turn into a supervised model by including a separate predictor for p θ y ( y | ˆ x , h ) , apart from the decoder p θ
Revised: With the aim at reinforcing the prediction of the variable of interest, we turn into a supervised model by including a separate predictor for p θ y ( y | ˆ x , h ) , apart from the decoder p θ

Original: z
Revised: z

Original: ( z | h ) .
Revised: ( z | h ) .

Original: The vector ˆ x = ( x i ∈
Revised: The vector ˆ x = ( x i ∈

Original: x O , ˆ x j ∈ x U ) includes the observed part and imputation of the missing variables ˆ x j by decoding the latent h into z using p ( z | h 1 ) , and each dimension z j ∈ z into ˆ x j using p ( x j | z j ) .
Revised: x O , ˆ x j ∈ x U ) includes the observed part and imputation of the missing variables ˆ x j by decoding the latent h into z using p ( z | h 1 ) , and each dimension z j ∈ z into ˆ x j using p ( x j | z j ) .

Original: The predictor parameters θ y are optimized along with the decoder parameters θ z .
Revised: The predictor parameters θ y are optimized along with the decoder parameters θ z .

Original: 3.4 Hierarchical reparameterized latent space
Revised: 3.4 Hierarchical reparameterized latent space

Original: A hierarchical structure over the latent space
Revised: A hierarchical structure over the latent space

Original: h
Revised: h

Original: = { h 1 , ..., h L } enriches the prior assumptions and permits a flexible generation of data in a more natural fashion.
Revised: = { h 1 , ..., h L } enriches the prior assumptions and permits a flexible generation of data in a more natural fashion.

Original: Nevertheless, as stated in [4, 5], HMC can be pathological when used for sampling from hierarchical densities, where the magnitude of autoregressive variations increase with the depth.
Revised: Nevertheless, as stated in [4, 5], HMC can be pathological when used for sampling from hierarchical densities, where the magnitude of autoregressive variations increase with the depth.

Original: For approximating the Hamiltonian dynamics, inside each Leapfrog integrator step, gradients ∇ h 1: L log p ∗ ( h 1: L ) are required.
Revised: For approximating the Hamiltonian dynamics, inside each Leapfrog integrator step, gradients ∇ h 1: L log p ∗ ( h 1: L ) are required.

Original: Due to the strong curvature regions, huge norm of high-order derivatives are backpropagated and might eventually explode, ending in overflow issues (Figure 35 in [5]).
Revised: Due to the strong curvature regions, huge norm of high-order derivatives are backpropagated and might eventually explode, ending in overflow issues (Figure 35 in [5]).

Original: If we were to run our HMC method over the hierarchical variables without any reparameterization (Figure 4a), by the time the states reached the aforementioned problematic regions, the integrator would diverge and we would experience the aforementioned overflow problems.
Revised: If we were to run our HMC method over the hierarchical variables without any reparameterization (Figure 4a), by the time the states reached the aforementioned problematic regions, the integrator would diverge and we would experience the aforementioned overflow problems.

Original: By rejecting these problematic states, chains would get stuck close to the proposal and the hierarchical density would not be properly explored (concluding that HMC would not improve the Gaussian proposal).
Revised: By rejecting these problematic states, chains would get stuck close to the proposal and the hierarchical density would not be properly explored (concluding that HMC would not improve the Gaussian proposal).

Original: To give an example, in [53], the AR path p ( h l | h <l ) would provoke instabilities inside the HMC integrator due to huge gradients in ∇ h 1: L log p ∗ ( h 1: L ) .
Revised: To give an example, in [52], the AR path p ( h l | h <l ) would provoke instabilities inside the HMC integrator due to huge gradients in ∇ h 1: L log p ∗ ( h 1: L ) .

Original: We successfully solved this issue by introducing a hierarchical reparameterization technique.
Revised: We successfully solved this issue by introducing a hierarchical reparameterization technique.

Original: The representation at each layer is reparameterized from variable ϵ l with standard Gaussian prior p ( ϵ l 
Revised: The representation at each layer is reparameterized from variable ϵ l with standard Gaussian prior p ( ϵ l )

Original: [Equation]
Revised: [Equation]

Original: where the functions f µ l ( h l +1 ) and f σ l ( h l +1 ) are applied by NNs with parameters θ l = { θ µ l , θ σ l } .
Revised: where the functions f µ l ( h l +1 ) and f σ l ( h l +1 ) are applied by NNs with parameters θ l = { θ µ l , θ σ l } .

Original: The result is equivalent as learning the mean and covariance of autoregressive variables (see Figurefor illustrative details).
Revised: The result is equivalent as learning the mean and covariance of autoregressive variables (see Figurefor illustrative details).

Original: However, thanks to this trick, we relax the dependencies among the latent variables, resulting in a smoother joint posterior density p ( ϵ | x
Revised: However, thanks to this trick, we relax the dependencies among the latent variables, resulting in a smoother joint posterior density p ( ϵ | x

Original: O , y O ) .
Revised: O , y O ) .

Original: Performing the inference over ϵ = { ϵ 1 , ..., ϵ L } leads to a better posed basis for running our HMC optimization, detailed in section 3.5, and avoids the necessity of employing more advanced HMC samplers like [5, 16].
Revised: Performing the inference over ϵ = { ϵ 1 , ..., ϵ L } leads to a better posed basis for running our HMC optimization, detailed in section 3.5, and avoids the necessity of employing more advanced HMC samplers like [5, 16].

Original: We include further details on the pathological behavior and demonstration of the effectiveness of our solution in Section A.8.
Revised: We include further details on the pathological behavior and demonstration of the effectiveness of our solution in Section B.3 of the Supplementary.

Original: Provided the promising results we obtain in Section 5, we propose our reparameterization trick as a novel contribution for solving the pathological behavior of HMC combined with hierarchical VAEs.
Revised: Provided the promising results we obtain in Section 5, we propose our reparameterization trick as a novel contribution for solving the pathological behavior of HMC combined with hierarchical VAEs.

Original: For the sake of simplicity, we name the generative parameters as θ = { θ z , θ y , θ 1 , ..., θ L } .
Revised: For the sake of simplicity, we name the generative parameters as θ = { θ z , θ y , θ 1 , ..., θ L } .

Original: The dependency ELBO under this hierarchical reparameterized model becomes
Revised: The dependency ELBO under this hierarchical reparameterized model becomes

Original: [Equation]
Revised: [Equation]

Original: We name r l the hidden representation at each layer, and defining r 

Revised: We name r l the hidden representation at each layer, and defining r

Original: = { x O , y O } , we employ NNs with parameters ψ r l for computing r l = f r
Revised: = { x O , y O } , we employ NNs with parameters ψ r l for computing r l = f r

Original: ( r l − 1 ) .
Revised: ( r l − 1 ) .

Original: These vectors are mapped into the parameters of the variational posterior q ψ l ( ϵ l | x O , y O ) , using NNs for computing the mean as g µ l ( r l ) and the covariance as g σ l ( r l ) , with parameters ψ µ l and ψ σ l .
Revised: These vectors are mapped into the parameters of the variational posterior q ψ l ( ϵ l | x O , y O ) , using NNs for computing the mean as g µ l ( r l ) and the covariance as g σ l ( r l ) , with parameters ψ µ l and ψ σ l .

Original: With compactness purposes, we will denote the encoder parameters as ψ = { ψ 1 , ..., ψ L } , with ψ l = { ψ r l , ψ µ l , ψ σ l } .
Revised: With compactness purposes, we will denote the encoder parameters as ψ = { ψ 1 , ..., ψ L } , with ψ l = { ψ r l , ψ µ l , ψ σ l } .

Original: 3.5 HMC over the hierarchical density
Revised: 3.5 HMC over the hierarchical density

Original: In recent works, HMC has been combined with deep generative models for improving the inference of the latent variables by obtaining better samples from the posterior [7, 19].
Revised: In recent works, HMC has been combined with deep generative models for improving the inference of the latent variables by obtaining better samples from the posterior [7, 19].

Original: In this work, we propose to transcend these previous approaches and build a generalized method for sampling from complicated, hierarchical latent structures composed by several layers.
Revised: In this work, we propose to transcend these previous approaches and build a generalized method for sampling from complicated, hierarchical latent structures composed by several layers.

Original: Inspired by [6] and their method for sampling from the posterior within a vanilla VAE framework while tuning the HMC hyperparameters, we follow a procedure for training the dependency model where i) during a pre-training stage, the encoder and decoder are optimized using standard VI and the ELBO from equation (6), and ii) using the pre-trained encoder for starting from a good proposal [19], HMC samples are obtained to follow the true posterior and jointly optimize the generative model and the HMC hyperparameters.
Revised: Inspired by [6] and their method for sampling from the posterior within a vanilla VAE framework while tuning the HMC hyperparameters, we follow a procedure for training the dependency model where i) during a pre-training stage, the encoder and decoder are optimized using standard VI and the ELBO from equation (6), and ii) using the pre-trained encoder for starting from a good proposal [19], HMC samples are obtained to follow the true posterior and jointly optimize the generative model and the HMC hyperparameters.

Original: In Figure 1 (b) we include an illustrative example.
Revised: In Figure 1 (b) we include an illustrative example.

Original: We denote by q ( T ) ϕ
Revised: We denote by q ( T ) ϕ

Original: ( ϵ |
Revised: ( ϵ |

Original: z O , x O , y O )
Revised: z O , x O , y O )

Original: the implicit distribution for the posterior after T HMC steps.
Revised: the implicit distribution for the posterior after T HMC steps.

Original: The hyper-parameters of HMC are named ϕ , a T × d matrix containing the step sizes for each dimension at each step of the chain.
Revised: The hyper-parameters of HMC are named ϕ , a T × d matrix containing the step sizes for each dimension at each step of the chain.

Original: Within this perspective, the hyper-parameters can be optimized using variational inference by maximizing the ELBO
Revised: Within this perspective, the hyper-parameters can be optimized using variational inference by maximizing the ELBO

Original: [Equation]
Revised: [Equation]

Original: (7) where the first part is the HMC objective, and can be easily estimated via Monte Carlo
Revised: (7) where the first part is the HMC objective, and can be easily estimated via Monte Carlo

Original: L HMC ( z O , y O ; { θ, ψ, ϕ } ) =
Revised: L HMC ( z O , y O ; { θ, ψ, ϕ } ) =

Original: E q ( T ) ϕ
Revised: E q ( T ) ϕ

Original: ( ϵ ) [log p θ ( z O |h 1 ) + log p θ ( y O
Revised: ( ϵ ) [log p [Equation] ( z O |

Original:  | ˆ x
Revised: [Equation] + log p θ ( y O | ˆ x

Original: [Equation]
Revised: [Equation]

Original: h 1 )
Revised: h [Equation]

Original: +
Revised: +

Original: [Equation]
Revised: [Equation]

Original: (cid:88)
Revised: (cid:88)

Original: [Equation]
Revised: [Equation]

Original: p
Revised: p

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Nevertheless, the entropy term H [ q ( T ) ϕ
Revised: Nevertheless, the entropy term H [ q ( T ) ϕ

Original: ( ϵ | x O , y O )] in Equation (7) is not tractable since we are not able to explicitly evaluate the distribution q ( T )
Revised: ( ϵ | x O , y O )] in Equation (7) is not tractable since we are not able to explicitly evaluate the distribution q ( T )

Original: ϕ
Revised: ϕ

Original: ( ϵ | x O , y O ) .
Revised: ( ϵ | x O , y O ) .

Original: Although optimizing the first term might result in a well-posed algorithm, this would bring consequences that must be considered.
Revised: Although optimizing the first term might result in a well-posed algorithm, this would bring consequences that must be considered.

Original: Namely, without a proper regularization term, and in case the initial proposal q ( T ) ϕ
Revised: Namely, without a proper regularization term, and in case the initial proposal q ( T ) ϕ

Original: ( ϵ | x O , y O ) is concentrated in high density regions, the chains would barely move from the initial state and only these regions with high density would be explored (see Section A.3 for illustrative details).
Revised: ( ϵ | x O , y O ) is concentrated in high density regions, the chains would barely move from the initial state and only these regions with high density would be explored (see Section B.1 of the Supplementary for illustrative details).

Original: To cope with this problem, we define an inflation parameter s to increase the variance of the proposal q ϕ ( ϵ |
Revised: To cope with this problem, we define an inflation parameter s to increase the variance of the proposal q ϕ ( ϵ |

Original: z O , x O , y O ) given by the Gaussian encoder, ending in the proposal (cid:81)
Revised: z O , x O , y O ) given by the Gaussian encoder, ending in the proposal (cid:81)

Original: Ll =1 N ( g µ l ( r l ) , s l · g σ l ( r l )) .
Revised: Ll =1 N ( g µ l ( r l ) , s l · g σ l ( r l )) .

Original: Whilst in [6] the authors define this parameter as a scalar factor applied to all the latent dimensions, in our work, we extend this to apply a different inflation factor at each latent level s = { s 1 , ..., s L } .
Revised: Whilst in [6] the authors define this parameter as a scalar factor applied to all the latent dimensions, in our work, we extend this to apply a different inflation factor at each latent level s = { s 1 , ..., s L } .

Original: In order to tune these inflations we ensure a wider coverage of the density by minimizing the Sliced Kernelized Stein Discrepancy (SKSD) 
Revised: In order to tune these inflations we ensure a wider coverage of the density by minimizing the Sliced Kernelized Stein Discrepancy (SKSD) [17]

Original: [Equation][Equation]
Revised: [Equation]

Original: which fits perfectly our requirements, since only requires samples from HMC and gradients ∇ ϵ log p ( z O , y O , ϵ ) for measuring a discrepancy between the implicit and the true posterior.
Revised: which fits perfectly our requirements, since only requires samples from HMC and gradients ∇ ϵ log p ( z O , y O , ϵ ) for measuring a discrepancy between the implicit and the true posterior.

Original: Further, the SKSD performs better than other discrepancies like [31] in high dimensional latent spaces.
Revised: Further, the SKSD performs better than other discrepancies like [31] in high dimensional latent spaces.

Original: We provide in Section A.3 a toy demonstration on the efficacy of the HMC optimization, and in Section A.4 a demonstration of the optimization convergence.
Revised: We provide in Section B.1 of the Supplementary a toy demonstration on the efficacy of the HMC optimization, and in Section B.2 a demonstration of the optimization convergence.

Original: 3.6 HH-VAEM optimization
Revised: 3.6 HH-VAEM optimization

Original: The optimization of the HH-VAEM is divided into three stages.
Revised: The optimization of the HH-VAEM is divided into three stages.

Original: In a first stage, we train one independent marginal VAE per dimension.
Revised: In a first stage, we train one independent marginal VAE per dimension.

Original: In a second stage, the dependency VAE is trained, using as inputs the concatenation of the encoded dimensions z
Revised: In a second stage, the dependency VAE is trained, using as inputs the concatenation of the encoded dimensions z

Original: = { z 1 , ..., z D } and the target y .
Revised: = { z 1 , ..., z D } and the target y .

Original: Finally, in a third stage, the HMC hyperparameters, the decoder and the predictor are tuned using the HMC objective, the inflation parameter is trained using the SKSD discrepancy, and the encoder is trained used the ELBO.
Revised: Finally, in a third stage, the HMC hyperparameters, the decoder and the predictor are tuned using the HMC objective, the inflation parameter is trained using the SKSD discrepancy, and the encoder is trained used the ELBO.

Original: The pseudocode for HH-VAEM training is shown in Algorithm 1.
Revised: The pseudocode for HH-VAEM training is shown in Algorithm 1.

Original: 3.7 Computational cost
Revised: 3.7 Computational cost

Original: In a VAE, the data is fed to the encoder with aim at obtaining the variational parameters for sampling from the Gaussian approximated posterior.
Revised: In a VAE, the data is fed to the encoder with aim at obtaining the variational parameters for sampling from the Gaussian approximated posterior.

Original: In our method, the data is similarly encoded to obtain the initial Gaussian proposal q (0) , and the samples from this distribution are updated for T cycles to obtain the implicit q ( T ) using HMC.
Revised: In our method, the data is similarly encoded to obtain the initial Gaussian proposal q (0) , and the samples from this distribution are updated for T cycles to obtain the implicit q ( T ) using HMC.

Original: Within each of these iterations, L leapfrog steps (1) are executed.
Revised: Within each of these iterations, L leapfrog steps (1) are executed.

Original: For each of these steps, the computation of the gradients ∇ ϵ l log p ( z , y , ϵ l ) and ∇ ϵ l +1 log p ( z , y , ϵ l +1 ) is required.
Revised: For each of these steps, the computation of the gradients ∇ ϵ l log p ( z , y , ϵ l ) and ∇ ϵ l +1 log p ( z , y , ϵ l +1 ) is required.

Original: To obtain these gradients, we need to i) compute the parameters of the likelihood p ( z , y | ϵ ) that are given by the decoder ( p ( z | h 1 ) ) and predictor ( p ( y | ˆ x , h 1 ) ), ii) evaluate the likelihood and iii) perform the automatic differentiation.
Revised: To obtain these gradients, we need to i) compute the parameters of the likelihood p ( z , y | ϵ ) that are given by the decoder ( p ( z | h 1 ) ) and predictor ( p ( y | ˆ x , h 1 ) ), ii) evaluate the likelihood and iii) perform the automatic differentiation.

Original: Thus, for running our method, an additional cost from both decoding and performing differentiation a total of 2 TL times is introduced.
Revised: Thus, for running our method, an additional cost from both decoding and performing differentiation a total of 2 TL times is introduced.

Original: Input: data (cid:16) x (1: N ) O , y (1: N ) O (cid:17) , steps: T d , T V
Revised: Input: data (cid:16) x (1: N ) O , y (1: N ) O (cid:17) , steps: T d , T V

Original: I , T HMC
Revised: I , T HMC

Original: Parameters: γ , θ , ψ , ϕ , s
Revised: Parameters: γ , θ , ψ , ϕ , s

Original: S TAGE 1: MARGINAL VAE S for d = 1 to D do
Revised: S TAGE 1: MARGINAL VAE S for d = 1 to D do

Original: Initialize marginal VAE { θ d , γ d } for t = 1 to T d do γ t +1 d , θ t +1 d ← Adam γ td ,θ td
Revised: Initialize marginal VAE { θ d , γ d } for t = 1 to T d do γ t +1 d , θ t +1 d ← Adam γ td ,θ td

Original: ( L d ) end for end for
Revised: ( L d ) end for end for

Original: S TAGE 2: DEPENDENCY VAE for t = 1 to T V AE do θ t +1 , ψ t +1 ← Adam θ t ,ψ t ( L V I ) end for
Revised: S TAGE 2: DEPENDENCY VAE for t = 1 to T V AE do θ t +1 , ψ t +1 ← Adam θ t ,ψ t ( L V I ) end for

Original: S TAGE 3: J OINTLY OPTIMIZING VAE + HMC for t = 1 to T HMC do ψ t +1 ← Adam ψ t ( L V I ) θ t +1 , ϕ t +1 ← Adam θ t ,ϕ t ( L HMC ) s t +1 ← Adam s t ( L SKSD ) end for
Revised: S TAGE 3: J OINTLY OPTIMIZING VAE + HMC for t = 1 to T HMC do ψ t +1 ← Adam ψ t ( L V I ) θ t +1 , ϕ t +1 ← Adam θ t ,ϕ t ( L HMC ) s t +1 ← Adam s t ( L SKSD ) end for

Original: By jointly optimizing the HMC hyperparameters we are able to achieve faster convergence with smaller lengths.
Revised: By jointly optimizing the HMC hyperparameters we are able to achieve faster convergence with smaller lengths.

Original: To reduce the computational cost, we optimize the hyperparameters in a final training stage, since convergence is rapidly achieved (as observed empirically in Fig.
Revised: To reduce the computational cost, we optimize the hyperparameters in a final training stage, since convergence is rapidly achieved (as demonstrated empirically in Section B.2 of the Supplementary).

Original: At test, samples from the Gaussian q (0) (faster and cheaper), or from HMC q ( T ) (slower and better) can be used to fit computational constraints.
Revised: At test, samples from the Gaussian q (0) (faster and cheaper), or from HMC q ( T ) (slower and better) can be used to fit computational constraints.

Original:  Sampling-based Active Learning
Revised:  Sampling-based Active Learning

Original: Considering that the input data are tuples of observed and missing features { x O , x U } , our Active Learning framework follows [33, 34] and determines which feature x i ∈
Revised: Considering that the input data are tuples of observed and missing features { x O , x U } , our Active Learning framework follows [33, 34] and determines which feature x i ∈

Original: x U will enhance the prediction of the target y the most for a particular x O .
Revised: x U will enhance the prediction of the target y the most for a particular x O .

Original: Concretely, in a Sequential Active Information Acquisition (SAIA) scenario, this decision is taken sequentially to optimally increase knowledge and accurately predict y .
Revised: Concretely, in a Sequential Active Information Acquisition (SAIA) scenario , this decision is taken sequentially to optimally increase knowledge and accurately predict y .

Original: From a information theoretical perspective, this task can be performed recurrently by maximizing a reward function R at each step d .
Revised: From a information theoretical perspective, this task can be performed recurrently by maximizing a reward function R at each step d .

Original: This reward might represent abstract quantities of interest like the cost or benefit of acquiring x
Revised: This reward might represent abstract quantities of interest like the cost or benefit of acquiring x

Original: i (depending on the sign).
Revised: i (depending on the sign).

Original: In Bayesian experimental analysis, R is the expected gain of information [29].
Revised: In Bayesian experimental analysis, R is the expected gain of information [29].

Original: Following [3], we can define it as
Revised: Following [3], we can define it as

Original: [Equation]
Revised: [Equation]

Original: (10) where i is the index of each unobserved feature.
Revised: where i is the index of each unobserved feature.

Original: Intuitively, this quantity can be interpreted as the expected change in the predictive distribution when x i is observed.
Revised: Intuitively, this quantity can be interpreted as the expected change in the predictive distribution when x i is observed.

Original: The reward needs to be estimated via Monte Carlo by sampling from p ( x i | x O ) .
Revised: The reward needs to be estimated via Monte Carlo by sampling from p ( x i | x O ) .

Original: With a robust generative model that handles missing data like HH-VAEM, these samples are easily obtained: first, using HMC, we sample ϵ ( T ) from p ( ϵ | x O ) .
Revised: With a robust generative model that handles missing data like HH-VAEM, these samples are easily obtained: first, using HMC, we sample ϵ ( T ) from p ( ϵ | x O ) .

Original: Second, we decode these samples to obtain x i from p ( z | h 1 ) and p ( x i | z d ) .
Revised: Second, we decode these samples to obtain x i from p ( z | h 1 ) and p ( x i | z d ) .

Original: Nonetheless, the reward defined in (10) is intractable since both p ( y | x i ) and p ( y | x i , x O ) are intractable: computing them requires to integrate out the latent variables.
Revised: Nonetheless, the reward defined in (10) is intractable since both p ( y | x i ) and p ( y | x i , x O ) are intractable: computing them requires to integrate out the latent variables.

Original: This motivates the authors of [33, 34] to present a transformation of the reward for being computed in the latent space using the encoder network.
Revised: This motivates the authors of [33, 34] to present a transformation of the reward for being computed in the latent space using the encoder network.

Original: Although they prove that this transformation effectively provides a good estimation in several datasets, we demonstrate that for low dimensional targets (commonly one or two dimensions), an approximation using histograms is more effective.
Revised: Although they prove that this transformation effectively provides a good estimation in several datasets, we demonstrate that for low dimensional targets (commonly one or two dimensions), an approximation using histograms is more effective.

Original: Concretely, the reward in (10) can be rewritten as
Revised: Concretely, the reward in (10) can be rewritten as

Original: [Equation]
Revised: [Equation]

Original: While a set of advanced non-parametric estimators of the mutual information are available [26, 42], many are not easilly adapted for parallelization.
Revised: While a set of advanced non-parametric estimators of the mutual information are available [26, 42], many are not easilly adapted for parallelization.

Original: We demonstrate that the simplest one,
Revised: We demonstrate that the simplest one,

Original: [Equation]
Revised: [Equation]

Original: based on binning the x i and y domains, is effective and easy to parallelize.
Revised: based on binning the x i and y domains, is effective and easy to parallelize.

Original: In the equation, p x ( i ) = n x ( i ) /N , p y ( j ) =
Revised: In the equation, p x ( i ) = n x ( i ) /N , p y ( j ) =

Original: n y ( j ) /N and p ( i, j )
Revised: n y ( j ) /N and p ( i, j )

Original: =
Revised: =

Original: n ( i, j ) /N are the relative frequencies that approximate
Revised: n ( i, j ) /N are the relative frequencies that approximate

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: p ( x ) , p ( y ) and p ( x, y ) for each bin.
Revised: p ( x ) , p ( y ) and p ( x, y ) for each bin.

Original: Thus, n x ( i ) , n y ( j ) and n ( i, j ) are the number of each interval.
Revised: Thus, n x ( i ) , n y ( j ) and n ( i, j ) are the inside each interval.

Original: The number of bins defines the width of uniformly distributed intervals over x d and y supports.
Revised: The number of bins defines the width of uniformly distributed intervals over x d and y supports.

Original: Since this estimator is sampling-based, under certain conditions (namely, if all densities exist as proper functions), Equation (13) indeed converges to I ( y ; x i | x O ) if we first let the number of samples N → ∞ [26].
Revised: Since this estimator is sampling-based, under certain conditions (namely, if all densities exist as proper functions), Equation (13) indeed converges to I ( y ; x i | x O ) if we first let the number of samples N → ∞ [26].

Original:  Experiments
Revised:  Experiments

Original: The evaluation of the HH-VAEM model is organized into three quantitative experiments and one qualitative experiment.
Revised: The evaluation of the HH-VAEM model is organized into three quantitative experiments and one qualitative experiment.

Original: The ablation study includes the validation of our proposed HMC-based, hierarchical model with respect to the Gaussian and one-layered alternatives.
Revised: The ablation study includes the validation of our proposed HMC-based, hierarchical model with respect to the Gaussian and one-layered alternatives.

Original: Namely, the comparison is performed with the following baseline models:
Revised: Namely, the comparison is performed with the following baseline models:

Original: • VAEM : 1 layer, Gaussian-based VAEM [33] (without including the Partial VAE).
Revised: • VAEM : 1 layer, Gaussian-based VAEM [33] (without including the Partial VAE).

Original: • MIWAEM : 1 layer, Gaussian-based, importance weighted IWAEM (VAEM + IWAE [36]).
Revised: • MIWAEM : 1 layer, Gaussian-based, importance weighted IWAEM (VAEM + IWAE [36]).

Original: • H-VAEM : 2 layers, Gaussian-based VAEM.
Revised: • H-VAEM : 2 layers, Gaussian-based VAEM.

Original: • HMC-VAEM : 1 layer, HMC-based (with our optimization method) VAEM.
Revised: • HMC-VAEM : 1 layer, HMC-based (with our optimization method) VAEM.

Original: [Figure][Figure] [Table]
Revised: [Figure] [Table]

Original: [Table][Figure] [Table]
Revised: [Table] [Table]

Original: For all the models, we manually introduce missing data in the training set by randomly setting per data point a feature as missing with a probability sampled uniformly in the interval [0 . 01 , 0 . 99] within each batch.
Revised: For all the models, we manually introduce missing data in the training set by randomly setting per data point a feature as missing with a probability sampled uniformly in the interval [0 . 01 , 0 . 99] within each batch.

Original: Both the input data x and the target y can be missing.
Revised: Both the input data x and the target y can be missing.

Original: For the test set, a fixed probability of 0.5 leads to about half of the input data being observed, whilst the target is completely unobserved.
Revised: For the test set, a fixed probability of 0.5 leads to about half of the input data being observed, whilst the target is completely unobserved.

Original: For the quantitative experiments, a total of 10 UCI datasets [11] that include mixed-type data are employed for the evaluation.
Revised: For the quantitative experiments, a total of 10 UCI datasets [11] that include mixed-type data are employed for the evaluation.

Original: We include both MNIST [27] and Fashion-MNIST [55] datasets for evaluating our model in higher dimensional observational and latent spaces and bigger architectures (3 layered convolutional nets for encoder/decoder).
Revised: We include both MNIST [27] and Fashion-MNIST [54] datasets for evaluating our model in higher dimensional observational and latent spaces and bigger architectures (3 layered convolutional nets for encoder/decoder).

Original: For the qualitative results, we evaluate our model in the image inpainting task on MNIST and CelebA [32].
Revised: For the qualitative results, we evaluate our model in the image inpainting task on MNIST and CelebA [32].

Original: For the three image datasets, the marginal
Revised: For the three image datasets, the marginal

Original: [Figure]
Revised: [Figure]

Original: We name this model HH-VAE, and similarly, the baselines are renamed asVAE MIWAE HMC andH-VAE.
Revised: We name this model HH-VAE, and similarly, the baselines are renamed as and

Original: Extended experiments and validations are provided in the Appendix.
Revised: Extended experiments and validations are provided in the Supplementary.

Original: The sourcecode for reproducing our work is available at https://github com/ipeis/HH
Revised: The source for reproducing our work is available at https

Original: 5.1 Mixed type conditional data imputation
Revised: 5.1 Mixed type conditional data imputation

Original: In order to evaluate the performance of the model in terms of data imputation, we opt by computing the negative log likelihood of the unobserved features.
Revised: In order to evaluate the performance of the model in terms of data imputation, we opt by computing the negative log likelihood of the unobserved features.

Original: We make use of the Monte Carlo approximation
Revised: We make use of the Monte Carlo approximation

Original: [Equation]
Revised: [Equation]

Original:  which is averaged over features in order to compare the imputation performance with the baselines.
Revised: i p ( [Equation] (13) which is averaged over features in order to compare the imputation performance with the baselines.

Original: Additionally, we include in Section A.7 similar results averaging each of the considered likelihoods.
Revised: Additionally, we include in Section C.3 similar results averaging each of the considered likelihoods.

Original: Results on the 10 UCI datasets and the MNIST datasets are included in Tables 1 and 3, showing that for most of the datasets, incremental improvement is obtained: VAEM < H-VAEM <
Revised: Results on the 10 UCI datasets and the MNIST datasets are included in Tables 1 and 3, showing that for most of the datasets, incremental improvement is obtained: VAEM < H-VAEM <

Original: HMC-VAEM <
Revised: HMC-VAEM <

Original: HH-VAEM.
Revised: HH-VAEM.

Original: Extended results with the imputation error, included in Section A.5, corroborate this.
Revised: Extended results with the imputation error, included in Section C.1, corroborate this.

Original: 5.2 Target prediction
Revised: 5.2 Target prediction

Original: For this experiment, we compute the negative log likelihood of the target under the predictive distribution using the same procedure as in the previous section.
Revised: For this experiment, we compute the negative log likelihood of the target under the predictive distribution using the same procedure as in the previous section.

Original: Results included in Tables 2 andshow the same incremental improvement in the prediction task.
Revised: Results included in Tables 2 andshow the same incremental improvement in the prediction task.

Original: 5.3 Sequential active information acquisition (SAIA)
Revised: 5.3 Sequential active information acquisition (SAIA)

Original: In this experiment, our HH-VAEM model and our acquisition method are evaluated in a SAIA task.
Revised: In this experiment, our HH-VAEM model and our acquisition method are evaluated in a SAIA task.

Original: Starting by predicting from completely unobserved inputs, at each step, the missing feature that maximizes the reward is acquired.
Revised: Starting by predicting from completely unobserved inputs, at each step, the missing feature that maximizes the reward is acquired.

Original: Figure 5 shows the error curves for the UCI datasets.
Revised: Figure 5 shows the error curves for the UCI datasets.

Original: Blue lines correspond to the Gaussian-based reward proposed by [33].
Revised: Blue lines correspond to the Gaussian-based reward proposed by [33].

Original: Orange lines are our sampling-based reward in a VAEM framework.
Revised: Orange lines are our sampling-based reward in a VAEM framework.

Original: Green lines correspond to HH-VAEM with our reward.
Revised: Green lines correspond to HH-VAEM with our reward.

Original: In most of the cases, our model and acquisition method obtains lower errors and faster discovery of information.
Revised: In most of the cases, our model and acquisition method obtains lower errors and faster discovery of information.

Original: 5.4 Conditional image inpainting
Revised: 5.4 Conditional image inpainting

Original: We include in this experiment qualitative results when comparing our model with the baselines on the image conditional inpainting task.
Revised: We include in this experiment qualitative results when comparing our model with the baselines on the image conditional inpainting task.

Original: We include results for MNIST and CelebA in Figure 6 (a)
Revised: We include results for MNIST and CelebA in Figure 6 (a)

Original: [Figure]
Revised: [Figure]

Original: and (b), respectively, that show the superiority of our method.
Revised: and (b), respectively, that show the superiority of our method.

Original: First, the hierarchical Gaussian model (fourth row) considerably improves the one-layered Gaussian alternative.
Revised: First, the hierarchical Gaussian model (fourth row) considerably improves the one-layered Gaussian alternative.

Original: Second, the HMC-based methods (two last rows) vastly improve the Gaussian methods.
Revised: Second, the HMC-based methods (two last rows) vastly improve the Gaussian methods.

Original: Third, in some specific cases, an extra improvement is added by HH-VAE with respect to the one-layered HMC-based model (columns 2, 4, 8, 9, 11, 12 and 13 in Figure 6 (a) or columns 1, 2, 5 and 12 in (b)).
Revised: Third, in some specific cases, an extra improvement is added by HH-VAE with respect to the one-layered HMC-based model (columns 2, 4, 8, 9, 11, 12 and 13 in Figure 6 (a) or columns 1, 2, 5 and 12 in (b)).

Original:  Conclusion
Revised:  Conclusion

Original: We presented HH-VAEM, to our knowledge, the first hierarchical VAE for mixed-type incomplete data that uses HMC with automatic hyper-parameter tuning for improved inference.
Revised: We presented HH-VAEM, to our knowledge, the first hierarchical VAE for mixed-type incomplete data that uses HMC with automatic hyper-parameter tuning for improved inference.

Original: We provide both quantitative and qualitative experiments that demonstrate its superiority with respect the baselines in the tasks of missing data imputation and supervised learning, placing HH-VAEM as a robust model for real-world datasets.
Revised: We provide both quantitative and qualitative experiments that demonstrate its superiority with respect the baselines in the tasks of missing data imputation and supervised learning, placing HH-VAEM as a robust model for real-world datasets.

Original: Further, we have developed a novel sampling-based technique for dynamic feature selection that outperforms the Gaussian-based alternatives and results in an efficient method for active learning in deep generative models.
Revised: Further, we have developed a novel sampling-based technique for dynamic feature selection that outperforms the Gaussian-based alternatives and results in an efficient method for active learning in deep generative models.

Original: Acknowledgements
Revised: Acknowledgments and Disclosure of Funding

Original: Ignacio Peis acknowledges support from Spanish government Ministerio de Ciencia, Innovación y Universidades under grants FPU18/00516, RTI2018-099655-B-I00, EST21/00467 and PID2021123182OB-I00 and from Comunidad de Madrid under grant Y2018/TCS-4705 PRACTICO-CM.
Revised: Ignacio Peis acknowledges support from Spanish government Ministerio de Ciencia, Innovación y Universidades under grants FPU18/00516, RTI2018-099655-B-I00, EST21/00467 and PID2021123182OB-I00 and from Comunidad de Madrid under grant Y2018/TCS-4705 PRACTICO-CM.

Original: José
Revised: José

Original: Miguel Hernández-Lobato acknowledges support from Boltzbit.Checklist For all authors...
Revised: Miguel Hernández-Lobato acknowledges support from Boltzbit.

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See Section 3.7 about the computational cost.

Original: (c) Did you discuss any potential negative societal impacts of your work?

Original: [No] We do not believe there could be any potential negative societal impacts.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] We provide the URL of the open source repository for reproducing all our experiments.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] See Appendix A.1 with all the training details.

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] We include error intervals in all our experiments.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Appendix A.1 with all the training details.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] We have cited every dataset/model employed or mentioned in our work.

Original: (b) Did you mention the license of the assets?

Original: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes] We provide the URL of the open source repository for reproducing all our experiments.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]

Original: A Appendix A.1 Experimental setup

Original: The networks for the encoder of the model with the MNIST datasets are 2 layered Deep CNNs with {16, 32, 32} output channels, kernel size 5, stride 2 and padding 2.

Original: For the experiments with CelebA, we use 5 layered Deep CNNs with {32, 32, 64, 64, 512} output channels, kernel size 4, strideand padding 1, followed by batch norm layers.

Original: They are followed by MLPs with 512 hidden units for obtaining the variational parameters for each layer.

Original: The decoder that obtains p θ ( x | h 1 ) is the symmetric CNN.

Original: All the NNs employed in the models trained with UCI datasets are one single layer

Original: MLPs with 256 hidden units.

Original: The noise variance for Gaussian likelihoods is set up to 0 . 1 .

Original: We employ learning rates of 1 × 10 − 3 for the models with MLP networks and 2 × 10 − 4 for the convolutional models.

Original: For the inflation parameter s , we increase to 1 × 10 − 2 for a faster convergence.

Original: A batch size of 100 is used for all the models except for Yatch and Wine dataset, where we use 20 samples per batch.

Original: The number of training steps is 2 × 10 4 for Boston, Energy, Wine, Yatch,

Original: Concrete, Diabetes and Yatch, and 5 × 10 4 for Naval, Avocado, Bank, and Insurance.

Original: For MNIST and Fashion-MNIST, we have 1 × 10 5 training steps.

Original: For CelebA, we use 1 , 5 × 10 5 training steps.

Original: For the marginal VAEs stage, we employ 1 × 10 3 training steps.

Original: The dimension of the latent variables is [ d 1 = 10 , d 2 = 5] for Boston, Energy, Wine, Naval Avocado, Bank and Insurance

Original: [ d 1 = 4 , d 2 = 2] for Concrete, Yatch and Diabetes, [ d 1 = 20 , d 2 = 10] for MNIST and FashionMNIST, and [ d 1 = 32 , d 2 = 16] for CelebA.

Original: We use LF = 5 Leapfrog steps in all cases, chains of T = 10 for Boston, Energy, Wine, Naval

Original: Avocado, Bank, Insurance, MNIST, Fashion-MNIST and CelebA, and T = 5 for Concrete, Yatch and Diabetes.

Original: The SKSD function is estimated using 30 HMC samples.

Original: For the MNIST and CelebA datasets, the use of Nvidia P100 GPU with Pascal architecture sped up the training with the CNN-based architecture.

Original: For the UCI datasets, due to the use of small networks, the differences when using CPU or GPU are almost imperceptible.

Original: A.2 Datasets information

Original: Apart from the MNIST, Fashion-MNIST and CelebA datasets, a total of 10 UCI datasets have been employed in this work including: Bank Marketing, Insurance Company Benchmark, Avocado sales,

Original: Naval Propulsion Plants, Yatch Hydrodynamics, Diabetes, Boston Housing, Wines, Energy efficiency and Bank Marketing.

Original: A.3 Efficacy of training HMC

Original: We include in this experiment results that demonstrate the efficacy of training the HMC hyperparameters using our proposed gradient-based strategy on 2D densities.

Original: In Figure 7, each row correspond to an example with a different density: first row is a wave density, second row is a dual moon density.

Original: In the first column, the initial set up is showed, including the density contour (dark blue), the initial

Original: Gaussian proposal contour (light blue) and samples from HMC (green).

Original: In both cases, due to the tightness of the proposal, chains do not properly explore the density and get stuck close the initial state.

Original: In the second column, results after training HMC hyperparameters and the inflation parameter are included.

Original: Again, in both cases, and more specially in the first one, the inflation of the horizontal variance of the proposal successfully increases to better cover the surface, and final HMC samples vastly improve the exploration.

Original: In the third column, the approximations of the HMC objective, the

Original: SKSD discrepancy and the inflation parameters over the optimization steps are included.

Original: A.4 HMC optimization

Original: We face the computational cost of running HMC by defining a small percentage of training steps for the last stage in Algorithm 1.

Original: A 10% of the total training steps for T HMC is sufficient for obtaining the convergence.

Original: In Figure 8 we include the validation metrics obtained during the optimization of

Original: Yatch dataset, where T HMC = 2 × 10 3 .

Original: In figure 9 (a), the mean acceptance rate of the HMC sampler over the training steps and the step sizes (b) are included.

Original: The steps are initialized from U (0 . 05 , 0 . 2) .

Original: After 2 × 10 3 steps, the mean acceptance rate converges to a value closer to ¯ p a = 0 .

Original: 65 , which is defined as the optimal desired acceptance probability [39].

Original: This empirical result provides evidence that, apart from reducing the computational cost, reducing the HMC training step to this value is sufficient for achieving convergence.

Original: A.5 Deterministic imputation metrics

Original: We include in Table 6 results on the RMSE obtained with other discriminative validated predictors, using mean imputation under the same missing rates.

Original: Additionally, we include here the missForest in the baselines, a wide-spread method for missing data imputation using a Random Forest approach [49].

Original: For classification tasks, the error rate is considered.

Original: In almost all cases, HH-VAEM outperforms the baselines.

Original: [Table]

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: A.6 Likelihood of the observed features

Original: We include in this section results on the negative log likelihood of the observed features

Original: [Equation]

Original: Results are included in Table 7.

Original: In almost all the cases, we confirm the incremental superiority when adding each part of our proposed design

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: A.7 Heterogeneous likelihoods

Original: In experiment 5.1 we reported an average likelihood across heterogeneous variables.

Original: Although this quantity is not a valid joint likelihood probability

Original: [Table]

Original: we employed this average to provide a fair comparison on models that have been trained on the same heterogeneous likelihoods.

Original: In this section

Original: [Table]

Original: we show the comparison on averaging separately the three considered marginal likelihoods (Gaussian,

Original: Bernoulli and Categorical) for two of the biggest datasets considered on Tables 8-10.

Original: Again, we show the incremental superiority when adding the different design choices of our model.

Original: [Table]

Original: [Table]

Original: [Table]

Original: A.8 Reparameterization trick for solving ill-posed HMC with hierarchical densities

Original: We provide in this section strong empirical demonstration on the efficacy of our proposed reparameterization trick.

Original: As stated in Section 3.4, naïve implementations of HMC are ill-posed when combined with hierarchical densities.

Original: The autoregressive correlations lead to non-smooth densities with huge peaks.

Original: The large gradients ∇ h 1: L log p ∗ ( h 1: L ) evaluated on these regions inside the Leapfrog steps of Equations 1 make huge modifications of the proposed states.

Original: If we denote these diverged states by h ( ill ) 1: L , evaluating the objective log p ∗ ( h ( ill ) 1: L ) lead to overflow issues.

Original: This undesired behavior is what we call divergence of the Leapfrog integrator , and is also illustrated by [4] in Figure 35.

Original: In order to avoid the aforementioned overflow issues, we can directly reject these problematic states.

Original: Nevertheless, by doing this, HMC will not properly explore the density by the time the states fall into the problematic regions, leading to extremely low acceptance rates.

Original: To demonstrate this, we include in Figure 10 (a) with blue line the evolution of the acceptance rates when optimizing HMC with the

Original: HH-VAEM variant without reparameterization (as illustrated in Figure 4 (a)).

Original: The acceptance rate is extremely low as expected, which is a clear indicator of a poorly mixing sampler.

Original: On the contrary, by using our proposed reparameterization trick (Figure 4 (b)), we are able to make

Original: HMC work properly and tune the step sizes, getting closer to the ideal acceptance rate ¯ p a = 0 .[39].

Original: Evaluations are applied to the same validation split of the Boston dataset.

Original: The step sizes of

Original: HMC are initialized from U (0 . 05 , 0 . 2) in both cases.

Original: Further, we also include in Figure 10 (b) the evolution of the imputation log likelihood metric of

Original: Equation (13) during the whole optimization for both approaches.

Original: First 1 , 8 × 10 4 steps correspond to the pretraining stage using only the ELBO.

Original: When introducing HMC without the reparameterization, due to the low acceptance rate, the states hardly move from the initial proposal, or move away from the density, and the joint optimization fail.

Original: We demonstrate again the effectiveness of the reparameterization trick by observing an increase in this metric.

Original: [Figure]

Original: leading to a considerable increase of the metric.

Original: A.9 Balancing the KLs

Original: Following [53], we define a short initial warming stage (10% of the total training steps) during the optimization where the KLs for the different layers are balanced according to their magnitude and the corresponding latent dimension, preventing the model for falling into posterior collapse by ignoring deepest layers.

Original: A factor is applied to each KL, following

Original: [Equation]

Original: The factors penalises the fact that a layer might be ignored by making the KL smaller when its magnitude is small compared to the rest layers.

Original: A.10 SAIA log-likelihoods

Original: In order to extend the results provided in Section 5.3, we include here the log-likelihoods curves when dynamically selecting features using the same procedure (Figure 11).

Original: [Figure]

Original: l o

Original: [Figure]

Original: li

Original: [Figure]

Original: li

Original: [Figure]

Original: d

Original: [Figure]

Original: A.11 Training times

Original: Table 11 shows the average training time in minutes for each model in the experiments for Tablesand 2.

Original: The ratio between training times for our method and the Gaussian baselines is approximately between 5 and

Original: [Figure]

Original: Energy

Original: [Figure]

Original: [Table]

Original: A.12 SAIA times

Original: The times for obtaining the SAIA metric curves in Section 5.3 are included in Figure 12.

Original: Although the performance is improved with HH-VAEM, it requires considerably longer time than the baselines to evaluate the reward, due to the HMC algorithm for sampling from the better approximated posterior.

Original: Future work might be oriented in proposing ways to measure and reduce this gap

Original: [Table]

Original: [Figure]