Original: Missing Data Imputation and Acquisition with Deep Hierarchical Models and Hamiltonian Monte Carlo
Revised: Missing Data Imputation and Acquisition with Deep Hierarchical Models and Hamiltonian Monte Carlo

Original: Abstract
Revised: Abstract

Original: Variational Autoencoders (VAEs) have recently been highly successful at imputingand acquiring heterogeneous missing data.
Revised: Variational Autoencoders (VAEs) have recently been highly successful at imputing and acquiring heterogeneous missing data.

Original: However, within this specific application domain, existing VAE methods are restricted by using only one layer oflatent variables and strictly Gaussian posterior approximations.
Revised: However, within this specific application domain, existing VAE methods are restricted by using only one layer of latent variables and strictly Gaussian posterior approximations.

Original: To address theselimitations, we present HH-VAEM, a Hierarchical VAE model for mixed-type incomplete data that uses Hamiltonian Monte Carlo with automatic hyper-parametertuning for improved approximate inference.
Revised: To address these limitations, we present HH-VAEM, a Hierarchical VAE model for mixed-type incomplete data that uses Hamiltonian Monte Carlo with automatic hyper-parameter tuning for improved approximate inference.

Original: Our experiments show that HH-VAEMoutperforms existing baselines in the tasks of missing data imputation and supervised learning with missing features.
Revised: Our experiments show that HH-VAEM outperforms existing baselines in the tasks of missing data imputation and supervised learning with missing features.

Original: Finally, we also present a sampling-basedapproach for efficiently computing the information gain when missing features areto be acquired with HH-VAEM.
Revised: Finally, we also present a sampling-based approach for efficiently computing the information gain when missing features are to be acquired with HH-VAEM.

Original: Our experiments show that this sampling-basedapproach is superior to alternatives based on Gaussian approximations.
Revised: Our experiments show that this sampling-based approach is superior to alternatives based on Gaussian approximations.

Original:  Introduction
Revised:  Introduction

Original: Many real-world unsupervised learning tasks require dealing with complicated datasets with mixedtypes (real, positive-valued, continuous, or discrete) and missing values.
Revised: Many real-world unsupervised learning tasks require dealing with complicated datasets with mixed types (real, positive-valued, continuous, or discrete) and missing values.

Original: For this purpose, variationalautoencoders [34, 35, 58] stand out in the recent literature as robust generative models that efficientlyhandle high-dimensional data.
Revised: For this purpose, variational autoencoders [24, 25, 41] stand out in the recent literature as robust generative models that efficiently handle high-dimensional data.

Original: However, in their naive configuration, every data dimension is assumedto have similar statistical properties (i e., homogeneity), and all dimensions are considered to becompletely observed.
Revised: However, in their naive configuration, every data dimension is assumed to have similar statistical properties (i e., homogeneity), and all dimensions are considered to be completely observed.

Original: Both assumptions won’t hold in many real-world scenarios.
Revised: Both assumptions won’t hold in many real-world scenarios.

Original: Recent workshave adapted VAEs to handle incomplete [12, 19, 45, 47] and mixed-type data [24, 44, 50] anddemonstrated improved performance in downstream tasks such as missing data imputation and activeinformation acquisition.
Revised: Recent works have adapted VAEs to handle incomplete [9, 14, 34, 36] and mixed-type data [18, 33, 38] and demonstrated improved performance in downstream tasks such as missing data imputation and active information acquisition.

Original: Despite these advances, existing approaches are far from optimal as they arebased on restrictive design choices: 1), only one layer of latent variables are considered; 2), Gaussianposterior approximations are usually adopted.
Revised: Despite these advances, existing approaches are far from optimal as they are based on restrictive design choices: 1), only one layer of latent variables are considered; 2), Gaussian posterior approximations are usually adopted.

Original: These will lead to limited flexibility and additionalbias, especially under real-world settings with complex mixed-type incomplete data.
Revised: These will lead to limited flexibility and additional bias, especially under real-world settings with complex mixed-type incomplete data.

Original: In the literature, the issue of model flexibility and inference bias are often addressed separately.
Revised: In the literature, the issue of model flexibility and inference bias are often addressed separately.

Original: For example, approximate inference bias can be reduced by using Monte Carlo sampling [63, 69].
Revised: For example, approximate inference bias can be reduced by using Monte Carlo sampling [46, 51].

Original: More specifically, Hamiltonian Monte Carlo (HMC) [6, 17] stands out among MCMC methods inmachine learning due to its superior efficiency for exploring the target density.
Revised: More specifically, Hamiltonian Monte Carlo (HMC) [5, 12] stands out among MCMC methods in machine learning due to its superior efficiency for exploring the target density.

Original: In the context of VAE, HMC has also been combined with stochastic variational inference [10] for improving the trainingof VAEs.
Revised: In the context of VAE, HMC has also been combined with stochastic variational inference [7] for improving the training of VAEs.

Original: On the other hand, the flexibility of VAEs can be improved by considering hierarchical VAEs with multiple layers of hidden variables [11, 46, 66, 72].
Revised: On the other hand, the flexibility of VAEs can be improved by considering hierarchical VAEs with multiple layers of hidden variables [8, 35, 48, 53].

Original: By using a hierarchical structure inthe latent space, they enforce the information to flow from high-level representations to more specificobservable factors, imitating the way information is often organized in the real world.
Revised: By using a hierarchical structure in the latent space, they enforce the information to flow from high-level representations to more specific observable factors, imitating the way information is often organized in the real world.

Original: However, the issue of modeling flexibility and approximate inference bias are often heavily intertwined, and addressing them simultaneously in a joint manner is highly non-trivial
Revised: However, the issue of modeling flexibility and approximate inference bias are often heavily intertwined, and addressing them simultaneously in a joint manner is highly non-trivial.

Original: The hierarchical organization of the latent variables creates complicated posterior dependencies that are not straightforward to deal with and require special consideration.
Revised: The hierarchical organization of the latent variables creates complicated posterior dependencies that are not straightforward to deal with and require special consideration.

Original: To improve Gaussian approximate inference,most works opt by defining shared paths between the recognition and generative networks.
Revised: To improve Gaussian approximate inference, most works opt by defining shared paths between the recognition and generative networks.

Original: While thismakes hierarchical VAEs practical, the bias introduced by the Gaussian approximations is still present.
Revised: While this makes hierarchical VAEs practical, the bias introduced by the Gaussian approximations is still present.

Original: To the best of our knowledge, none of the aforementioned hierarchical VAEs has been previouslycombined with Monte Carlo algorithms for improving over standard Gaussian approximate inference.
Revised: To the best of our knowledge, none of the aforementioned hierarchical VAEs has been previously combined with Monte Carlo algorithms for improving over standard Gaussian approximate inference.

Original: To overcome such limitations, we focus on training new hierarchical VAE models for heterogeneousmixed-type data with HMC.
Revised: To overcome such limitations, we focus on training new hierarchical VAE models for heterogeneous mixed-type data with HMC.

Original: Our models can be used for missing data imputation and for supervisedlearning with missing data.
Revised: Our models can be used for missing data imputation and for supervised learning with missing data.

Original: We also present a sampling-based framework that allows our models toperform accurate sequential active information acquisition.
Revised: We also present a sampling-based framework that allows our models to perform accurate sequential active information acquisition.

Original: Our main contributions are as follows:
Revised: Our main contributions are as follows:

Original: • We present HH-VAEM, a deep hierarchical model for handling mixed-type incompletedata that uses HMC with automatic hyper-parameter tuning for outperforming amortizedvariational inference by generating low bias samples from the true posterior.• We propose a sampling-based strategy for missing feature acquisition that benefits from theimproved inference of HH-VAEM.
Revised: • We present HH-VAEM, a deep hierarchical model for handling mixed-type incomplete data that uses HMC with automatic hyper-parameter tuning for outperforming amortized variational inference by generating low bias samples from the true posterior. • We propose a sampling-based strategy for missing feature acquisition that benefits from the improved inference of HH-VAEM.

Original: By using histograms to estimate the mutual information,this strategy achieves lower bias than other Gaussian-based alternatives.
Revised: By using histograms to estimate the mutual information, this strategy achieves lower bias than other Gaussian-based alternatives.

Original: • We exhaustively evaluate HH-VAEM in the tasks of 1) missing data imputation, 2) supervisedlearning with missing data and 3) information acquisition with our sampling-based strategy.
Revised: • We exhaustively evaluate HH-VAEM in the tasks of 1) missing data imputation, 2) supervised learning with missing data and 3) information acquisition with our sampling-based strategy.

Original: In all cases we report significant gains with respect to baselines.
Revised: In all cases we report significant gains with respect to baselines.

Original:  Related work
Revised:  Related work

Original: 2.1 VAEs for mixed-type incomplete data
Revised: VAEs for mixed-type incomplete data

Original: Variational Autoencoders [34, 58] are deep generative models that make use of encoder and decodernetworks for mapping data into a latent Gaussian distribution, and reconstructing the latent codes intothe original observational space, respectively.
Revised: Variational Autoencoders [24, 41] are deep generative models that make use of encoder and decoder networks for mapping data into a latent Gaussian distribution, and reconstructing the latent codes into the original observational space, respectively.

Original: The parameters of these networks are trained usingamortized Variational Inference [13, 76] optimizing a lower bound (ELBO) on the log evidence:
Revised: The parameters of these networks are trained using amortized Variational Inference [10, 56] optimizing a lower bound (ELBO) on the log evidence:

Original: E q z ( z | x ) log p θ ( x , z ) q ψ ( z | x ) , where the generative model p θ ( x , z ) can be expressed in terms of the likelihoodp θ ( x | z ) and the prior p ( z ) .
Revised: E q z ( z | x ) log p θ ( x , z ) q ψ ( z | x ) , where the generative model p θ ( x , z ) can be expressed in terms of the likelihood p θ ( x | z ) and the prior p ( z ) .

Original: In mixed-type data, the vector x is composed by data from differenttypes: real, positive real, categorical, binary, etc.
Revised: In mixed-type data, the vector x is composed by data from different types: real, positive real, categorical, binary, etc.

Original: A naive approach is to consider a factorized decoderusing different likelihood contributions p θ ( x | z ) = (cid:81)
Revised: A naive approach is to consider a factorized decoder using different likelihood contributions p θ ( x | z ) = (cid:81)

Original: d p θ ( x d | z ) [2, 50].
Revised: d p θ ( x d | z ) [1, 38].

Original: Nonetheless, the problem ofhandling unbalanced likelihoods leads to the domination of some dimensions during the optimizationprocess.
Revised: Nonetheless, the problem of handling unbalanced likelihoods leads to the domination of some dimensions during the optimization process.

Original: In [44], authors propose a solution using a set of marginal VAEs that encode each featureinto a Gaussian uni-dimensional space, and a dependency VAE that captures the inter-dimensionaldependencies more effectively using balanced Gaussian likelihoods.
Revised: In [33], authors propose a solution using a set of marginal VAEs that encode each feature into a Gaussian uni-dimensional space, and a dependency VAE that captures the inter-dimensional dependencies more effectively using balanced Gaussian likelihoods.

Original: By marginalizing each dimension of the decoder, incomplete data can be easily handled by dividingthe vector x into the observed x O and unobserved x U parts.
Revised: By marginalizing each dimension of the decoder, incomplete data can be easily handled by dividing the vector x into the observed x O and unobserved x U parts.

Original: This methodology is completely validwhen using the missing-at-random (MAR) assumption [41], i.e. assuming the missing mechanism isindependent of the missing values.
Revised: This methodology is completely valid when using the missing-at-random (MAR) assumption [30], i.e. assuming the missing mechanism is independent of the missing values.

Original: In this our work, we use the same assumption.
Revised: In this our work, we use the same assumption.

Original: As proposed in[50] and [47], the ELBO objective is transformed into a lower bound on the observed data, and theunobserved data is replaced with zeros.
Revised: As proposed in [38] and [36], the ELBO objective is transformed into a lower bound on the observed data, and the unobserved data is replaced with zeros.

Original: 2.2 Hierarchical VAEs
Revised: 2.2 Hierarchical VAEs

Original: Hierarchical models have been successfully employed in deep generative modeling, [3, 61, 62].
Revised: Hierarchical models have been successfully employed in deep generative modeling, [2, 44, 45].

Original: In
Revised: In

Original: VAEs, defining a hierarchical latent space for VAEs can be straightforward.
Revised: VAEs, defining a hierarchical latent space for VAEs can be straightforward.

Original: Nevertheless, potentialpitfalls require special attention.
Revised: Nevertheless, potential pitfalls require special attention.

Original: Concretely, if the decoder is powerful enough, the model tendsto uniquely use the shallowest layers, ignoring the deepest ones and falling into the well-knownproblem of posterior collapse [46, 55, 73].
Revised: Concretely, if the decoder is powerful enough, the model tends to uniquely use the shallowest layers, ignoring the deepest ones and falling into the well-known problem of posterior collapse [35, 40, 54].

Original: In the last few years, several works have investigatedpossible hierarchical structures for VAEs.
Revised: In the last few years, several works have investigated possible hierarchical structures for VAEs.

Original: In [66], a bottom-up deterministic path is used along with atop-down inference path that shares parameters with the top-down structure of the generative model.
Revised: In [48], a bottom-up deterministic path is used along with a top-down inference path that shares parameters with the top-down structure of the generative model.

Original: In [46], the authors use a bidirectional stochastic inference path.
Revised: In [35], the authors use a bidirectional stochastic inference path.

Original: More recently, [72] or [11] haveadapted these architectures to complex datasets and high quality images.
Revised: More recently, [53] or [8] have adapted these architectures to complex datasets and high quality images.

Original: Possibly motivated by the residual connections in [32], all these works use a shared path between recognition and generativemodels that helps in tying the divergences between approximations and priors in the ELBO.
Revised: Possibly motivated by the residual connections in [23], all these works use a shared path between recognition and generative models that helps in tying the divergences between approximations and priors in the ELBO.

Original: 2.3 Hamiltonian Monte Carlo
Revised: 2.3 Hamiltonian Monte Carlo

Original: HMC [5, 17, 52] is a particularly effective MCMC algorithm for sampling from a target distributionp ( z ) =
Revised: HMC [4, 12, 39] is a particularly effective MCMC algorithm for sampling from a target distribution p ( z ) =

Original: 1 Z p ∗ ( z ) where Z is the normalization constant and z is a d -dimensional vector.
Revised: 1 Z p ∗ ( z ) where Z is the normalization constant and z is a d -dimensional vector.

Original: Byaugmenting this model to p ( r , z ) =
Revised: By augmenting this model to p ( r , z ) =

Original: N ( r ; 0 , M ) p ( z ) , and denoting r as the momentum variablewith diagonal covariance matrix M , with the same dimensionality as z , HMC samples  from thedistribution by simulating the time-evolution of a fictitious physical system.
Revised: N ( r ; 0 , M ) p ( z ) , and denoting r as the momentum variable with diagonal covariance matrix M , with the same dimensionality as z , HMC samples are obtained from the distribution by simulating the time-evolution of a fictitious physical system.

Original: The algorithm starts by firstly sampling z and r from an initial proposal and the momentum distribution, respectively.
Revised: The algorithm starts by firstly sampling z and r from an initial proposal and the momentum distribution, respectively.

Original: Chains with length T are built by recurrently proposing and accepting newstates.
Revised: Chains with length T are built by recurrently proposing and accepting new states.

Original: To propose a new state, the Hamiltonian dynamics are simulated using a symplectic integrator,
Revised: To propose a new state, the Hamiltonian dynamics are simulated using a symplectic integrator,

Original: Leapfrog being the most common choice.
Revised: Leapfrog being the most common choice.

Original: The following updates are repeated for l = 1 : LF steps:
Revised: The following updates are repeated for l = 1 : LF steps:

Original: [Equation]
Revised: [Equation]

Original: (1) where ⊙ refers to the Hadamard product, and ϕ is the step size hyperparameter.
Revised: (1) where ⊙ refers to the Hadamard product, and ϕ is the step size hyperparameter.

Original: Although it istypically defined as a scalar for simplicity, a d -dimensional vector can be considered to applydifferent step sizes per dimension, or further, as considered in this work, a T × d matrix to applydifferent steps per each proposal of the chain.
Revised: Although it is typically defined as a scalar for simplicity, a d -dimensional vector can be considered to apply different step sizes per dimension, or further, as considered in this work, a T × d matrix to apply different steps per each proposal of the chain.

Original: The new proposal ( z ′ , r ′ ) is accepted with probabilitymin [1 , exp( − H ( z ′ , r ′ )
Revised: The new proposal ( z ′ , r ′ ) is accepted with probability min [1 , exp( − H ( z ′ , r ′ )

Original: + H ( z , r ))] , where
Revised: + H ( z , r ))] , where

Original: [Equation]
Revised: [Equation]

Original: For the consecutive T proposals, a new momentum r is resampled and the updates of eq.
Revised: For the consecutive T proposals, a new momentum r is resampled and the updates of eq.

Original: (1) arerepeated for LF steps to update the state if ( z ′ , r ′ ) is accepted.
Revised: (1) are repeated for LF steps to update the state if ( z ′ , r ′ ) is accepted.

Original: 2.4 Active Feature Acquisition
Revised: 2.4 Active Feature Acquisition

Original: Among all the Active Learning techniques, Active Feature Acquisition [29, 48, 60, 68] is of specialinterest in cost-sensitive applications for modeling a trade-off between the improvement of predictionsand the cost of acquiring new data at the feature level.
Revised: Among all the Active Learning techniques, Active Feature Acquisition [20, 37, 43, 50] is of special interest in cost-sensitive applications for modeling a trade-off between the improvement of predictions and the cost of acquiring new data at the feature level.

Original: Several works in the recent literature havestudied methods for performing the task of sequentially acquiring high-value information by selectingfeatures that maximize an information theoretical reward function and enhance the accuracy of thepredictions.
Revised: Several works in the recent literature have studied methods for performing the task of sequentially acquiring high-value information by selecting features that maximize an information theoretical reward function and enhance the accuracy of the predictions.

Original: This task is denoted by SAIA (Sequential Active Information Acquisition).
Revised: This task is denoted by SAIA (Sequential Active Information Acquisition).

Original: In [45], itis proposed an efficient method  for approximating a non-tractable reward by using the encoder of a VAE that handles missing data.
Revised: In [34], an efficient method is proposed for approximating a non-tractable reward by using the encoder of a VAE that handles missing data.

Original: In [44] they extend this method for handling mixed-type data.
Revised: In [33] they extend this method for handling mixed-type data.

Original: Bothworks computing the reward by relying on Gaussian approximations given by the encoder networks.
Revised: Both works estimate the reward by relying on Gaussian approximations given by the encoder networks.

Original:  Hamiltonian hierarchical VAE for mixed-type incomplete data (HH-VAEM)
Revised:  Hamiltonian hierarchical VAE for mixed-type incomplete data (HH-VAEM)

Original: The HH-VAEM model (Figure 1 (a)) is a Hierarchical VAE for mixed-type, incomplete data thatincorporates HMC with automatic hyper-parameter optimization for sampling from the posteriorof the latent variables.
Revised: The HH-VAEM model (Figure 1 (a)) is a Hierarchical VAE for mixed-type, incomplete data that incorporates HMC with automatic hyper-parameter optimization for sampling from the posterior of the latent variables.

Original: In a first stage, the mixed-type data is encoded into marginal Gaussianposterior distributions as given by univariate VAEs fitted to each data dimension.
Revised: In a first stage, the mixed-type data is encoded into marginal Gaussian posterior distributions as given by univariate VAEs fitted to each data dimension.

Original: In a second stage, ahierarchical structure captures the dependencies among the standarized, homogeneous dimensionswith well-balanced Gaussian likelihoods.
Revised: In a second stage, a hierarchical structure captures the dependencies among the standarized, homogeneous dimensions with well-balanced Gaussian likelihoods.

Original: The model is trained using samples from the posterior of thehierarchical latent variables by means of HMC, whilst the HMC hyper-parameters are automaticallytuned.
Revised: The model is trained using samples from the posterior of the [Figure] hierarchical latent variables by means of HMC, whilst the HMC hyper-parameters are automatically tuned.

Original: A more detailed description is provided in the following subsections.
Revised: A more detailed description is provided in the following subsections.

Original: 3.1 Notation
Revised: 3.1 Notation

Original: The model generates both data x ∈ R D and output y ∈ R P , where each of these variables is dividedinto observed parts x O , y O and unobserved parts x U , y U .
Revised: The model generates both data x ∈ R D and output y ∈ R P , where each of these variables is divided into observed parts x O , y O and unobserved parts x U , y U .

Original: Each dimension of x is denoted by x d .
Revised: Each dimension of x is denoted by x d .

Original: A[Figure] training set is composed of N observations as tuples ( x ( n ) O , y ( n ) O ) .
Revised: A training set is composed of N observations as tuples ( x ( n ) O , y ( n ) O ) .

Original: For ease of notation,  omit theobservation index n , and the objectives are presented for a single observation.
Revised: For ease of notation, we omit the observation index n , and the objectives are presented for a single observation.

Original:  dependency  variables  ϵ  ϵ L 
Revised: The dependency latent space is composed of L latent variables [ ϵ 1 , ..., ϵ L ] , with ϵ l ∈ R m l . The dimension of the joint latent distribution is (cid:80) l m l = M . In the marginal VAEs, the latent variables z d are unidimensional.

Original: 3.2 Handling heterogeneous incomplete data
Revised: 3.2 Handling heterogeneous incomplete data

Original: Following the strategy proposed by [44], we perform a two-staged approach for handling heterogeneous data.
Revised: Following the strategy proposed by [33], we perform a two-staged approach for handling heterogeneous data.

Original: The marginal distribution of each feature p θ d ( x d ) is modeled by a one-dimensional
Revised: The marginal distribution of each feature p θ d ( x d ) is modeled by a one-dimensional

Original: VAE.
Revised: VAE.

Original: First, the D marginal VAEs are trained independently by maximizing the marginal ELBO overobserved points
Revised: First, the D marginal VAEs are trained independently by maximizing the marginal ELBO over observed points

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: where 1 ( x d ∈ x O ) is an indicator function that activates the ELBO when the feature is observed.
Revised: where 1 ( x d ∈ x O ) is an indicator function that activates the ELBO when the feature is observed.

Original: Under the missing-at-random assumption, which  the one considered in this work, Equation (3) leadsto a lower bound on observed data likelihoods.
Revised: Under the missing-at-random assumption, which is the one considered in this work, Equation (3) leads to a lower bound on observed data likelihoods.

Original: Second, a dependency VAE encodes a vector z withthe concatenated samples from the marginal posteriors q γ d ( z d | x d ) into the global latent variable h ,using zero-filling for the unobserved variables.
Revised: Second, a dependency VAE encodes a vector z with the concatenated samples from the marginal posteriors q γ d ( z d | x d ) into the global latent variable h , using zero-filling for the unobserved variables.

Original: By using this approach, z is now homogeneous andcan then be easily modeled using a standard Gaussian decoder.
Revised: By using this approach, z is now homogeneous and can then be easily modeled using a standard Gaussian decoder.

Original: By contrast, other works [18, 50]directly operate with different decoding likelihoods per data type.
Revised: By contrast, other works [13, 38] directly operate with different decoding likelihoods per data type.

Original: These often have very differentmagnitudes in the ELBO and may reduce learning efficiency.
Revised: This approach often leads to having very different magnitudes in the ELBO and may reduce learning efficiency.

Original: The ELBO for the second stagedependency VAE is
Revised: The ELBO for the second stage dependency VAE is

Original: [Equation]
Revised: [Equation]

Original: where ϵ = { ϵ 1 , ..., ϵ L } is a set of reparameterized hierarchical latent variables.
Revised: where ϵ = { ϵ 1 , ..., ϵ L } is a set of reparameterized hierarchical latent variables.

Original: Further details on thedesign of the hierarchical dependency VAE are provided below.
Revised: Further details on the design of the hierarchical dependency VAE are provided below.

Original: 3.3 Predictive enhancement
Revised: 3.3 Predictive enhancement

Original: The combination of generative and discriminative models is an effective well-studied strategy fordealing with predictive models under missing data [21, 71].
Revised: The combination of generative and discriminative models is an effective well-studied strategy for dealing with predictive models under missing data [15, 52].

Original: In [21], they model p ( x ) for imputingmissing data using a Gaussian Mixture Model.
Revised: In [15], they model p ( x ) for imputing missing data using a Gaussian Mixture Model.

Original: In a deep learning context, recent supervised VAEmodels have revisited this combination [30, 64] or used factorisations of type p ( z ) p ( x | z ) p ( y | z )[31] to learn meaningful representations.
Revised: In a deep learning context, recent supervised VAE models have revisited this combination [21, 47] or used factorisations of type p ( z ) p ( x | z ) p ( y | z ) [22] to learn meaningful representations.

Original: In [39] the authors propose a deep generative model withfactorisation p ( z ) p ( x | z ) p ( y | x , z ) for detecting adversarial attacks.
Revised: In [28] the authors propose a deep generative model with factorisation p ( z ) p ( x | z ) p ( y | x , z ) for detecting adversarial attacks.

Original: With the aim at reinforcing the prediction of the variable of interest, we turn into a supervisedmodel by including a separate predictor for p θ y ( y | ˆ x , h ) , apart from the decoder p θ
Revised: With the aim at reinforcing the prediction of the variable of interest, we turn into a supervised model by including a separate predictor for p θ y ( y | ˆ x , h ) , apart from the decoder p θ

Original: z
Revised: z

Original: ( z | h ) .
Revised: ( z | h ) .

Original: Thevector ˆ x = ( x i ∈
Revised: The vector ˆ x = ( x i ∈

Original: x O , ˆ x j ∈ x U ) includes the observed part and imputation of the missing part ˆ x j bydecoding the latent h into z using p ( z | h 1 ) , and each dimension z j ∈ z into ˆ x j using p ( x j | z j ) .
Revised: x O , ˆ x j ∈ x U ) includes the observed part and imputation of the missing variables ˆ x j by decoding the latent h into z using p ( z | h 1 ) , and each dimension z j ∈ z into ˆ x j using p ( x j | z j ) .

Original: Thepredictor parameters θ y are optimized along with the decoder parameters θ z .
Revised: The predictor parameters θ y are optimized along with the decoder parameters θ z .

Original: 3.4 Hierarchical reparameterized latent space
Revised: 3.4 Hierarchical reparameterized latent space

Original: A hierarchical structure over the latent space
Revised: A hierarchical structure over the latent space

Original: h
Revised: h

Original: = { h 1 , ..., h L } enriches the prior assumptions andpermits a flexible generation of data in a more natural fashion.
Revised: = { h 1 , ..., h L } enriches the prior assumptions and permits a flexible generation of data in a more natural fashion.

Original: Nevertheless, as stated in [5, 6], HMCcan be pathological when used for sampling from hierarchical densities, where the magnitude ofautoregressive variations increase with the depth.
Revised: Nevertheless, as stated in [4, 5], HMC can be pathological when used for sampling from hierarchical densities, where the magnitude of autoregressive variations increase with the depth.

Original: For approximating the Hamiltonian dynamics,inside each Leapfrog integrator step, gradients ∇ h 1: L log p ∗ ( h 1: L ) are required.
Revised: For approximating the Hamiltonian dynamics, inside each Leapfrog integrator step, gradients ∇ h 1: L log p ∗ ( h 1: L ) are required.

Original: Due to the strong165 curvature regions, huge norm of high-order derivatives are backpropagated and might eventuallyexplode, ending in overflow issues (Figure 35 in [6]).
Revised: Due to the strong curvature regions, huge norm of high-order derivatives are backpropagated and might eventually explode, ending in overflow issues (Figure 35 in [5]).

Original: If we were to run our HMC method over the hierarchical variables without any reparameterization(Figure 4a), by the time the states reached the aforementioned problematic regions, the integratorwould diverge and we would experience the aforementioned overflow problems.
Revised: If we were to run our HMC method over the hierarchical variables without any reparameterization (Figure 4a), by the time the states reached the aforementioned problematic regions, the integrator would diverge and we would experience the aforementioned overflow problems.

Original: By rejecting theseproblematic states, chains would get stuck close to the proposal and the hierarchical density wouldnot be properly explored (concluding that HMC would not improve the Gaussian proposal).
Revised: By rejecting these problematic states, chains would get stuck close to the proposal and the hierarchical density would not be properly explored (concluding that HMC would not improve the Gaussian proposal).

Original: To givean example, in [72], the AR path p ( h l | h <l ) would provoke instabilities inside the HMC integratordue to huge gradients in ∇ h 1: L log p ∗ ( h 1: L ) .
Revised: To give an example, in [53], the AR path p ( h l | h <l ) would provoke instabilities inside the HMC integrator due to huge gradients in ∇ h 1: L log p ∗ ( h 1: L ) .

Original: This is considered as a pathological design and thus isnot included in the validation experiments.

Original: We successfully solved this issue by introducing a hierarchical reparameterization technique.
Revised: We successfully solved this issue by introducing a hierarchical reparameterization technique.

Original: Therepresentation at each layer is reparameterized from variable ϵ l with standard Gaussian prior p ( ϵ l)
Revised: The representation at each layer is reparameterized from variable ϵ l with standard Gaussian prior p ( ϵ l

Original: [Equation]
Revised: [Equation]

Original: where the functions f µ l ( h l +1 ) and f σ l ( h l +1 ) are applied by NNs with parameters θ l = { θ µ l , θ σ l } .
Revised: where the functions f µ l ( h l +1 ) and f σ l ( h l +1 ) are applied by NNs with parameters θ l = { θ µ l , θ σ l } .

Original: The result is equivalent as learning the mean and covariance of autoregressive variables (see Figure 4for illustrative details).
Revised: The result is equivalent as learning the mean and covariance of autoregressive variables (see Figurefor illustrative details).

Original: However, thanks to this trick, we relax the dependencies among the latentvariables, resulting in a smoother joint posterior density p ( ϵ | x
Revised: However, thanks to this trick, we relax the dependencies among the latent variables, resulting in a smoother joint posterior density p ( ϵ | x

Original: O , y O ) .
Revised: O , y O ) .

Original: Performing the inference overϵ = { ϵ 1 , ..., ϵ L } leads to a better posed basis for running our HMC optimization, detailed in section3.5, and avoids the necessity of employing more advanced HMC samplers like [6, 22].
Revised: Performing the inference over ϵ = { ϵ 1 , ..., ϵ L } leads to a better posed basis for running our HMC optimization, detailed in section 3.5, and avoids the necessity of employing more advanced HMC samplers like [5, 16].

Revised: We include further details on the pathological behavior and demonstration of the effectiveness of our solution in Section A.8.

Original: Providedthe promising results we obtain in Section 5, we propose our reparameterization trick as a novelcontribution for solving the pathological behavior of HMC combined with hierarchical VAEs.
Revised: Provided the promising results we obtain in Section 5, we propose our reparameterization trick as a novel contribution for solving the pathological behavior of HMC combined with hierarchical VAEs.

Original: For the sake of simplicity, we name the generative parameters as θ = { θ z , θ y , θ 1 , ..., θ L } .
Revised: For the sake of simplicity, we name the generative parameters as θ = { θ z , θ y , θ 1 , ..., θ L } .

Original: Thedependency ELBO under this hierarchical reparameterized model becomes
Revised: The dependency ELBO under this hierarchical reparameterized model becomes

Original: [Equation]
Revised: [Equation]

Original: where we name r l the hidden representation at each layer, and defining r 

Revised: We name r l the hidden representation at each layer, and defining r

Original: = { x O , y O } , we employ NNs with parameters ψ r l for computing r l = f r ( r l − 1 ) .
Revised: = { x O , y O } , we employ NNs with parameters ψ r l for computing r l = f r ( r l − 1 ) .

Original: These vectors are mapped into the parametersof the variational posterior q ψ l ( ϵ l | x O , y O ) , using NNs for computing the mean as g µ l ( r l ) and thecovariance as g σ l ( r l ) , with parameters ψ µ l and ψ σ l .
Revised: These vectors are mapped into the parameters of the variational posterior q ψ l ( ϵ l | x O , y O ) , using NNs for computing the mean as g µ l ( r l ) and the covariance as g σ l ( r l ) , with parameters ψ µ l and ψ σ l .

Original: With compactness purposes, we will denote theencoder parameters as ψ = { ψ 1 , ..., ψ L } , with ψ l = { ψ r l , ψ µ l , ψ σ l } .
Revised: With compactness purposes, we will denote the encoder parameters as ψ = { ψ 1 , ..., ψ L } , with ψ l = { ψ r l , ψ µ l , ψ σ l } .

Original: 3.5 HMC over the hierarchical density
Revised: 3.5 HMC over the hierarchical density

Original: In recent works, HMC has been combined with deep generative models for improving the inferenceof the latent variables by obtaining better samples from the posterior [10, 28].
Revised: In recent works, HMC has been combined with deep generative models for improving the inference of the latent variables by obtaining better samples from the posterior [7, 19].

Original: In this work, wepropose to transcend these previous approaches and build a generalized method for sampling fromcomplicated, hierarchical latent structures composed by several layers.
Revised: In this work, we propose to transcend these previous approaches and build a generalized method for sampling from complicated, hierarchical latent structures composed by several layers.

Original: Inspired by [9] and theirmethod for sampling from the posterior within a vanilla VAE framework while tuning the HMC hyperparameters, we follow a procedure for training the dependency model where i) during a pretraining stage, the encoder and decoder are optimized using standard VI and the ELBO from equation(6), and ii) using the pre-trained encoder for starting from a good proposal [28], HMC samplesare obtained to follow the true posterior and jointly optimize the generative model and the HMChyperparameters.
Revised: Inspired by [6] and their method for sampling from the posterior within a vanilla VAE framework while tuning the HMC hyperparameters, we follow a procedure for training the dependency model where i) during a pre-training stage, the encoder and decoder are optimized using standard VI and the ELBO from equation (6), and ii) using the pre-trained encoder for starting from a good proposal [19], HMC samples are obtained to follow the true posterior and jointly optimize the generative model and the HMC hyperparameters.

Original: In Figure 1 (b) we include an illustrative example.
Revised: In Figure 1 (b) we include an illustrative example.

Original: We denote by q ( T ) ϕ
Revised: We denote by q ( T ) ϕ

Original: ( ϵ |
Revised: ( ϵ |

Original: z O , x O , y O )
Revised: z O , x O , y O )

Original: the implicit distribution for the posterior after T HMC steps.
Revised: the implicit distribution for the posterior after T HMC steps.

Original: Thehyper-parameters of HMC are named ϕ , a T × d matrix containing the step sizes for each dimension ateach step of the Leapfrog integration.
Revised: The hyper-parameters of HMC are named ϕ , a T × d matrix containing the step sizes for each dimension at each step of the chain.

Original: Within this perspective, the hyper-parameters can be optimizedusing variational inference by maximizing the ELBO
Revised: Within this perspective, the hyper-parameters can be optimized using variational inference by maximizing the ELBO

Original: [Equation]
Revised: [Equation]

Original:  where the first part is the HMC objective, and can be easily estimated via Monte Carlo
Revised: (7) where the first part is the HMC objective, and can be easily estimated via Monte Carlo

Revised: L HMC ( z O , y O ; { θ, ψ, ϕ } ) =

Revised: E q ( T ) ϕ

Revised: ( ϵ ) [log p θ ( z O | h 1 ) + log p θ ( y O

Revised: | ˆ x

Original: [Equation]
Revised: [Equation]

Revised: h 1 )

Revised: +

Revised: [Equation]

Revised: (cid:88)

Revised: [Equation]

Original:  Nevertheless, the entropy term H [ q ( T ) ϕ
Revised: p [Equation] [Equation] Nevertheless, the entropy term H [ q ( T ) ϕ

Original: ( ϵ | x O , y O )] in Equation (7) is not tractable since we are notable to explicitly evaluate the distribution q ( T ) ϕ
Revised: ( ϵ | x O , y O )] in Equation (7) is not tractable since we are not able to explicitly evaluate the distribution q ( T ) ϕ

Original: ( ϵ | x O , y O ) .
Revised: ( ϵ | x O , y O ) .

Original: Although optimizing the first term mightresult in a well-posed algorithm, this would bring consequences that must be considered.
Revised: Although optimizing the first term might result in a well-posed algorithm, this would bring consequences that must be considered.

Original: Namely,without a proper regularization term, and in case the initial proposal q ( T ) ϕ
Revised: Namely, without a proper regularization term, and in case the initial proposal q ( T ) ϕ

Original: ( ϵ | x O , y O ) is concentratedin high density regions, the chains would barely move from the initial state and only these regionswith high density would be explored.
Revised: ( ϵ | x O , y O ) is concentrated in high density regions, the chains would barely move from the initial state and only these regions with high density would be explored (see Section A.3 for illustrative details).

Original: To cope with this problem, we define an inflation parameters to increase the variance of the proposal q ϕ ( ϵ |
Revised: To cope with this problem, we define an inflation parameter s to increase the variance of the proposal q ϕ ( ϵ |

Original: z O , x O , y O ) given by the Gaussian encoder, endingin the proposal (cid:81)
Revised: z O , x O , y O ) given by the Gaussian encoder, ending in the proposal (cid:81)

Original: Ll =1 N ( g µ l ( r l ) , s l · g σ l ( r l )) .
Revised: Ll =1 N ( g µ l ( r l ) , s l · g σ l ( r l )) .

Original: Whilst in [9] the authors define this parameter as a216 scalar factor applied to all the latent dimensions, in our work, we extend this to apply a differentinflation factor at each latent level s = { s 1 , ..., s L } .
Revised: Whilst in [6] the authors define this parameter as a scalar factor applied to all the latent dimensions, in our work, we extend this to apply a different inflation factor at each latent level s = { s 1 , ..., s L } .

Original: In order to tune these inflations we ensure awider coverage of the density by minimizing the Sliced Kernelized Stein Discrepancy (SKSD) [23]
Revised: In order to tune these inflations we ensure a wider coverage of the density by minimizing the Sliced Kernelized Stein Discrepancy (SKSD) [Equation]

Original: [Equation]
Revised: [Equation]

Original: which fits perfectly our requirements, since only requires samples from HMC and gradients∇ ϵ log p ( z O , y O , ϵ ) for measuring a discrepancy between the implicit and the true posterior.
Revised: which fits perfectly our requirements, since only requires samples from HMC and gradients ∇ ϵ log p ( z O , y O , ϵ ) for measuring a discrepancy between the implicit and the true posterior.

Original: Further,the SKSD performs better than other discrepancies like [42] in high dimensional latent spaces.
Revised: Further, the SKSD performs better than other discrepancies like [31] in high dimensional latent spaces.

Original: We provide in Section A.3 a toy demonstration on the efficacy of the HMC optimization, and in
Revised: We provide in Section A.3 a toy demonstration on the efficacy of the HMC optimization, and in

Original: Section A.4 a demonstration of the optimization convergence.
Revised: Section A.4 a demonstration of the optimization convergence.

Original: 3.6 HH-VAEM optimization
Revised: 3.6 HH-VAEM optimization

Original: The optimization of the HH-VAEM is divided into three stages.
Revised: The optimization of the HH-VAEM is divided into three stages.

Original: In a first stage, we train oneindependent marginal VAE per dimension.
Revised: In a first stage, we train one independent marginal VAE per dimension.

Original: In a second stage, the dependency VAE is trained, usingas inputs the concatenation of the encoded dimensions z
Revised: In a second stage, the dependency VAE is trained, using as inputs the concatenation of the encoded dimensions z

Original: = { z 1 , ..., z D } and the target y .
Revised: = { z 1 , ..., z D } and the target y .

Original: Finally,in a third stage, the HMC hyperparameters, the decoder and the predictor are tuned using the HMCobjective, the inflation parameter is trained using the SKSD discrepancy, and the encoder is trainedused the ELBO.
Revised: Finally, in a third stage, the HMC hyperparameters, the decoder and the predictor are tuned using the HMC objective, the inflation parameter is trained using the SKSD discrepancy, and the encoder is trained used the ELBO.

Original: The pseudocode for HH-VAEM training is shown in Algorithm 1.
Revised: The pseudocode for HH-VAEM training is shown in Algorithm 1.

Original: 3.7 Computational cost
Revised: 3.7 Computational cost

Original: In a VAE, the data is fed to the encoder with aim at obtaining the variational parameters for samplingfrom the Gaussian approximated posterior.
Revised: In a VAE, the data is fed to the encoder with aim at obtaining the variational parameters for sampling from the Gaussian approximated posterior.

Original: In our method, the data is similarly encoded to obtain theinitial Gaussian proposal q (0) , and the samples from this distribution are updated for T cycles to obtainthe implicit q ( T ) using HMC.
Revised: In our method, the data is similarly encoded to obtain the initial Gaussian proposal q (0) , and the samples from this distribution are updated for T cycles to obtain the implicit q ( T ) using HMC.

Original: Within each of these iterations, L leapfrog steps (1) are executed.
Revised: Within each of these iterations, L leapfrog steps (1) are executed.

Original: Foreach of these steps, the computation of the gradients ∇ ϵ l log p ( z , y , ϵ l ) and ∇ ϵ l +1 log p ( z , y , ϵ l +1 ) is237 required.
Revised: For each of these steps, the computation of the gradients ∇ ϵ l log p ( z , y , ϵ l ) and ∇ ϵ l +1 log p ( z , y , ϵ l +1 ) is required.

Original: To obtain these gradients, we need to i) compute the parameters of the likelihood p ( z , y | ϵ )238 that are given by the decoder ( p ( z | h 1 ) ) and predictor ( p ( y | ˆ x , h 1 ) ), ii) evaluate it and iii) perform theautomatic differentiation.
Revised: To obtain these gradients, we need to i) compute the parameters of the likelihood p ( z , y | ϵ ) that are given by the decoder ( p ( z | h 1 ) ) and predictor ( p ( y | ˆ x , h 1 ) ), ii) evaluate the likelihood and iii) perform the automatic differentiation.

Original: Thus, for running our method, an additional cost from both decoding andperforming differentiation a total of 2 TL times is introduced.
Revised: Thus, for running our method, an additional cost from both decoding and performing differentiation a total of 2 TL times is introduced.

Original: Input: data (cid:16) x (1: N ) O , y (1: N ) O (cid:17) , steps: T d , T V
Revised: Input: data (cid:16) x (1: N ) O , y (1: N ) O (cid:17) , steps: T d , T V

Original: I , T HMC
Revised: I , T HMC

Original: Parameters: γ , θ , ψ , ϕ , s
Revised: Parameters: γ , θ , ψ , ϕ , s

Original: S TAGE 1: MARGINAL VAE S for d = 1 to D do
Revised: S TAGE 1: MARGINAL VAE S for d = 1 to D do

Original: Initialize marginal VAE { θ d , γ d } for t = 1 to T d do γ t +1 d , θ t +1 d ← Adam γ td ,θ td
Revised: Initialize marginal VAE { θ d , γ d } for t = 1 to T d do γ t +1 d , θ t +1 d ← Adam γ td ,θ td

Original: ( L d ) end for end for
Revised: ( L d ) end for end for

Original: S TAGE 2: DEPENDENCY VAE for t = 1 to T V AE do θ t +1 , ψ t +1 ← Adam θ t ,ψ t ( L V I ) end for
Revised: S TAGE 2: DEPENDENCY VAE for t = 1 to T V AE do θ t +1 , ψ t +1 ← Adam θ t ,ψ t ( L V I ) end for

Original: S TAGE 3: J OINTLY OPTIMIZING VAE + HMC for t = 1 to T HMC do ψ t +1 ← Adam ψ t ( L V I ) θ t +1 , ϕ t +1 ← Adam θ t ,ϕ t ( L HMC ) s t +1 ← Adam s t ( L SKSD ) end for
Revised: S TAGE 3: J OINTLY OPTIMIZING VAE + HMC for t = 1 to T HMC do ψ t +1 ← Adam ψ t ( L V I ) θ t +1 , ϕ t +1 ← Adam θ t ,ϕ t ( L HMC ) s t +1 ← Adam s t ( L SKSD ) end for

Original: By jointly optimizing the HMChyperparameters we are able toachieve faster convergence withsmaller lengths.
Revised: By jointly optimizing the HMC hyperparameters we are able to achieve faster convergence with smaller lengths.

Original: To reduce the computational cost, we optimize  hyperparameters in a final training stage, sinceconvergence is rapidly achieved (asobserved empirically in Fig.7).
Revised: To reduce the computational cost, we optimize the hyperparameters in a final training stage, since convergence is rapidly achieved (as observed empirically in Fig.

Original: Attest, samples from the Gaussian q (0)(faster and cheaper), or from HMCq ( T ) (slower and better) can be usedto fit computational constraints.
Revised: At test, samples from the Gaussian q (0) (faster and cheaper), or from HMC q ( T ) (slower and better) can be used to fit computational constraints.

Original:  Sampling-basedActive Learning
Revised:  Sampling-based Active Learning

Original: Considering that the input data aretuples of observed and missing features { x O , x U } , our Active Learning258 framework follows [44, 45] and determines which feature x i ∈ x U will260 enhance the prediction of the target ythe most for a particular x O .
Revised: Considering that the input data are tuples of observed and missing features { x O , x U } , our Active Learning framework follows [33, 34] and determines which feature x i ∈ x U will enhance the prediction of the target y the most for a particular x O .

Original: Concretely, in a Sequential Active Information Acquisition (SAIA)scenario, this decision is taken sequentially to optimally increase knowledge and accurately predict y .
Revised: Concretely, in a Sequential Active Information Acquisition (SAIA) scenario, this decision is taken sequentially to optimally increase knowledge and accurately predict y .

Original: From a information theoretical perspective, this task can be performed recurrently by maximizing areward function R at each step d .
Revised: From a information theoretical perspective, this task can be performed recurrently by maximizing a reward function R at each step d .

Original: This reward might represent abstract quantities of interest like thecost or benefit of acquiring x i (depending on the sign).
Revised: This reward might represent abstract quantities of interest like the cost or benefit of acquiring x i (depending on the sign).

Original: In Bayesian experimental analysis, R is theexpected gain of information [40].
Revised: In Bayesian experimental analysis, R is the expected gain of information [29].

Original: Following [4], we can define it as
Revised: Following [3], we can define it as

Original: [Equation]
Revised: [Equation]

Original:  where i is the index of each unobserved feature.
Revised: (10) where i is the index of each unobserved feature.

Original: Intuitively, this quantity can be interpreted as theexpected change in the predictive distribution when x i is observed.
Revised: Intuitively, this quantity can be interpreted as the expected change in the predictive distribution when x i is observed.

Original: The reward needs to be estimatedvia Monte Carlo by sampling from p ( x i | x O ) .
Revised: The reward needs to be estimated via Monte Carlo by sampling from p ( x i | x O ) .

Original: With a robust generative model that handles missingdata like HH-VAEM, these samples are easily obtained: first, using HMC, we sample ϵ ( T ) fromp ( ϵ | x O ) .
Revised: With a robust generative model that handles missing data like HH-VAEM, these samples are easily obtained: first, using HMC, we sample ϵ ( T ) from p ( ϵ | x O ) .

Original: Second, we decode these samples to obtain x i from p ( z | h 1 ) and p ( x i | z d ) .
Revised: Second, we decode these samples to obtain x i from p ( z | h 1 ) and p ( x i | z d ) .

Original: Nonetheless, the272 reward defined in (10) is intractable since both p ( y | x i ) and p ( y | x i , x O ) are intractable: computing273 them requires to integrate out the latent variables.
Revised: Nonetheless, the reward defined in (10) is intractable since both p ( y | x i ) and p ( y | x i , x O ) are intractable: computing them requires to integrate out the latent variables.

Original: This motivates the authors of [44, 45] to presenta transformation of the reward for being computed in the latent space using the encoder network.
Revised: This motivates the authors of [33, 34] to present a transformation of the reward for being computed in the latent space using the encoder network.

Original: Although they prove that this transformation effectively provides a good estimation in severaldatasets, we demonstrate that for low dimensional targets (commonly one or two dimensions), anapproximation using histograms is more effective.
Revised: Although they prove that this transformation effectively provides a good estimation in several datasets, we demonstrate that for low dimensional targets (commonly one or two dimensions), an approximation using histograms is more effective.

Original: Concretely, the reward in (10) can be rewritten as
Revised: Concretely, the reward in (10) can be rewritten as

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: which is exactly the Mutual Information between the target and the missign feature x i , conditionedon observing x O .

Original: While a set of advanced non-parametric estimators of the mutual information areavailable [37, 59], many are not easilly adapted for parallelization.
Revised: While a set of advanced non-parametric estimators of the mutual information are available [26, 42], many are not easilly adapted for parallelization.

Original: We demonstrate that the simplestone, based on binning the x i and y domains
Revised: We demonstrate that the simplest one,

Original: [Equation]
Revised: [Equation]

Original:  is effective and easy to parallelize.
Revised: based on binning the x i and y domains, is effective and easy to parallelize.

Original: In the equation, p x ( i ) = n x ( i ) /N , p y ( j ) =
Revised: In the equation, p x ( i ) = n x ( i ) /N , p y ( j ) =

Original: n y ( j ) /N andp ( i, j ) =
Revised: n y ( j ) /N and p ( i, j ) =

Original: n ( i, j ) /N are the relative frequencies that approximate  p ( x ) , p ( y ) and p ( x, y ) for eachbin.
Revised: n ( i, j ) /N are the relative frequencies that approximate [Table] [Table] p ( x ) , p ( y ) and p ( x, y ) for each bin.

Original: Thus, n x ( i ) , n y ( j ) and n ( i, j ) are the number ofsamples inside each interval.
Revised: Thus, n x ( i ) , n y ( j ) and n ( i, j ) are the number of each interval.

Original: The number ofbins defines  width of uniformly distributed intervals over x d and y supports.
Revised: The number of bins defines the width of uniformly distributed intervals over x d and y supports.

Original: Since this estimator issampling-based, under certain conditions (namely, if all densities exist as proper functions), Equation(13) indeed converges to I ( y ; x i | x O ) if we first let the number of samples N → ∞ [37]. [Table] [Table]
Revised: Since this estimator is sampling-based, under certain conditions (namely, if all densities exist as proper functions), Equation (13) indeed converges to I ( y ; x i | x O ) if we first let the number of samples N → ∞ [26].

Original:  Experiments
Revised:  Experiments

Original: The evaluation of the HH-VAEM model is organized into three quantitative experiments and onequalitative experiment.
Revised: The evaluation of the HH-VAEM model is organized into three quantitative experiments and one qualitative experiment.

Original: The ablation study includes the validation of our proposed HMC-based,hierarchical model with respect to the Gaussian and one-layered alternatives.
Revised: The ablation study includes the validation of our proposed HMC-based, hierarchical model with respect to the Gaussian and one-layered alternatives.

Original: Namely, the comparisonis performed with the following baseline models:
Revised: Namely, the comparison is performed with the following baseline models:

Original: • VAEM : 1 layer, Gaussian-based VAEM [44] (without including the Partial VAE).
Revised: • VAEM : 1 layer, Gaussian-based VAEM [33] (without including the Partial VAE).

Original: • MIWAEM : 1 layer, Gaussian-based, importance weighted IWAEM (VAEM + IWAE [47]).
Revised: • MIWAEM : 1 layer, Gaussian-based, importance weighted IWAEM (VAEM + IWAE [36]).

Original: • H-VAEM : 2 layers, Gaussian-based VAEM.
Revised: • H-VAEM : 2 layers, Gaussian-based VAEM.

Original: • HMC-VAEM : 1 layer, HMC-based (with our optimization method) VAEM.(a) Autoregressive
Revised: • HMC-VAEM : 1 layer, HMC-based (with our optimization method) VAEM.

Original: [Figure]
Revised: [Figure]

Revised: [Figure]

Original: [Table] [Table]
Revised: [Table] [Table]

Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: For all the models, we manually introduce missing data in the training set by randomly setting perdata point a feature as missing with a probability sampled uniformly in the interval [0 . 01 , 0 . 99] withineach batch.
Revised: For all the models, we manually introduce missing data in the training set by randomly setting per data point a feature as missing with a probability sampled uniformly in the interval [0 . 01 , 0 . 99] within each batch.

Original: Both the input data x and the target y can be missing.
Revised: Both the input data x and the target y can be missing.

Original: For the test set, a fixed probabilityof 0.5 leads to about half of the input data being observed, whilst the target is completely unobserved.
Revised: For the test set, a fixed probability of 0.5 leads to about half of the input data being observed, whilst the target is completely unobserved.

Original: For the quantitative experiments, a total of 10 UCI datasets [16] that include mixed-type data areemployed for the evaluation.
Revised: For the quantitative experiments, a total of 10 UCI datasets [11] that include mixed-type data are employed for the evaluation.

Original: We include both MNIST [38] and Fashion-MNIST [75] datasets forevaluating our model in higher dimensional observational and latent spaces and bigger architectures(3 layered convolutional nets for encoder/decoder).
Revised: We include both MNIST [27] and Fashion-MNIST [55] datasets for evaluating our model in higher dimensional observational and latent spaces and bigger architectures (3 layered convolutional nets for encoder/decoder).

Original: The qualitative results are included in Section A.8, where we evaluate our model in the image inpainting task on MNIST and CelebA [43].
Revised: For the qualitative results, we evaluate our model in the image inpainting task on MNIST and CelebA [32].

Original: Forthese two image datasets, the marginalVAEs are not included and the dependency VAE is fed directlywith the Bernoulli-distributed pixels.
Revised: For the three image datasets, the marginal

Original:  We name this model HH-VAE, and similarly, the baselinesare renamed as VAE, MIWAE, HMC-VAE and H-VAE.
Revised: [Figure] We name this model HH-VAE, and similarly, the baselines are renamed as VAE MIWAE HMC and H-VAE.

Original: Extended experiments and details on theemployed architectures and evaluation times are provided in the Appendix. [Figure]
Revised: Extended experiments and validations are provided in the Appendix. The source code for reproducing our work is available at https://github com/ipeis/HH

Original: 5.1 Mixed type conditional data imputation
Revised: 5.1 Mixed type conditional data imputation

Original: In order to evaluate the performance of the model in terms of data imputation, we opt by computing thenegative log likelihood of the unobserved features.
Revised: In order to evaluate the performance of the model in terms of data imputation, we opt by computing the negative log likelihood of the unobserved features.

Original: We make use of the Monte Carlo approximation
Revised: We make use of the Monte Carlo approximation

Original: [Equation]
Revised: [Equation]

Original: (13) which is averaged over features in order to compare the imputation performance with the baselines.
Revised: which is averaged over features in order to compare the imputation performance with the baselines.

Revised: Additionally, we include in Section A.7 similar results averaging each of the considered likelihoods.

Original: Results on the 10 UCI datasets and the MNIST datasets are included in Tables 1 and 3, showing thatfor most of the datasets, incremental improvement is obtained: VAEM < H-VAEM <
Revised: Results on the 10 UCI datasets and the MNIST datasets are included in Tables 1 and 3, showing that for most of the datasets, incremental improvement is obtained: VAEM < H-VAEM <

Original: HMC-VAEM<
Revised: HMC-VAEM <

Original: HH-VAEM.
Revised: HH-VAEM.

Original: Extended results with the imputation error, included in Section A.5, corroborate this.
Revised: Extended results with the imputation error, included in Section A.5, corroborate this.

Original: 5.2 Target prediction
Revised: 5.2 Target prediction

Original: For this experiment, we compute the negative log likelihood of the target under the predictivedistribution using the same procedure as in the previous section.
Revised: For this experiment, we compute the negative log likelihood of the target under the predictive distribution using the same procedure as in the previous section.

Original: Results included in Tables 2 and 4show the same incremental improvement in the prediction task.
Revised: Results included in Tables 2 andshow the same incremental improvement in the prediction task.

Original: 5.3 Sequential active information acquisition (SAIA)
Revised: 5.3 Sequential active information acquisition (SAIA)

Original: In this experiment, the HH-VAEM model and ourproposed acquisition function are evaluated in a SAIA task.
Revised: In this experiment, our HH-VAEM model and our acquisition method are evaluated in a SAIA task.

Original: We start by predicting from completely unobserved inputs. At each step, the missingfeature that maximizes the reward is acquired sequentially.
Revised: Starting by predicting from completely unobserved inputs, at each step, the missing feature that maximizes the reward is acquired.

Original: Figure 5 shows theresulting error curvesfor the UCI datasets.
Revised: Figure 5 shows the error curves for the UCI datasets.

Original: Blue lines correspond to the transformed reward proposed by [44].
Revised: Blue lines correspond to the Gaussian-based reward proposed by [33].

Original: Orange linesare our  reward in a VAEM framework (not including the hierarchy and the HMC sampler).
Revised: Orange lines are our sampling-based reward in a VAEM framework.

Original: Greenlines correspond to HH-VAEM with our reward.
Revised: Green lines correspond to HH-VAEM with our reward.

Original: In most of the cases, our model and acquisitiontechnique outperform the baselines, obtaining lower errors anda faster discovery of information. 
Revised: In most of the cases, our model and acquisition method obtains lower errors and faster discovery of information. 5.4 Conditional image inpainting

Revised: We include in this experiment qualitative results when comparing our model with the baselines on the image conditional inpainting task.

Revised: We include results for MNIST and CelebA in Figure 6 (a)

Revised: [Figure]

Revised: and (b), respectively, that show the superiority of our method.

Original:   Conclusion
Revised: First, the hierarchical Gaussian model (fourth row) considerably improves the one-layered Gaussian alternative. Second, the HMC-based methods (two last rows) vastly improve the Gaussian methods. Third, in some specific cases, an extra improvement is added by HH-VAE with respect to the one-layered HMC-based model (columns 2, 4, 8, 9, 11, 12 and 13 in Figure 6 (a) or columns 1, 2, 5 and 12 in (b)).  Conclusion

Original: We presented HH-VAEM, to our knowledge, the first hierarchical VAE for mixed-type incompletedata that uses HMC with automatic hyper-parameter tuning for improved inference.
Revised: We presented HH-VAEM, to our knowledge, the first hierarchical VAE for mixed-type incomplete data that uses HMC with automatic hyper-parameter tuning for improved inference.

Original: We provide bothquantitative and qualitative experiments that demonstrate its superiority with respect the baselines inthe tasks of missing data imputation and supervised learning, placing HH-VAEM as a robust modelfor real-world datasets.
Revised: We provide both quantitative and qualitative experiments that demonstrate its superiority with respect the baselines in the tasks of missing data imputation and supervised learning, placing HH-VAEM as a robust model for real-world datasets.

Original: Further, we have developed a novel sampling-based technique for dynamicfeature selection that outperforms the Gaussian-based alternatives and results in an efficient methodfor active learning in deep generative models. 
Revised: Further, we have developed a novel sampling-based technique for dynamic feature selection that outperforms the Gaussian-based alternatives and results in an efficient method for active learning in deep generative models. Acknowledgements

Revised: Ignacio Peis acknowledges support from Spanish government Ministerio de Ciencia, Innovación y Universidades under grants FPU18/00516, RTI2018-099655-B-I00, EST21/00467 and PID2021123182OB-I00 and from Comunidad de Madrid under grant Y2018/TCS-4705 PRACTICO-CM.

Revised: José

Revised: Miguel Hernández-Lobato acknowledges support from Boltzbit.

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes](b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See Section 3.7 about thecomputational cost.(c)
Revised: [Yes] See Section 3.7 about the computational cost.

Original:  Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [No] We do notbelieve there could be any potential negative societal impacts.(d)
Revised: [No] We do not believe there could be any potential negative societal impacts.

Original:  Have you read the ethics review guidelines and ensured that your paper conforms tothem?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] We include the code for reproducing all the presented experiments in the supplemental material.
Revised: [Yes] We provide the URL of the open source repository for reproducing all our experiments.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] See Appendix A.1 with all the training details.
Revised: [Yes] See Appendix A.1 with all the training details.

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] We include error intervals in all our experiments.
Revised: [Yes] We include error intervals in all our experiments.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Appendix A.1 with all the training details.
Revised: [Yes] See Appendix A.1 with all the training details.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] We have cited every dataset/model employed or mentioned in our work.
Revised: [Yes] We have cited every dataset/model employed or mentioned in our work.

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes] We include the code for reproducing all the presented experiments in the supplemental
Revised: [Yes] We provide the URL of the open source repository for reproducing all our experiments.

Original: material. (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [N/A] 
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: A AppendixA.1 Experimental setup
Revised: A Appendix A.1 Experimental setup

Original: The networks for the encoder of the model with the MNIST datasets are 2 layered Deep CNNs with{16, 32, 32} output channels, kernel size 5, stride 2 and padding 2.
Revised: The networks for the encoder of the model with the MNIST datasets are 2 layered Deep CNNs with {16, 32, 32} output channels, kernel size 5, stride 2 and padding 2.

Original: For the experiments with CelebA,we use 5 layered Deep CNNs with {32, ,32, 64, 64, 512} output channels, kernel size 4, stride 2 andpadding 1, followed by batch norm layers.
Revised: For the experiments with CelebA, we use 5 layered Deep CNNs with {32, 32, 64, 64, 512} output channels, kernel size 4, strideand padding 1, followed by batch norm layers.

Original: They are followed by MLPs  for obtaining the variationalparameters for each layer.
Revised: They are followed by MLPs with 512 hidden units for obtaining the variational parameters for each layer.

Original: The decoder that obtains p θ ( x | h 1 ) is the symmetric convolutional network.
Revised: The decoder that obtains p θ ( x | h 1 ) is the symmetric CNN.

Original: All the neural networks employed in the models trained with UCI datasets are one single layer MLPswith 256 hidden units.
Revised: All the NNs employed in the models trained with UCI datasets are one single layer MLPs with 256 hidden units.

Original: The noise variance for Gaussian likelihoods is set up to 0 . 1 .
Revised: The noise variance for Gaussian likelihoods is set up to 0 . 1 .

Original: We employa learning rate of 1 × 10 − 3 forall the parameters except for the  inflation parameter s ,where we increase to 1 × 10 − 2 for a faster convergence.
Revised: We employ learning rates of 1 × 10 − 3 for the models with MLP networks and 2 × 10 − 4 for the convolutional models. For the inflation parameter s , we increase to 1 × 10 − 2 for a faster convergence.

Original: A batch size of 100 is used for all the modelsexcept for Yatch and Wine dataset, where we use 20 samples per batch.
Revised: A batch size of 100 is used for all the models except for Yatch and Wine dataset, where we use 20 samples per batch.

Original: The number of training stepsis 20 × 10 3 for Boston, Energy, Wine, Yatch, Concrete, Diabetes and Yatch, and 50 × 10 3 for Naval, Avocado, Bank, and Insurance.
Revised: The number of training steps is 2 × 10 4 for Boston, Energy, Wine, Yatch, Concrete, Diabetes and Yatch, and 5 × 10 4 for Naval, Avocado, Bank, and Insurance.

Original: For MNIST and Fashion-MNIST, we have 100 × 10 3 training steps.
Revised: For MNIST and Fashion-MNIST, we have 1 × 10 5 training steps.

Revised: For CelebA, we use 1 , 5 × 10 5 training steps.

Original: For the marginal VAEs stage, we employ 1 × 10 3 training steps.
Revised: For the marginal VAEs stage, we employ 1 × 10 3 training steps.

Original: The dimension of the latent variablesis [ d 1 = 10 , d 2 = 5] for Boston, Energy, 
Revised: The dimension of the latent variables is [ d 1 = 10 , d 2 = 5] for Boston, Energy, Wine, Naval Avocado, Bank and Insurance

Revised: [ d 1 = 4 , d 2 = 2] for Concrete, Yatch and Diabetes, [ d 1 = 20 , d 2 = 10] for MNIST and FashionMNIST, and [ d 1 = 32 , d 2 = 16] for CelebA.

Original: We use LF = 5 Leapfrog steps, chains of T = 10 for Boston, Energy, Wine, NavalAvocado, Bank,
Revised: We use LF = 5 Leapfrog steps in all cases, chains of T = 10 for Boston, Energy, Wine, Naval

Original:  Insurance, MNIST and Fashion-MNIST, and T = 5 for Concrete, Yatch and Diabetes.
Revised: Avocado, Bank, Insurance, MNIST, Fashion-MNIST and CelebA, and T = 5 for Concrete, Yatch and Diabetes.

Original: The SKSDfunction is estimated using 30 HMC samples.
Revised: The SKSD function is estimated using 30 HMC samples.

Original: For the MNIST  datasets, the use of Nvidia P100 GPU with Pascal architecture sped up the trainingwith the CNN-based architecture.
Revised: For the MNIST and CelebA datasets, the use of Nvidia P100 GPU with Pascal architecture sped up the training with the CNN-based architecture.

Original: For the UCI datasets, due to the use of small networks, thedifferences using CPU or GPU are almost imperceptible.
Revised: For the UCI datasets, due to the use of small networks, the differences when using CPU or GPU are almost imperceptible.

Original: A.2 Datasets information
Revised: A.2 Datasets information

Original: Apart from the MNIST and Fashion-MNIST  datasets, a total of 10 UCI datasets have been employedin this work including: Bank Marketing, Insurance Company Benchmark, Avocado sales,Naval
Revised: Apart from the MNIST, Fashion-MNIST and CelebA datasets, a total of 10 UCI datasets have been employed in this work including: Bank Marketing, Insurance Company Benchmark, Avocado sales,

Original:  Propulsion Plants, Yatch Hydrodynamics, Diabetes, Boston Housing, Wines, Energy efficiency and Bank Marketing.
Revised: Naval Propulsion Plants, Yatch Hydrodynamics, Diabetes, Boston Housing, Wines, Energy efficiency and Bank Marketing.

Original: A.3 Efficacy of training HMC
Revised: A.3 Efficacy of training HMC

Original: We include in this experiment results that demonstrate the efficacy of training thehyperparametersof HMC  using our proposed gradient-based strategy onseveral 2D densities.
Revised: We include in this experiment results that demonstrate the efficacy of training the HMC hyperparameters using our proposed gradient-based strategy on 2D densities.

Original: In Figure 6, each rowcorrespond to an example with a different density: first row is a wave density,while second row isa dual moon density.
Revised: In Figure 7, each row correspond to an example with a different density: first row is a wave density, second row is a dual moon density.

Original: In the first column, the initial set up is showed, including the density contour(dark blue), the initial Gaussian proposal contour (light blue) and samples from HMC (green).
Revised: In the first column, the initial set up is showed, including the density contour (dark blue), the initial Gaussian proposal contour (light blue) and samples from HMC (green).

Original: Inboth cases, due to the tightness of the proposal, chains do not properly explore the density and getstuck close the initial state.
Revised: In both cases, due to the tightness of the proposal, chains do not properly explore the density and get stuck close the initial state.

Original: In the second column, results after training HMC hyperparameters andthe inflation parameter are included.
Revised: In the second column, results after training HMC hyperparameters and the inflation parameter are included.

Original: Again, in both cases, and more specially in the first one, theinflation of the horizontal variance of the proposal successfully increases to better cover the surface,and final HMC samples vastly improve the exploration.
Revised: Again, in both cases, and more specially in the first one, the inflation of the horizontal variance of the proposal successfully increases to better cover the surface, and final HMC samples vastly improve the exploration.

Original: In the third column, the approximations ofthe HMC objective, the SKSD discrepancy and the inflation parameters over the optimization stepsare included.
Revised: In the third column, the approximations of the HMC objective, the SKSD discrepancy and the inflation parameters over the optimization steps are included.

Original: A.4 HMC optimization
Revised: A.4 HMC optimization

Original: We face the computational cost of running HMC by defining a small percentage of training steps forthe last stage in Algorithm 1.
Revised: We face the computational cost of running HMC by defining a small percentage of training steps for the last stage in Algorithm 1.

Original: A 10% of the total training steps for T HMC is sufficient for obtainingthe convergence.
Revised: A 10% of the total training steps for T HMC is sufficient for obtaining the convergence.

Original: In Figure 7 we include the validation metrics obtained during the optimization of
Revised: In Figure 8 we include the validation metrics obtained during the optimization of

Original: Yatch dataset, where T HMC = 2 e 3 .
Revised: Yatch dataset, where T HMC = 2 × 10 3 .

Original: In figure 8 (a), the mean acceptance rate of the HMC sampler over the training steps and the step sizes(b) are included.
Revised: In figure 9 (a), the mean acceptance rate of the HMC sampler over the training steps and the step sizes (b) are included.

Original: The steps are initialized from U (0 . 05 , 0 . 2) .
Revised: The steps are initialized from U (0 . 05 , 0 . 2) .

Original: After 2 × 10 3 steps, the mean acceptancerate converges to a value closer to ¯ p a = 0 .
Revised: After 2 × 10 3 steps, the mean acceptance rate converges to a value closer to ¯ p a = 0 .

Original: 65 , which is defined as the optimal desired acceptanceprobability [52].
Revised: 65 , which is defined as the optimal desired acceptance probability [39].

Original: This empirical result provides evidence that, apart from reducing the computationalcost, reducing the HMC training step to this value is sufficient for achieving convergence.
Revised: This empirical result provides evidence that, apart from reducing the computational cost, reducing the HMC training step to this value is sufficient for achieving convergence.

Original: A.5 Deterministic imputation metrics
Revised: A.5 Deterministic imputation metrics

Original: We include in Table 6 results on the RMSE obtained with other discriminative validated predictors,using mean imputation under the same missing rates.
Revised: We include in Table 6 results on the RMSE obtained with other discriminative validated predictors, using mean imputation under the same missing rates.

Original: Additionally, we include here the missForest inthe baselines, a wide-spread method for missing data imputation using a Random Forest approach[67].
Revised: Additionally, we include here the missForest in the baselines, a wide-spread method for missing data imputation using a Random Forest approach [49].

Original: For classification tasks, the error rate is considered.
Revised: For classification tasks, the error rate is considered.

Original: In almost all cases, HH-VAEM outperformsthe baselines.
Revised: In almost all cases, HH-VAEM outperforms the baselines.

Original: [Table]
Revised: [Table]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: A.6 Likelihood of the observed features
Revised: A.6 Likelihood of the observed features

Original: [Figure]

Original: We include in this section results on the negative log likelihood of the observed features
Revised: We include in this section results on the negative log likelihood of the observed features

Original: [Figure]

Original: [Equation]
Revised: [Equation]

Original: Results are included in Table 7.
Revised: Results are included in Table 7.

Original: In almost all the cases [Figure] we confirm the incremental superiority whenadding each part of our proposed design
Revised: In almost all the cases, we confirm the incremental superiority when adding each part of our proposed design

Original: [Figure]
Revised: [Figure]

Original: [Table][Figure] [Table] [Figure]
Revised: [Table]

Original: [Table] [Figure] [Table] [Figure]
Revised: [Figure]

Original: [Table][Figure] [Table] [Figure]
Revised: [Table]

Original: [Table] [Figure] [Table] [Figure]
Revised: [Figure]

Original: [Table][Figure] [Table] [Figure]
Revised: [Table]

Original: [Table] [Figure] [Table] [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: A.7 Heterogeneous likelihoods
Revised: A.7 Heterogeneous likelihoods

Original: In experiment 5.1 we reported an average likelihood across heterogeneous variables.
Revised: In experiment 5.1 we reported an average likelihood across heterogeneous variables.

Original: Althoughthis quantity is not a valid joint likelihood probability
Revised: Although this quantity is not a valid joint likelihood probability

Original: [Table]
Revised: [Table]

Original: we employed this average to provide a faircomparison on models that have been trained on the same heterogeneous likelihoods.
Revised: we employed this average to provide a fair comparison on models that have been trained on the same heterogeneous likelihoods.

Original: In this section
Revised: In this section

Original: [Table]
Revised: [Table]

Original: we show the comparison on averaging separately the three considered marginal likelihoods (Gaussian,
Revised: we show the comparison on averaging separately the three considered marginal likelihoods (Gaussian,

Original: Bernoulli and Categorical) for two of the biggest datasets considered on Tables 8-10.
Revised: Bernoulli and Categorical) for two of the biggest datasets considered on Tables 8-10.

Original: Again, we showthe incremental superiority when adding the different design choices of our model
Revised: Again, we show the incremental superiority when adding the different design choices of our model.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: A.8 Conditional image inpainting
Revised: A.8 Reparameterization trick for solving ill-posed HMC with hierarchical densities

Original: In this section, we include qualitative results when comparing our model with the baselines onimage conditional inpainting task. A better pixel imputation is expecting when conditioning onthe observed pixels, since samples from the approximate posterior are required. We evaluate theinpainting experiment by applying the following sequence of steps:
Revised: We provide in this section strong empirical demonstration on the efficacy of our proposed reparameterization trick. As stated in Section 3.4, naïve implementations of HMC are ill-posed when combined with hierarchical densities.

Original: (i) We generate a square mask of missing pixels covering part of the image (with side of 14pixels for 28 × 28 images, and 32 for 64 × 64 images). (ii)
Revised: The autoregressive correlations lead to non-smooth densities with huge peaks.

Original: We use the model encoder to obtain q ( z | x O ) (or q ( ϵ | x O ) in our reparameterized hierarchy). (iii) We draw S samples { z (1) , ..., z ( S ) } or { ϵ (1) , ..., ϵ ( S ) } of the posterior approximation.
Revised: The large gradients ∇ h 1: L log p ∗ ( h 1: L ) evaluated on these regions inside the Leapfrog steps of Equations 1 make huge modifications of the proposed states.

Original: (iv) We use the decoder to obtain p ( x U | z ( s ) ) (or p  (x U | h ( s ) 1 ) for the hierarchical models).
Revised: If we denote these diverged states by h ( ill ) 1: L , evaluating the objective log p ∗ ( h ( ill ) 1: L ) lead to overflow issues.

Original: (v) The imputation is obtained by averaging [Equation]
Revised: This undesired behavior is what we call divergence of the Leapfrog integrator , and is also illustrated by [4] in Figure 35.

Original: where ˆ x ( s ) U are the decoded Bernoulli parameters for each sample.
Revised: In order to avoid the aforementioned overflow issues, we can directly reject these problematic states. Nevertheless, by doing this, HMC will not properly explore the density by the time the states fall into the problematic regions, leading to extremely low acceptance rates.

Original: We includethe results for MNIST and CelebA in Figure 9 (a) and (b), respectively. From top tobottom, each row correspond to: i) original image, ii) observed pixels.
Revised: To demonstrate this, we include in Figure 10 (a) with blue line the evolution of the acceptance rates when optimizing HMC with the

Original: From iii) to vi): inpaintingwith VAE, HVAE, HMC-VAE and HH-VAE (last row), respectively.
Revised: HH-VAEM variant without reparameterization (as illustrated in Figure 4 (a)).

Original: As observed, we prove thehypothesis of obtaining improved imputation when using better samples from our tuned HMC.
Revised: The acceptance rate is extremely low as expected, which is a clear indicator of a poorly mixing sampler.

Original: Byobserving the results, several conclusions can be made. First, the hierarchical Gaussian-based model(fourth row) considerably improves the one-layered Gaussian alternative.
Revised: On the contrary, by using our proposed reparameterization trick (Figure 4 (b)), we are able to make

Original: Second, the HMC-basedmethods (two last rows) vastly improve the Gaussian alternatives. Third, in some specific cases, anextra improvement is added by HH-VAE with respect to the one-layered HMC-based alternative (forexample, columns 2, 4, 8, 9, 11, 12 and 13 in Figure 9 (a) or column 1 in (b)).
Revised: HMC work properly and tune the step sizes, getting closer to the ideal acceptance rate ¯ p a = 0 .[39]. Evaluations are applied to the same validation split of the Boston dataset.

Original: [Figure]
Revised: The step sizes of

Original: , our model, where ˆ x U is decoded from samples of the approximate posterior (Gaussian for VAE and H-VAE, or HMC-based for HMC-VAE and HH-VAE). [Figure]
Revised: HMC are initialized from U (0 . 05 , 0 . 2) in both cases. Further, we also include in Figure 10 (b) the evolution of the imputation log likelihood metric of

Original: o
Revised: Equation (13) during the whole optimization for both approaches. First 1 , 8 × 10 4 steps correspond to the pretraining stage using only the ELBO. When introducing HMC without the reparameterization, due to the low acceptance rate, the states hardly move from the initial proposal, or move away from the density, and the joint optimization fail. We demonstrate again the effectiveness of the reparameterization trick by observing an increase in this metric.

Original: [Figure]
Revised: [Figure]

Original: li [Figure]
Revised: leading to a considerable increase of the metric.

Original: A.9 Balancing the KLs
Revised: A.9 Balancing the KLs

Original: [Figure] Following [72], we define a short initial warming stage (10% of the total training steps) during theoptimization where the KLs  are balanced according to their magnitude and the corresponding latentdimension, preventing the model for falling into posterior collapse.
Revised: Following [53], we define a short initial warming stage (10% of the total training steps) during the optimization where the KLs for the different layers are balanced according to their magnitude and the corresponding latent dimension, preventing the model for falling into posterior collapse by ignoring deepest layers.

Original: A factor is applied to each KL,following
Revised: A factor is applied to each KL, following

Original: [Equation]
Revised: [Equation]

Original: The factors penalises the fact that a layer might be ignored by making the KL smaller when itsmagnitude is small compared to the rest layers.
Revised: The factors penalises the fact that a layer might be ignored by making the KL smaller when its magnitude is small compared to the rest layers.

Original: A.10 SAIA log-likelihoods
Revised: A.10 SAIA log-likelihoods

Original: In order to extend the results provided in Section 5.
Revised: In order to extend the results provided in Section 5.3, we include here the log-likelihoods curves when dynamically selecting features using the same procedure (Figure 11).

Original: [Table]
Revised: [Figure]

Original: we include here the log-likelihoods curveswhen dynamically selecting features using the same procedure (Figure 10)
Revised: l o [Figure] li

Original: [Table]
Revised: [Figure]

Original: A.11 Training times
Revised: li [Figure] d

Original: [Table]
Revised: [Figure]

Original: Table 11 shows the average training time in minutes for each model in the experiments for Tables 1and 2.
Revised: Table 11 shows the average training time in minutes for each model in the experiments for Tablesand 2.

Original: The ratio between training times for our method and the Gaussian baselines is approximatelybetween 5 and 10.
Revised: The ratio between training times for our method and the Gaussian baselines is approximately between 5 and [Figure] Energy [Figure]

Original: [Table]
Revised: [Table]

Original: A.12 SAIA times
Revised: A.12 SAIA times

Original: The times for obtaining the SAIA metric curves in Section 5.3 are included in Figure 11.
Revised: The times for obtaining the SAIA metric curves in Section 5.3 are included in Figure 12.

Original: Althoughthe performance is improved with HH-VAEM, it requires considerably higher time than the baselines[Figure] to evaluate the reward, due to the HMC algorithm for sampling from the true posterior.
Revised: Although the performance is improved with HH-VAEM, it requires considerably longer time than the baselines to evaluate the reward, due to the HMC algorithm for sampling from the better approximated posterior.

Original: Future workmight be oriented in proposing ways to measure and reduce this gap.
Revised: Future work might be oriented in proposing ways to measure and reduce this gap [Table]

Revised: [Figure]