Original: Missing Data Imputation and Acquisition with Deep Hierarchical Models and Hamiltonian Monte Carlo
Revised: Missing Data Imputation and Acquisition with Deep Hierarchical Models and Hamiltonian Monte Carlo

Original: Abstract
Revised: Abstract

Original: Variational Autoencoders (VAEs) have recently been highly successful at imputingand acquiring heterogeneous missing data.
Revised: Variational Autoencoders (VAEs) have recently been highly successful at imputingand acquiring heterogeneous missing data.

Original: However, within this specific application domain, existing VAE methods are restricted by using only one layer oflatent variables and strictly Gaussian posterior approximations.
Revised: However, within this specific application domain, existing VAE methods are restricted by using only one layer oflatent variables and strictly Gaussian posterior approximations.

Original: To address theselimitations, we present HH-VAEM, a Hierarchical VAE model for mixed-type incomplete data that uses Hamiltonian Monte Carlo with automatic hyper-parametertuning for improved approximate inference.
Revised: To address theselimitations, we present HH-VAEM, a Hierarchical VAE model for mixed-type incomplete data that uses Hamiltonian Monte Carlo with automatic hyper-parametertuning for improved approximate inference.

Original: Our experiments show that HH-VAEMoutperforms existing baselines in the tasks of missing data imputation and supervised learning with missing features.
Revised: Our experiments show that HH-VAEMoutperforms existing baselines in the tasks of missing data imputation and supervised learning with missing features.

Original: Finally, we also present a sampling-basedapproach for efficiently computing the information gain when missing features areto be acquired with HH-VAEM.
Revised: Finally, we also present a sampling-basedapproach for efficiently computing the information gain when missing features areto be acquired with HH-VAEM.

Original: Our experiments show that this sampling-basedapproach is superior to alternatives based on Gaussian approximations.
Revised: Our experiments show that this sampling-basedapproach is superior to alternatives based on Gaussian approximations.

Original:  Introduction
Revised:  Introduction

Original: Many real-world unsupervised learning tasks require dealing with complicated datasets with mixedtypes (real, positive-valued, continuous, or discrete) and missing values.
Revised: Many real-world unsupervised learning tasks require dealing with complicated datasets with mixedtypes (real, positive-valued, continuous, or discrete) and missing values.

Original: For this purpose, variationalautoencoders [22, 23, 37] stand out in the recent literature as robust generative models that efficientlyhandle high-dimensional data.
Revised: For this purpose, variationalautoencoders [34, 35, 57] stand out in the recent literature as robust generative models that efficientlyhandle high-dimensional data.

Original: However, in their naive configuration, every data dimension is assumedto have similar statistical properties (i e., homogeneity), and all dimensions are considered to becompletely observed.
Revised: However, in their naive configuration, every data dimension is assumedto have similar statistical properties (i e., homogeneity), and all dimensions are considered to becompletely observed.

Original: Both assumptions won’t hold in many real-world scenarios.
Revised: Both assumptions won’t hold in many real-world scenarios.

Original: Recent workshave adapted VAEs to handle incomplete [10, 15, 29, 31] and mixed-type data [17, 28, 33] anddemonstrated improved performance in downstream tasks such as missing data imputation and activeinformation acquisition.
Revised: Recent workshave adapted VAEs to handle incomplete [12, 19, 44, 46] and mixed-type data [24, 43, 49] anddemonstrated improved performance in downstream tasks such as missing data imputation and activeinformation acquisition.

Original: Despite these advances, existing approaches are far from optimal as they arebased on restrictive design choices: 1), only one layer of latent variables are considered; 2), Gaussianposterior approximations are usually adopted.
Revised: Despite these advances, existing approaches are far from optimal as they arebased on restrictive design choices: 1), only one layer of latent variables are considered; 2), Gaussianposterior approximations are usually adopted.

Original: These will lead to limited flexibility and additionalbias, especially under real-world settings with complex mixed-type incomplete data.
Revised: These will lead to limited flexibility and additionalbias, especially under real-world settings with complex mixed-type incomplete data.

Original: In the literature, the issue of model flexibility and inference bias are often addressed separately.
Revised: In the literature, the issue of model flexibility and inference bias are often addressed separately.

Original: For example, approximate inference bias can be reduced by using Monte Carlo sampling [42, 45].
Revised: For example, approximate inference bias can be reduced by using Monte Carlo sampling [62, 68].

Original: More specifically, Hamiltonian Monte Carlo (HMC) [6, 13] stands out among MCMC methods in machine learning due to its superior efficiency for exploring the target density.
Revised: More specifically, Hamiltonian Monte Carlo (HMC) [6, 17] stands out among MCMC methods inmachine learning due to its superior efficiency for exploring the target density.

Original: In the context of VAE, HMChas also been combined with stochastic variational inference [8] for improving the training of VAEs.
Revised: In the context of VAE, HMC has also been combined with stochastic variational inference [10] for improving the trainingof VAEs.

Original: On the other hand, the flexibility of VAEs can be improved by considering hierarchical VAEs withmultiple layers of hidden variables [9, 30, 43, 46].
Revised: On the other hand, the flexibility of VAEs can be improved by considering hierarchical VAEs with multiple layers of hidden variables [11, 45, 65, 71].

Original: By using a hierarchical structure in the latent space,they enforce the information to flow from high-level representations to more specific observablefactors, imitating the way information is often organized in the real world.
Revised: By using a hierarchical structure inthe latent space, they enforce the information to flow from high-level representations to more specificobservable factors, imitating the way information is often organized in the real world.

Original: However, the issue of modeling flexibility and approximate inference bias are often heavily inter- twined, and addressing them simultaneously in a joint manner is highly non-trivial
Revised: However, the issue of modeling flexibility and approximate inference bias are often heavily intertwined, and addressing them simultaneously in a joint manner is highly non-trivial

Original: The hierarchical organization of the latent variables creates complicated posterior dependencies that are not straightforward to deal with and require special consideration.
Revised: The hierarchical organization of the latent variables creates complicated posterior dependencies that are not straightforward to deal with and require special consideration.

Original: To improve Gaussian approximate inference,most works opt by defining shared paths between the recognition and generative networks.
Revised: To improve Gaussian approximate inference,most works opt by defining shared paths between the recognition and generative networks.

Original: While thismakes hierarchical VAEs practical, the bias introduced by the Gaussian approximations is still present.
Revised: While thismakes hierarchical VAEs practical, the bias introduced by the Gaussian approximations is still present.

Original: To the best of our knowledge, none of the aforementioned hierarchical VAEs has been previouslycombined with Monte Carlo algorithms for improving over standard Gaussian approximate inference.
Revised: To the best of our knowledge, none of the aforementioned hierarchical VAEs has been previouslycombined with Monte Carlo algorithms for improving over standard Gaussian approximate inference.

Original: To overcome such limitations, we focus on training new hierarchical VAE models for heterogeneousmixed-type data with HMC.
Revised: To overcome such limitations, we focus on training new hierarchical VAE models for heterogeneousmixed-type data with HMC.

Original: Our models can be used for missing data imputation and for supervisedlearning with missing data.
Revised: Our models can be used for missing data imputation and for supervisedlearning with missing data.

Original: We also present a sampling-based framework that allows our models toperform accurate sequential active information acquisition.
Revised: We also present a sampling-based framework that allows our models toperform accurate sequential active information acquisition.

Original: Our main contributions are as follows:
Revised: Our main contributions are as follows:

Original: • We present HH-VAEM, a deep hierarchical model for handling mixed-type incompletedata that uses HMC with automatic hyper-parameter tuning for outperforming amortizedvariational inference by generating low bias samples from the true posterior.• We propose a sampling-based strategy for missing feature acquisition that benefits from theimproved inference of HH-VAEM.
Revised: • We present HH-VAEM, a deep hierarchical model for handling mixed-type incompletedata that uses HMC with automatic hyper-parameter tuning for outperforming amortizedvariational inference by generating low bias samples from the true posterior.• We propose a sampling-based strategy for missing feature acquisition that benefits from theimproved inference of HH-VAEM.

Original: By using histograms to estimate the mutual information,this strategy achieves lower bias than other Gaussian-based alternatives.
Revised: By using histograms to estimate the mutual information,this strategy achieves lower bias than other Gaussian-based alternatives.

Original: • We exhaustively evaluate HH-VAEM in the tasks of 1) missing data imputation, 2) supervisedlearning with missing data and 3) information acquisition with our sampling-based strategy.
Revised: • We exhaustively evaluate HH-VAEM in the tasks of 1) missing data imputation, 2) supervisedlearning with missing data and 3) information acquisition with our sampling-based strategy.

Original: In all cases we report significant gains with respect to baselines.
Revised: In all cases we report significant gains with respect to baselines.

Original:  Related work
Revised:  Related work

Original: 2.1 VAEs for mixed-type incomplete data
Revised: 2.1 VAEs for mixed-type incomplete data

Original: Variational Autoencoders [22, 37] are deep generative models that make use of encoder and decodernetworks for mapping data into a latent Gaussian distribution, and reconstructing the latent codes intothe original observational space, respectively.
Revised: Variational Autoencoders [34, 57] are deep generative models that make use of encoder and decodernetworks for mapping data into a latent Gaussian distribution, and reconstructing the latent codes intothe original observational space, respectively.

Original: The parameters of these networks are trained usingamortized Variational Inference [11, 50] optimizing a lower bound (ELBO) on the log evidence:
Revised: The parameters of these networks are trained usingamortized Variational Inference [13, 75] optimizing a lower bound (ELBO) on the log evidence:

Original: [Equation] where the generative model p θ ( x , z ) can be expressed in terms of the likelihood p θ ( x | z ) and the priorp ( z ) .
Revised: E q z ( z | x ) log p θ ( x , z ) q ψ ( z | x ) , where the generative model p θ ( x , z ) can be expressed in terms of the likelihoodp θ ( x | z ) and the prior p ( z ) .

Original: In mixed-type data, the vector x is composed by data from different types: real, positive real,categorical, binary, etc.
Revised: In mixed-type data, the vector x is composed by data from differenttypes: real, positive real, categorical, binary, etc.

Original: A naive approach is to consider a factorized decoder using different likelihoodcontributions p θ ( x | z ) = 
Revised: A naive approach is to consider a factorized decoderusing different likelihood contributions p θ ( x | z ) = (cid:81)

Original: (cid:81) d p θ ( x d | z ) [2, 33].
Revised: d p θ ( x d | z ) [2, 49].

Original: Nonetheless, the problem of handling unbalancedlikelihoods leads to the domination of some dimensions during the optimization process.
Revised: Nonetheless, the problem ofhandling unbalanced likelihoods leads to the domination of some dimensions during the optimizationprocess.

Original: In [28],authors propose a solution using a set of marginal VAEs that encode each feature into a Gaussianuni-dimensional space, and a dependency VAE that captures the inter-dimensional dependenciesmore effectively using balanced Gaussian likelihoods.
Revised: In [43], authors propose a solution using a set of marginal VAEs that encode each featureinto a Gaussian uni-dimensional space, and a dependency VAE that captures the inter-dimensionaldependencies more effectively using balanced Gaussian likelihoods.

Original: By marginalizing each dimension of the decoder, incomplete data can be easily handled by dividingthe vector x into the observed x O and unobserved x U parts.
Revised: By marginalizing each dimension of the decoder, incomplete data can be easily handled by dividingthe vector x into the observed x O and unobserved x U parts.

Original:  As proposed in [33] and [31], the ELBOobjective is transformed into a lower bound on the observed data, and the unobserved data is replacedwith zeros.All these methods rely on Gaussian approximations to the true posterior and thus operateby optimizing lower bounds on the log evidence.
Revised: This methodology is completely validwhen using the missing-at-random (MAR) assumption [41], i.e. assuming the missing mechanism isindependent of the missing values. In this our work, we use the same assumption. As proposed in[49] and [46], the ELBO objective is transformed into a lower bound on the observed data, and theunobserved data is replaced with zeros.

Original: 2.2 Hierarchical VAEs
Revised: 2.2 Hierarchical VAEs

Original: Hierarchical models have been successfully employed in deep generative modeling, [3, 40, 41].
Revised: Hierarchical models have been successfully employed in deep generative modeling, [3, 60, 61].

Original: In
Revised: In

Original: VAEs, defining a hierarchical latent space for VAEs can be straightforward.
Revised: VAEs, defining a hierarchical latent space for VAEs can be straightforward.

Original: Nevertheless, potentialpitfalls require special attention.
Revised: Nevertheless, potentialpitfalls require special attention.

Original: Concretely, if the decoder is powerful enough, the model tendsto uniquely use the shallowest layers, ignoring the deepest ones and falling into the well-knownproblem of posterior collapse [30, 35, 47].
Revised: Concretely, if the decoder is powerful enough, the model tendsto uniquely use the shallowest layers, ignoring the deepest ones and falling into the well-knownproblem of posterior collapse [45, 54, 72].

Original: In the last few years, several works have investigatedpossible hierarchical structures for VAEs.
Revised: In the last few years, several works have investigatedpossible hierarchical structures for VAEs.

Original: In [43], a bottom-up deterministic path is used along with atop-down inference path that shares parameters with the top-down structure of the generative model.
Revised: In [65], a bottom-up deterministic path is used along with atop-down inference path that shares parameters with the top-down structure of the generative model.

Original: In [30], the authors use a bidirectional stochastic inference path.
Revised: In [45], the authors use a bidirectional stochastic inference path.

Original: More recently, [46] or [9] haveadapted these architectures to complex datasets and high quality images.
Revised: More recently, [71] or [11] haveadapted these architectures to complex datasets and high quality images.

Original: Possibly motivated by theresidual connections in [21], all these works use a shared path between recognition and generativemodels that helps in tying the divergences between approximations and priors in the ELBO.
Revised: Possibly motivated by the residual connections in [32], all these works use a shared path between recognition and generativemodels that helps in tying the divergences between approximations and priors in the ELBO.

Original: 2.3 Hamiltonian Monte Carlo
Revised: 2.3 Hamiltonian Monte Carlo

Original: HMC [5, 13, 34] is a particularly effective MCMC algorithm for sampling from a target distributionp ( z )
Revised: HMC [5, 17, 51] is a particularly effective MCMC algorithm for sampling from a target distributionp ( z )

Original: =
Revised: =

Original: 1 Z p ∗ ( z ) where Z is the normalization constant and z is a d -dimensional vector.
Revised: 1 Z p ∗ ( z ) where Z is the normalization constant and z is a d -dimensional vector.

Original: Byaugmenting this model to p ( r , z ) =
Revised: Byaugmenting this model to p ( r , z ) =

Original: N ( r ; 0 , M ) p ( z ) , and denoting r as the momentum variablewith diagonal covariance matrix M , with the same dimensionality as z , HMC samples from thedistribution by simulating the time-evolution of a fictitious physical system.
Revised: N ( r ; 0 , M ) p ( z ) , and denoting r as the momentum variablewith diagonal covariance matrix M , with the same dimensionality as z , HMC samples from thedistribution by simulating the time-evolution of a fictitious physical system.

Original: The algorithm starts by firstly sampling z and r from an initial proposal and the momentum distribution, respectively.
Revised: The algorithm starts by firstly sampling z and r from an initial proposal and the momentum distribution, respectively.

Original: Chains with length T are built by recurrently proposing and accepting newstates.
Revised: Chains with length T are built by recurrently proposing and accepting newstates.

Original: To propose a new state, the Hamiltonian dynamics are simulated using a symplectic integrator,
Revised: To propose a new state, the Hamiltonian dynamics are simulated using a symplectic integrator,

Original: Leapfrog being the most common choice.
Revised: Leapfrog being the most common choice.

Original: Thethree following updates are repeated for L steps:
Revised: The following updates are repeated for l = 1 : LF steps:

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: ) where ⊙ refers to the Hadamard product, and ϕ is the step size hyperparameter.
Revised: (1) where ⊙ refers to the Hadamard product, and ϕ is the step size hyperparameter.

Original: Although it istypically defined as a scalar for simplicity, a d -dimensional vector can be considered to applydifferent step sizes per dimension, or further, as considered in this work, a T × d matrix to applydifferent steps per each proposal of the chain.
Revised: Although it istypically defined as a scalar for simplicity, a d -dimensional vector can be considered to applydifferent step sizes per dimension, or further, as considered in this work, a T × d matrix to applydifferent steps per each proposal of the chain.

Original: The new proposal ( z ′ , r ′ ) is accepted with probabilitymin [1 , exp( − H ( z ′ , r ′ )
Revised: The new proposal ( z ′ , r ′ ) is accepted with probabilitymin [1 , exp( − H ( z ′ , r ′ )

Original: + H ( z , r ))] , where
Revised: + H ( z , r ))] , where

Original: [Equation]
Revised: [Equation]

Original: For the consecutive T proposals, a new momentum r is resampled and the updates of eq.
Revised: For the consecutive T proposals, a new momentum r is resampled and the updates of eq.

Original: (2) arerepeated for L steps to update the state if ( z ′ , r ′ ) is accepted.
Revised: (1) arerepeated for LF steps to update the state if ( z ′ , r ′ ) is accepted.

Original: 2.4 Active Feature Acquisition
Revised: 2.4 Active Feature Acquisition

Original: Among all the Active Learning techniques, Active Feature Acquisition [20, 32, 39, 44] is of specialinterest in cost-sensitive applications for modeling a trade-off between the improvement of predictionsand the cost of acquiring new data at the feature level.
Revised: Among all the Active Learning techniques, Active Feature Acquisition [29, 47, 59, 67] is of specialinterest in cost-sensitive applications for modeling a trade-off between the improvement of predictionsand the cost of acquiring new data at the feature level.

Original: Several works in the recent literature havestudied methods for performing the task of sequentially acquiring high-value information by selectingfeatures that maximize an information theoretical reward function and enhance the accuracy of thepredictions.
Revised: Several works in the recent literature havestudied methods for performing the task of sequentially acquiring high-value information by selectingfeatures that maximize an information theoretical reward function and enhance the accuracy of thepredictions.

Original: This task is denoted by SAIA (Sequential Active Information Acquisition).
Revised: This task is denoted by SAIA (Sequential Active Information Acquisition).

Original: In [29], itis proposed an efficient method for approximating a non-tractable reward by using the encoder of a
Revised: In [44], itis proposed an efficient method for approximating a non-tractable reward by using the encoder of a

Original: VAE that handles missing data.
Revised: VAE that handles missing data.

Original: In [28] they extend this method for handling mixed-type data.
Revised: In [43] they extend this method for handling mixed-type data.

Original: Bothworks computing the reward by relying on Gaussian approximations given by the encoder networks.
Revised: Bothworks computing the reward by relying on Gaussian approximations given by the encoder networks.

Original:  Hamiltonian hierarchical VAE for mixed-type incomplete data (HH-VAEM)
Revised:  Hamiltonian hierarchical VAE for mixed-type incomplete data (HH-VAEM)

Original: The HH-VAEM model (Figure 1 (a)) is a Hierarchical VAE for mixed-type, incomplete data thatincorporates HMC with automatic hyper-parameter optimization for sampling from the posteriorof the latent variables.
Revised: The HH-VAEM model (Figure 1 (a)) is a Hierarchical VAE for mixed-type, incomplete data thatincorporates HMC with automatic hyper-parameter optimization for sampling from the posteriorof the latent variables.

Original: In a first stage, the mixed-type data is encoded into marginal Gaussianposterior distributions as given by univariate VAEs fitted to each data dimension.
Revised: In a first stage, the mixed-type data is encoded into marginal Gaussianposterior distributions as given by univariate VAEs fitted to each data dimension.

Original: In a second stage, ahierarchical structure captures the dependencies among the standarized, homogeneous dimensionswith well-balanced Gaussian likelihoods.
Revised: In a second stage, ahierarchical structure captures the dependencies among the standarized, homogeneous dimensionswith well-balanced Gaussian likelihoods.

Original: The model is trained using samples from the posterior of thehierarchical latent variables by means of HMC, whilst the HMC hyper-parameters are automaticallytuned.
Revised: The model is trained using samples from the posterior of thehierarchical latent variables by means of HMC, whilst the HMC hyper-parameters are automaticallytuned.

Original: A more detailed description is provided in the following subsections.
Revised: A more detailed description is provided in the following subsections.

Original: [Figure]

Original: 3.1 Notation
Revised: 3.1 Notation

Original: The model generates both data x ∈ R D and output y ∈ R P , where each of these variables is dividedinto observed parts x O , y O and unobserved parts x U , y U .
Revised: The model generates both data x ∈ R D and output y ∈ R P , where each of these variables is dividedinto observed parts x O , y O and unobserved parts x U , y U .

Original: Each dimension of the input data isdenoted by x d . 
Revised: Each dimension of x is denoted by x d . A

Original: A training dataset is composed of N observations as tuples ( x ( n ) O , y ( n ) O ) .
Revised: [Figure] training set is composed of N observations as tuples ( x ( n ) O , y ( n ) O ) .

Original: For ease ofnotation, we omit the observation index n , and the objectives are presented for a single observation.
Revised: For ease of notation, omit theobservation index n , and the objectives are presented for a single observation.

Original: The dependencylatent space is composed of L latent variables[ ϵ1 , ..., ϵ L] , with ϵ l ∈ R m l . The totaldimension of the joint latent distribution is (cid:80) l m l = M . In the marginal VAEs, the latent variablesz d are unidimensional.
Revised: dependency variables ϵ ϵ L

Original: 3.2 Handling heterogeneous incomplete data
Revised: 3.2 Handling heterogeneous incomplete data

Original: Following the strategy proposed by [28], we perform a two-staged approach for handling heterogeneous data.
Revised: Following the strategy proposed by [43], we perform a two-staged approach for handling heterogeneous data.

Original: The marginal distribution of each feature p θ d ( x d ) is modeled by a one-dimensional
Revised: The marginal distribution of each feature p θ d ( x d ) is modeled by a one-dimensional

Original: VAE.
Revised: VAE.

Original: First, the D marginal VAEs are trained independently by maximizing the marginal ELBO overobserved points
Revised: First, the D marginal VAEs are trained independently by maximizing the marginal ELBO overobserved points

Original: [Equation]
Revised: [Equation]

Original:  where 1 ( x d ∈ x O ) is an indicator function that activates the ELBO when the feature is observed.
Revised: [Equation] where 1 ( x d ∈ x O ) is an indicator function that activates the ELBO when the feature is observed.

Revised: Under the missing-at-random assumption, which the one considered in this work, Equation (3) leadsto a lower bound on observed data likelihoods.

Original: Second, a dependency VAE encodes a vector z with the concatenated samples from the marginalposteriors q γ d ( z d | x d ) into the global latent variable h , using zero-filling for the unobserved variables.
Revised: Second, a dependency VAE encodes a vector z withthe concatenated samples from the marginal posteriors q γ d ( z d | x d ) into the global latent variable h ,using zero-filling for the unobserved variables.

Original: By using this approach, z is now homogeneous and can then be easily modeled using a standard Gaussian decoder.
Revised: By using this approach, z is now homogeneous andcan then be easily modeled using a standard Gaussian decoder.

Original: By contrast, other works [14, 33] directly operate with different decoding likelihoods per data type.
Revised: By contrast, other works [18, 49]directly operate with different decoding likelihoods per data type.

Original: These often have very different magnitudes in the ELBO and may reducelearning efficiency.
Revised: These often have very differentmagnitudes in the ELBO and may reduce learning efficiency.

Original: The ELBO for the second stage dependency VAE is
Revised: The ELBO for the second stagedependency VAE is

Original: [Equation]
Revised: [Equation]

Revised: where ϵ = { ϵ 1 , ..., ϵ L } is a set of reparameterized hierarchical latent variables.

Revised: Further details on thedesign of the hierarchical dependency VAE are provided below.

Original: 3.3 Predictive enhancement
Revised: 3.3 Predictive enhancement

Revised: The combination of generative and discriminative models is an effective well-studied strategy fordealing with predictive models under missing data [21, 70].

Revised: In [21], they model p ( x ) for imputingmissing data using a Gaussian Mixture Model.

Revised: In a deep learning context, recent supervised VAEmodels have revisited this combination [30, 63] or used factorisations of type p ( z ) p ( x | z ) p ( y | z )[31] to learn meaningful representations.

Revised: In [39] the authors propose a deep generative model withfactorisation p ( z ) p ( x | z ) p ( y | x , z ) for detecting adversarial attacks.

Original: With the aim at reinforcing the prediction of the variable of interest, we turn into a supervised modelby including a separate predictor for p θ y ( y | ˆ x , h ) , apart from the decoder p θ
Revised: With the aim at reinforcing the prediction of the variable of interest, we turn into a supervisedmodel by including a separate predictor for p θ y ( y | ˆ x , h ) , apart from the decoder p θ

Original: z
Revised: z

Original: ( z | h ) .
Revised: ( z | h ) .

Original: The notation ˆ xrefers to the whole observational vector, including the  imputation of the missing part by decoding thelatent h into z  and each marginal zd into ˆ x u .
Revised: Thevector ˆ x = ( x i ∈ x O , ˆ x j ∈ x U ) includes the observed part and imputation of the missing part ˆ x j bydecoding the latent h into z using p ( z | h 1 ) , and each dimension z j ∈ z into ˆ x j using p ( x j | z j ) .

Original: The predictor parameters θ y are optimized along withthe decoder parameters θ z .
Revised: Thepredictor parameters θ y are optimized along with the decoder parameters θ z .

Original: 3.4 Hierarchical reparameterized latent space
Revised: 3.4 Hierarchical reparameterized latent space

Original: A hierarchical structure over the latent space
Revised: A hierarchical structure over the latent space

Original: h
Revised: h

Original: = { h 1 , ..., h L } enriches the prior assumptions andpermits a flexible generation of data in a more natural fashion.
Revised: = { h 1 , ..., h L } enriches the prior assumptions andpermits a flexible generation of data in a more natural fashion.

Original: Nevertheless, Hierarchical VAEsrequire special care with their optimization.
Revised: Nevertheless, as stated in [5, 6], HMCcan be pathological when used for sampling from hierarchical densities, where the magnitude ofautoregressive variations increase with the depth. For approximating the Hamiltonian dynamics,inside each Leapfrog integrator step, gradients ∇ h 1: L log p ∗ ( h 1: L ) are required. Due to the strong 165 curvature regions, huge norm of high-order derivatives are backpropagated and might eventuallyexplode, ending in overflow issues (Fig. 35 in [6]). If we were to run our HMC method over the hierarchical variables without any reparameterization(Fig. 4a), by the time the states reached the aforementioned problematic regions, the integratorwould diverge and we would experience the aforementioned overflow problems.

Original: Most of the recent works employ a deterministic topdown path, shared between the inference and generative models, that helps in tying the prior andtheposterior approximate distributions during the optimization. On the contrary, in our model, the latentvariables ϵ l are disconnected from each other and the hierarchical structure present in  h l is modeledby deterministic networks, thereby this shared path is no longer needed.
Revised: By rejecting theseproblematic states, chains would get stuck close to the proposal and the hierarchical density wouldnot be properly explored (concluding that HMC would not improve the Gaussian proposal). To givean example, in [71], the AR path p ( h l | h <l ) would provoke instabilities inside the HMC integratordue to huge gradients in ∇ h 1: L log p ∗ ( h 1: L ) . This is considered as a pathological design and thus isnot included in the validation experiments.

Revised: We successfully solved this issue by introducing a hierarchical reparameterization technique.

Original: Thus, the representation ateach layer is a reparametrization from variable ϵ l with standard Gaussian prior p ( ϵ l )
Revised: Therepresentation at each layer is reparameterized from variable ϵ l with standard Gaussian prior p ( ϵ l )

Original: [Equation]
Revised: [Equation]

Original: (6) where the functions f µ l ( h l +1 ) and f σ l ( h l +1 ) are applied by NNs with parameters θ l = { θ µ l , θ σ l } .
Revised: where the functions f µ l ( h l +1 ) and f σ l ( h l +1 ) are applied by NNs with parameters θ l = { θ µ l , θ σ l } .

Original: The result is equivalent as learning the mean and covariance of autoregressive variables (see Figure 4for illustrative details).
Revised: The result is equivalent as learning the mean and covariance of autoregressive variables (see Figure 4for illustrative details).

Original: However, by choosing this reparameterization, we relax the dependenciesamong the latent variables, resulting in a smoother joint posterior density p ( ϵ | x
Revised: However, thanks to this trick, we relax the dependencies among the latentvariables, resulting in a smoother joint posterior density p ( ϵ | x

Original: O , y O ) .
Revised: O , y O ) .

Original: Performingthe inference over ϵ = { ϵ 1 , ..., ϵ L } leads to a better posed basis for running HMC, as detailedinsection 3.5. To give an example, in [46], the AR path p ( z l | z <l ) would provoke instabilities inside the 
Revised: Performing the inference overϵ = { ϵ 1 , ..., ϵ L } leads to a better posed basis for running our HMC optimization, detailed in section3.5, and avoids the necessity of employing more advanced HMC samplers like [6, 22].

Original: HMC integrator due to huge gradients in ∇ z 1: L log p ∗ ( z 1: L ) (Betancourt et at., 2015). We solve thisby disconnecting the latent variables ( p ( ϵ l ) are standard Gaussians).
Revised: Providedthe promising results we obtain in Section 5, we propose our reparameterization trick as a novelcontribution for solving the pathological behavior of HMC combined with hierarchical VAEs.

Original: For the sake of simplicity, wename the generative parameters as θ = { θ z , θ y , θ 1 , ..., θ L } .
Revised: For the sake of simplicity, we name the generative parameters as θ = { θ z , θ y , θ 1 , ..., θ L } .

Original: In our work, a deterministic bottom-up path is built from the observational space to each latentlayer for encoding the information.
Revised: Thedependency ELBO under this hierarchical reparameterized model becomes [Equation]

Original: Naming r l the hidden representation at each layer, and definingr 0{+defining r 
+} = { x O , y O } , we employ NNs with parameters ψ r l for computing r l = f r ( r l − 1 ) .
Revised: where we name r l the hidden representation at each layer, and {+defining r 
+} = { x O , y O } , we employ NNs with parameters ψ r l for computing r l = f r ( r l − 1 ) .

Original: These vectorsare mapped into the parameters of the variational posterior q ψ l ( ϵ l | x O , y O ) , using NNs for computingthe mean as g µ l ( r l ) and the covariance as g σ l ( r l ) , with parameters ψ µ l and ψ σ l .
Revised: These vectors are mapped into the parametersof the variational posterior q ψ l ( ϵ l | x O , y O ) , using NNs for computing the mean as g µ l ( r l ) and thecovariance as g σ l ( r l ) , with parameters ψ µ l and ψ σ l .

Original: With compactnesspurposes, we will denote the encoder parameters as ψ = { ψ 1 , ..., ψ L } , with ψ l = { ψ r l , ψ µ l , ψ σ l } .
Revised: With compactness purposes, we will denote theencoder parameters as ψ = { ψ 1 , ..., ψ L } , with ψ l = { ψ r l , ψ µ l , ψ σ l } .

Original: The dependency ELBO under this hierarchical reparameterized model becomes

Original: [Equation]

Original: 3.5 HMC over the hierarchical density
Revised: 3.5 HMC over the hierarchical density

Original: In recent works, HMC has been combined with deep generative models for improving the inference ofthe latent variables by obtaining better samples from the posterior [8, 19].
Revised: In recent works, HMC has been combined with deep generative models for improving the inferenceof the latent variables by obtaining better samples from the posterior [10, 28].

Original: In this work, we propose totranscend these previous approaches and build a generalized method for sampling from complicated,hierarchical latent structures composed by several layers.
Revised: In this work, wepropose to transcend these previous approaches and build a generalized method for sampling fromcomplicated, hierarchical latent structures composed by several layers.

Original: Inspired by [7] and their method forsampling from the posterior within a vanilla VAE framework while tuning the HMC hyperparameters,we follow a procedure for training the dependency model where i) during a pre-training stage, theencoder and decoder are optimized using standard VI and the ELBO from equation (7), and ii) usingthe pre-trained encoder for starting from a good proposal [19], HMC samples are obtained to followthe true posterior and jointly optimize the generative model and the HMC hyperparameters.
Revised: Inspired by [9] and theirmethod for sampling from the posterior within a vanilla VAE framework while tuning the HMC hyperparameters, we follow a procedure for training the dependency model where i) during a pretraining stage, the encoder and decoder are optimized using standard VI and the ELBO from equation(6), and ii) using the pre-trained encoder for starting from a good proposal [28], HMC samplesare obtained to follow the true posterior and jointly optimize the generative model and the HMChyperparameters.

Original: In Figure1 (b) we include an illustrative example.
Revised: In Figure 1 (b) we include an illustrative example.

Original: We denote by q ( T ) ϕ
Revised: We denote by q ( T ) ϕ

Original: ( ϵ |
Revised: ( ϵ |

Original: z O , x O , y O )
Revised: z O , x O , y O )

Original: the implicit distribution for the posterior after T HMC steps.
Revised: the implicit distribution for the posterior after T HMC steps.

Original: Thehyper-parameters of HMC are named ϕ , a T × d matrix containing the step sizes for each dimensionat each step of the Leapfrog integration.
Revised: Thehyper-parameters of HMC are named ϕ , a T × d matrix containing the step sizes for each dimensionat each step of the Leapfrog integration.

Original: As exposed by [7], optimizing the momentum variancesdoes no improve performance.
Revised: As exposed by [9], optimizing the momentum variancesdoes not improve performance.

Original: Within this perspective, the hyper-parameters can be optimized usingvariational inference by maximizing the ELBO
Revised: Within this perspective, the hyper-parameters can be optimized usingvariational inference by maximizing the ELBO

Original: [Equation]
Revised: [Equation]

Original: where the first part is the HMC objective, and can be easily estimated via Monte Carlo
Revised: where the first part is the HMC objective, and can be easily estimated via Monte Carlo

Original: [Equation]
Revised: [Equation]

Original:  Nevertheless, the entropy term H (cid:104) q ( T ) ϕ
Revised: L [Equation] Nevertheless, the entropy term H [ q ( T ) ϕ

Original: ( ϵ | x O , y O ) (cid:105) in Equation (8) is not tractable since we are notable to explicitly evaluate the distribution q ( T ) ϕ
Revised: ( ϵ | x O , y O )] in Equation (7) is not tractable since we are notable to explicitly evaluate the distribution q ( T ) ϕ

Original: ( ϵ | x O , y O ) .
Revised: ( ϵ | x O , y O ) .

Original: Although optimizing the first term mightresult in a well-posed algorithm, this would bring consequences that must be considered.
Revised: Although optimizing the first term mightresult in a well-posed algorithm, this would bring consequences that must be considered.

Original: Namely,without a proper regularization term, and in case the initial proposal q ( T ) ϕ
Revised: Namely,without a proper regularization term, and in case the initial proposal q ( T ) ϕ

Original: ( ϵ | x O , y O ) is concentratedin high density regions, the chains would barely move from the initial state and only these regionswith high density would be explored.
Revised: ( ϵ | x O , y O ) is concentratedin high density regions, the chains would barely move from the initial state and only these regionswith high density would be explored.

Original: To cope with this problem, we define an inflation parameters to increase the variance of the proposal q ϕ
Revised: To cope with this problem, we define an inflation parameters to increase the variance of the proposal q ϕ

Original: ( ϵ |
Revised: ( ϵ |

Original: z O , x O , y O ) given by the Gaussian encoder.
Revised: z O , x O , y O ) given by the Gaussian encoder, endingin the proposal (cid:81)

Original: Whilstin [7] the authors define this parameter as a  scalar factor applied to all the latent dimensions, in ourwork, we extend this to apply a different inflation factor at each latent level s = { s 1 , ..., s L } .
Revised: Ll =1 N ( g µ l ( r l ) , s l · g σ l ( r l )) . Whilst in [9] the authors define this parameter as a 217 scalar factor applied to all the latent dimensions, in our work, we extend this to apply a differentinflation factor at each latent level s = { s 1 , ..., s L } .

Original: In orderto tune these inflation parameters, we minimize the SKSD discrepancy measure [16]
Revised: In order to tune these inflations we ensure awider coverage of the density by minimizing the Sliced Kernelized Stein Discrepancy (SKSD) [23]

Original: [Equation]
Revised: [Equation]

Original: which fits perfectly our requirements, since only requires samples from HMC and
Revised: which fits perfectly our requirements, since only requires samples from HMC and

Original: gradients∇ ϵ log p ( z O , y O , ϵ ) for measuring a discrepancy between the implicit and the true posterior.
Revised: gradients∇ ϵ log p ( z O , y O , ϵ ) for measuring a discrepancy between the implicit and the true posterior.

Original: Further,the SKSD performs better than other discrepancies like [27] in high dimensional latent spaces.
Revised: Further,the SKSD performs better than other discrepancies like [42] in high dimensional latent spaces.

Original: Input: data (cid:16) x (1: N ) O , y (1: N ) O (cid:17) , steps: T d , T V I , T HMC Parameters: γ , θ , ψ , ϕ , s S TAGE 1: MARGINAL VAE S for d = 1 to D do
Revised: We provide in Section A.4 a toy demonstration on the efficacy of the HMC optimization.

Original: Initialize marginal VAE { θ d , γ d } for t = 1 to T d do γ t +1 d , θ t +1 d ← Adam γ td ,θ td

Original: ( L d ) end for end for

Original: S TAGE 2: DEPENDENCY VAE for t = 1 to T V AE do θ t +1 , ψ t +1 ← Adam θ t ,ψ t ( L V I ) end for

Original: S TAGE 3: J OINTLY OPTIMIZING VAE + HMC for t = 1 to T HMC do ψ t +1 ← Adam ψ t ( L V I ) θ t +1 , ϕ t +1 ← Adam θ t ,ϕ t ( L HMC ) s t +1 ← Adam s t ( L SKSD ) end for

Original: Despite of the robustness and successof HMC methods, it remains a wellknown issue that hierarchical objectives lead to computational issues inthe integrator [5].

Original: The autoregressivedependencies among variables createregions of high curvature in the typical set of the joint density that canbe pathological for symplectic integrators.

Original: These type of issues can besolved by manipulating the HMC algorithm by means of a parameterisedcovariance for the momentum distribution [6].

Original: Nonetheless, by using ourreparameterized version, we are ableto relax these dependencies and maintain the standard HMC method forobtaining samples that approximatelyfollow the true posterior.

Original: 3.6 HH-VAEM optimization
Revised: 3.6 HH-VAEM optimization

Original: The optimization of the HH-VAEMis divided into three stages.
Revised: The optimization of the HH-VAEM is divided into three stages.

Original: In a firststage, we train one independent marginal VAE per dimension.
Revised: In a first stage, we train oneindependent marginal VAE per dimension.

Original: In a second stage, the dependency VAE is trained, using as inputs the concatenation of the encoded dimensions z
Revised: In a second stage, the dependency VAE is trained, usingas inputs the concatenation of the encoded dimensions z

Original: = { z 1 , ..., z D } and thetarget y .
Revised: = { z 1 , ..., z D } and the target y .

Original: Finally, in a third stage, the HMC hyperparameters, the decoder and the predictor are tunedusing the HMC objective, the inflation parameter is trained using the SKSD discrepancy, and theencoder is trained used the ELBO.
Revised: Finally,in a third stage, the HMC hyperparameters, the decoder and the predictor are tuned using the HMCobjective, the inflation parameter is trained using the SKSD discrepancy, and the encoder is trainedused the ELBO.

Original: The pseudocode for HH-VAEM training is shown in Algorithm 1.
Revised: The pseudocode for HH-VAEM training is shown in Algorithm 1.

Original: 3.7 Computational cost
Revised: 3.7 Computational cost

Original: In a VAE, the data is fed to the encoder with aim at obtaining the variational parameters for samplingfrom the Gaussian approximated posterior.
Revised: In a VAE, the data is fed to the encoder with aim at obtaining the variational parameters for samplingfrom the Gaussian approximated posterior.

Original: In our method, the data is similarly encoded to obtain theinitial Gaussian proposal q (0) , and the samples from this distribution are updated for T cycles to obtainthe implicit q ( T ) using HMC.
Revised: In our method, the data is similarly encoded to obtain theinitial Gaussian proposal q (0) , and the samples from this distribution are updated for T cycles to obtainthe implicit q ( T ) using HMC.

Original: Within each of these iterations, L leapfrog steps (2) are executed.
Revised: Within each of these iterations, L leapfrog steps (1) are executed.

Original: Foreach of these steps, the computation of the gradients ∇ ϵ l log p ( z , y , ϵ l ) and ∇ ϵ l +1 log p ( z , y , ϵ l +1 ) isrequired.
Revised: Foreach of these steps, the computation of the gradients ∇ ϵ l log p ( z , y , ϵ l ) and ∇ ϵ l +1 log p ( z , y , ϵ l +1 ) is 237 required.

Original: To obtain these gradients, we need to i) compute the parameters of the likelihood p ( z , y | ϵ )that are given by the decoder ( p ( z | h 1 ) ) and predictor ( p ( y | ˆ x , h 1 ) ), ii) evaluate it and iii) perform theautomatic differentiation.
Revised: To obtain these gradients, we need to i) compute the parameters of the likelihood p ( z , y | ϵ ) 238 that are given by the decoder ( p ( z | h 1 ) ) and predictor ( p ( y | ˆ x , h 1 ) ), ii) evaluate it and iii) perform theautomatic differentiation.

Original: Thus, for running our method, an additional cost from both decoding andperforming differentiation a total of 2 TL times is introduced.
Revised: Thus, for running our method, an additional cost from both decoding andperforming differentiation a total of 2 TL times is introduced.

Revised: Input: data (cid:16) x (1: N ) O , y (1: N ) O (cid:17) , steps: T d , T V

Original: By jointly optimizing the HMC hyperparameters we are able toachieve faster convergence of thechains with smaller lengths ( T ), as demonstrated in [7].
Revised: I , T HMC Parameters: γ , θ , ψ , ϕ , s S TAGE 1: MARGINAL VAE S for d = 1 to D do Initialize marginal VAE { θ d , γ d } for t = 1 to T d do γ t +1 d , θ t +1 d ← Adam γ td ,θ td

Revised: ( L d ) end for end for

Original: Further, to  reduce the computational costas much as possible, we optimizethe HMC hyperparametersonly for a few steps in a final trainingstage, since convergence is rapidly achieved (as observed empirically in Fig. 6).
Revised: S TAGE 2: DEPENDENCY VAE for t = 1 to T V AE do θ t +1 , ψ t +1 ← Adam θ t ,ψ t ( L V I ) end for S TAGE 3: J OINTLY OPTIMIZING VAE + HMC for t = 1 to T HMC do ψ t +1 ← Adam ψ t ( L V I ) θ t +1 , ϕ t +1 ← Adam θ t ,ϕ t ( L HMC ) s t +1 ← Adam s t ( L SKSD ) end for By jointly optimizing the HMChyperparameters we are able toachieve faster convergence withsmaller lengths. To reduce the computational cost, we optimize hyperparameters in a final training stage, sinceconvergence is rapidly achieved (asobserved empirically in Fig. 7).

Original: At test time, samplesfrom the Gaussianproposal q (0) (faster and cheaper), or from HMC q ( T ) (slower and better) can beused to fit computational constraints.
Revised: Attest, samples from the Gaussian q (0)(faster and cheaper), or from HMCq ( T ) (slower and better) can be usedto fit computational constraints.

Original:  Sampling-based Active Learning
Revised:  Sampling-basedActive Learning

Original: Considering that the input data are tuples of observed and missing features { x O , x U } , our Active Learning  framework follows [28, 29] and determines which feature x i ∈ x U will  enhance theprediction of the target y the most for a particular x O .
Revised: Considering that the input data aretuples of observed and missing features { x O , x U } , our Active Learning 258 framework follows [43, 44] and determines which feature x i ∈ x U will 260 enhance the prediction of the target ythe most for a particular x O .

Original: Concretely, in a Sequential Active Information Acquisition (SAIA) scenario, this decision is taken sequentially to optimally increase knowledgeand accurately predict y .
Revised: Concretely, in a Sequential Active Information Acquisition (SAIA)scenario, this decision is taken sequentially to optimally increase knowledge and accurately predict y .

Original: From a information theoretical perspective, this task can be performedrecurrently by maximizing a reward function R at each step d .
Revised: From a information theoretical perspective, this task can be performed recurrently by maximizing areward function R at each step d .

Original: This reward might represent abstractquantities of interest like the cost or benefit of acquiring x i (depending on the sign).
Revised: This reward might represent abstract quantities of interest like thecost or benefit of acquiring x i (depending on the sign).

Original: In Bayesianexperimental analysis, R is the expected gain of information [26].
Revised: In Bayesian experimental analysis, R is theexpected gain of information [40].

Original: Following [4], we can define it as
Revised: Following [4], we can define it as

Original: [Equation]
Revised: [Equation]

Original: where i is the index of each unobserved feature.
Revised: where i is the index of each unobserved feature.

Original: Intuitively, this reward can be interpreted as theexpected change in the predictive distribution when x i is observed.
Revised: Intuitively, this reward can be interpreted as theexpected change in the predictive distribution when x i is observed.

Original: As one might appreciate, thereward needs to be estimated via Monte Carlo by sampling from p ( x i | x O ) .
Revised: As one might appreciate, thereward needs to be estimated via Monte Carlo by sampling from p ( x i | x O ) .

Original: With a robust generativemodel that handles missing data like HH-VAEM, these samples are easily obtained: first, using HMC,we sample ϵ ( T ) from p ( ϵ | x O ) .
Revised: With a robust generativemodel that handles missing data like HH-VAEM, these samples are easily obtained: first, using HMC,we sample ϵ ( T ) from p ( ϵ | x O ) .

Original: Second, we decode these samples to obtain x i from p ( z | h 1 ) andp ( x i | z d ) .
Revised: Second, we decode these samples to obtain x i from p ( z | h 1 ) andp ( x i | z d ) .

Original: Nonetheless, the reward defined in (11) is intractable since both p ( y | x i ) and p ( y | x i , x O )are intractable: computing them requires to integrate out the latent variables.
Revised: Nonetheless, the reward defined in (10) is intractable since both p ( y | x i ) and p ( y | x i , x O ) 273 are intractable: computing them requires to integrate out the latent variables.

Original: This motivates theauthors of [28, 29] to present a transformation of the reward for being computed in the latent spaceusing the encoder network.
Revised: This motivates theauthors of [43, 44] to present a transformation of the reward for being computed in the latent spaceusing the encoder network.

Original: Although they prove that this transformation effectively provides a goodestimation in several datasets, we demonstrate that for low dimensional targets (commonly with oneor two dimensions), the estimation of the reward with histograms is more effective.
Revised: Although they prove that this transformation effectively provides a goodestimation in several datasets, we demonstrate that for low dimensional targets (commonly with oneor two dimensions), the estimation of the reward with histograms is more effective.

Original: The reward in (11) can be rewritten as
Revised: The reward in (10) can be rewritten as

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: which is exactly the Mutual Information between the target and the missign feature x i , conditionedon observing x O .
Revised: which is exactly the Mutual Information between the target and the missign feature x i , conditionedon observing x O .

Original: While a set of advanced non-parametric estimators of the mutual information areavailable [24, 38], many are not easilly adapted for parallelization.
Revised: While a set of advanced non-parametric estimators of the mutual information areavailable [37, 58], many are not easilly adapted for parallelization.

Original: We demonstrate that the simplestone, based on binning the x i and y domains
Revised: We demonstrate that the simplestone, based on binning the x i and y domains

Original: [Equation]
Revised: [Equation]

Original: is effective and easy to parallelize.
Revised: is effective and easy to parallelize.

Original: In the equation, p x ( i ) = n x ( i ) /N , p y ( j ) =
Revised: In the equation, p x ( i ) = n x ( i ) /N , p y ( j ) =

Original: n y ( j ) /N andp ( i, j ) =
Revised: n y ( j ) /N andp ( i, j ) =

Original: n ( i, j ) /N are the relative frequencies that approximate p ( x ) , p ( y ) and p ( x, y ) for each bin.
Revised: n ( i, j ) /N are the relative frequencies that approximate p ( x ) , p ( y ) and p ( x, y ) for each bin.

Original: Thus, n x ( i ) , n y ( j ) and n ( i, j ) are the number of samples inside each interval.
Revised: Thus, n x ( i ) , n y ( j ) and n ( i, j ) are the number of samples inside each interval.

Original: This biased estimatoris sufficient for discriminating among important features that will enhance predictions.
Revised: The number of binsdefines the resolution of the histograms, i.e. uniformly distributed intervals over x d and y supports. In case the feature or the target is discrete, we use the categories as bins. Since this estimator issampling-based, under certain conditions (namely, if all densities exist as proper functions), Eq. (13)indeed converges to I ( y ; x i | x O ) if we first let the number of samples N → ∞ [37].

Revised: [Table]

Revised: [Table]

Original:  Experiments
Revised:  Experiments

Original: The evaluation of the HH-VAEM model is organized into four experiments.
Revised: The evaluation of the HH-VAEM model is organized into four experiments.

Revised: The ablation studyincludes the validation of including the the proposed components wrt to the Gaussian alternative.

Original: The comparison isperformed with the following baseline models:
Revised: Namely, the comparison is performed with the following baseline models:

Original: • VAEM : The VAEM strategy [28] (without including the Partial VAE).• MIWAEM : VAEM combined with the importance weighted estimation proposed in [31].
Revised: • VAEM : 1 layer, Gaussian-based VAEM [43] (without including the Partial VAE). • MIWAEM : 1 layer, Gaussian-based, importance weighted IWAEM (VAEM + IWAE [46]).

Original: • H-VAEM : A Hierarchical VAEM with two layers of latent variables and a Gaussian encoder.
Revised: • H-VAEM : 2 layers, Gaussian-based VAEM.

Original: • HMC-VAEM : A VAEM that includes a tuned HMC sampler for the true posterior.
Revised: • HMC-VAEM : 1 layer, HMC-based (with our optimization method) VAEM. [Figure] - rameterized (b).

Original: [Table]
Revised: [Table]

Revised: [Table]

Original: [Table]
Revised: [Table]

Original: For all the models, we manually introduce missing data in the training set by randomly setting perdata point a feature as missing with a probability sampled uniformly in the interval [0 . 01 , 0 . 99] withineach batch.
Revised: For all the models, we manually introduce missing data in the training set by randomly setting perdata point a feature as missing with a probability sampled uniformly in the interval [0 . 01 , 0 . 99] withineach batch.

Original: Both the input data x and the target y can be missing.
Revised: Both the input data x and the target y can be missing.

Original: For the test set, a fixed probabilityof 0.5 leads to about half of the input data being observed, whilst the target is completely unobserved.
Revised: For the test set, a fixed probabilityof 0.5 leads to about half of the input data being observed, whilst the target is completely unobserved.

Original: A total of 10 UCI datasets [12] that include mixed-type data are employed for the evaluation, Further,we include both MNIST [25] and Fashion-MNIST [49] datasets for evaluating our model in higherdimensional observational and latent spaces and bigger architectures (3 layered convolutional netsfor encoder/decoder).
Revised: A total of 10 UCI datasets [16] that include mixed-type data are employed for the evaluation, Further,we include both MNIST [38] and Fashion-MNIST [74] datasets for evaluating our model in higherdimensional observational and latent spaces and bigger architectures (3 layered convolutional netsfor encoder/decoder).

Original: For these two special cases, the marginal VAEs are not included and thedependency VAE is fed directly with the Bernoulli-distributed pixels.
Revised: For these two special cases, the marginal VAEs are not included and thedependency VAE is fed directly with the Bernoulli-distributed pixels.

Original: We name this model HH-VAE,and similarly, the baselines are renamed as VAE, MIWAE, HMC-VAE and H-VAE.
Revised: We name this model HH-VAE,and similarly, the baselines are renamed as VAE, MIWAE, HMC-VAE and H-VAE.

Original: Extended detailson the employed architectures and evaluation times are provided in Appendix A.1.[Figure] [Table] [Table]
Revised: Extended detailson the employed architectures and evaluation times are provided in Appendix A.1.

Original: [Table]

Original: 5.1 Mixed type conditional data imputation
Revised: 5.1 Mixed type conditional data imputation

Original: In order to evaluate the performance of the model in terms of data imputation, we opt by computing thenegative log likelihood of the unobserved features.
Revised: In order to evaluate the performance of the model in terms of data imputation, we opt by computing thenegative log likelihood of the unobserved features.

Original: We make use of the Monte Carlo approximation
Revised: We make use of the Monte Carlo approximation

Original: [Equation]
Revised: [Equation]

Revised: [Figure]

Original: which is averaged over features in order to compare the imputation performance with the baselines.
Revised: which is averaged over features in order to compare the imputation performance with the baselines.

Revised: Additionally, we include in Section A.7 similar results separating the considered likelihoods.

Original: Results on the 10 UCI datasets and the MNIST datasets are included in Tables 1 and 3, showing thatboth HH-VAE and HH-VAEM outperform the baselines in almost all the cases. 
Revised: Resultson the 10 UCI datasets and the MNIST datasets are included in Tables 1 and 3, showing that both HH-VAE and HH-VAEM outperform the baselines in almost all the cases. Further, for most of thedatasets, incremental improvement is obtained: VAEM < H-VAEM < HMC-VAEM <

Revised: HH-VAEM.

Original: 5.2 Target prediction
Revised: 5.2 Target prediction

Original: [Figure]

Original: For this experiment, we compute the negative log likelihood of the target under the predictivedistribution using the same procedure as in the previous section.
Revised: For this experiment, we compute the negative log likelihood of the target under the predictivedistribution using the same procedure as in the previous section.

Original: Results included in Tables 2 and 4show that our model again outperforms the baselines in almost all the cases. [Figure]
Revised: Results included in Tables 2 and 4show that our model again outperforms the baselines in almost all the cases, and the same incrementalimprovement is observed in the prediction task.

Original: 5.3 Sequential active information acquisition (SAIA)
Revised: 5.3 Sequential active information acquisition (SAIA)

Original: [Figure]

Original: In this experiment, the HH-VAEM model and our proposed acquisition function are evaluated in a
Revised: In this experiment, the HH-VAEM model and our proposed acquisition function are evaluated in a

Original: SAIA task.
Revised: SAIA task.

Original: We start by predicting from completely unobserved inputs.
Revised: We start by predicting from completely unobserved inputs.

Original: At each step, the missingfeature that maximizes the reward is acquired sequentially.
Revised: At each step, the missingfeature that maximizes the reward is acquired sequentially.

Original: Figure 5 shows the resulting error curvesfor the UCI datasets.
Revised: Figure 5 shows the resulting error curvesfor the UCI datasets.

Original: Blue lines correspond to the transformed reward proposed by [28].
Revised: Blue lines correspond to the transformed reward proposed by [43].

Original: Orange linesare our reward in a VAEM framework (not including the hierarchy and the HMC sampler).
Revised: Orange linesare our reward in a VAEM framework (not including the hierarchy and the HMC sampler).

Original: Greenlines correspond to HH-VAEM with our reward.
Revised: Greenlines correspond to HH-VAEM with our reward.

Original: In most of the cases, our model and acquisitiontechnique outperform the baselines, obtaining lower errors and a faster discovery of information.[Figure] e s t r m [Figure]
Revised: In most of the cases, our model and acquisitiontechnique outperform the baselines, obtaining lower errors and a faster discovery of information.

Original: e

Original: [Figure]

Original: [Figure]

Original: Table 6 shows the average training time in minutes for each model in the experiments for Tables 1and 2.

Original: The ratio between training times for our method and the Gaussian baselines is approximatelybetween 5 and 10.

Original: [Table]

Original:  Conclusion
Revised:  Conclusion

Original: We presented HH-VAEM, to our knowledge, the first hierarchical VAE for mixed-type incompletedata that uses HMC with automatic hyper-parameter tuning for improved inference.
Revised: We presented HH-VAEM, to our knowledge, the first hierarchical VAE for mixed-type incompletedata that uses HMC with automatic hyper-parameter tuning for improved inference.

Original: We provideexperiments that demonstrate its superiority with respect the baselines in the tasks of missing dataimputation and supervised learning, placing HH-VAEM as a robust model for real-world datasets.
Revised: We provideexperiments that demonstrate its superiority with respect the baselines in the tasks of missing dataimputation and supervised learning, placing HH-VAEM as a robust model for real-world datasets.

Original: Further, we have developed a novel sampling-based technique for dynamic feature selection thatoutperforms the Gaussian-based alternatives and results in an efficient method for active learning indeep generative models.
Revised: Further, we have developed a novel sampling-based technique for dynamic feature selection thatoutperforms the Gaussian-based alternatives and results in an efficient method for active learning indeep generative models.

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?

Original: [Yes]
Revised: [Yes](

Original: (b) Did you describe the limitations of your work?
Revised: b ) Did you describe the limitations of your work

Original: [Yes] See Section 3.7 about the computational cost.
Revised: Yes ] See Section 3.7 about thecomputational cost

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: c ) Did you discuss any potential negative societal impacts of your work

Original: [No] We donot
Revised: [No We do

Original: believe there could be any potential negative societal impacts. 
Revised: notbelieve there could be any potential negative societal impacts. d ) Have you

Original: (d) Have you read the ethics review guidelines and ensured that yourpaper conforms to
Revised: read the ethics review guidelines and ensured that your

Revised: paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: a ) Did you state the full set of assumptions of all theoretical results

Original: [N/A]
Revised: [N/A]

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b ) Did you include complete proofs of all theoretical results

Original: [N/A]
Revised: [N/A]

Original: If you ran experiments...
Revised: If you ran experiments

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] We include thecode for reproducing all the presented experiments in the supplemental material.
Revised: [Yes] We include thecode for reproducing all the presented experiments in the supplemental material.

Original: (b) Did you specify all the training details (e g., data splits, hyperparameters,
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters,

Original: how they
Revised: how they

Original: were chosen)?
Revised: were chosen)?

Original: [Yes] See Appendix A.1 with all the training details.
Revised: [Yes] See Appendix A.1 with all the training details.

Original: (c) Did you report error bars (e
Revised: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experi-
Revised: g., with respect to the random seed after running experi-

Original: ments multiple times)?
Revised: ments multiple times)?

Original: [Yes] We include error intervals in all our experiments.
Revised: [Yes] We include error intervals in all our experiments.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Appendix A.1 with all thetraining details.
Revised: [Yes] See Appendix A.1 with all the training details.

Original: If you are using existing assets (e
Revised: If you are using existing assets (e

Original: g.,

Original: code , data, models ) or curating/releasing new assets..
Revised: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] We have cited every
Revised: [Yes] We have cited every

Original: dataset/model employed or mentioned in our work.
Revised: dataset/model employed or mentioned in our work.

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes]
Revised: [Yes]

Original: We include the code for reproducing all the presented experiments in the supplemental material.
Revised: We include the code for reproducing all the presented experiments in the supplementalmaterial.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [N/A
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/ curating contains personally identifiableinformation or offensive content
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A](
Revised: [N/A]

Original: b) Did you describe any potential participant risks with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals , if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/ A](
Revised: [N/A]

Original: c) Did you include the estimated hourly wage paid to participants and the total amountspent
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original:  on participant compensation
Revised: spent on participant compensation?

Original: [N/ A]
Revised: [N/A]

Original: A AppendixA.1 Experimental setup
Revised: A AppendixA.1 Experimental setup

Original: The networks for the encoder of the model with the MNIST datasets are 2 layered Deep CNNs with{16, 32, 32} output channels, kernel size 5, stride 2 and padding 2.
Revised: The networks for the encoder of the model with the MNIST datasets are 2 layered Deep CNNs with{16, 32, 32} output channels, kernel size 5, stride 2 and padding 2.

Original: They are followed by MLPsfor obtaining the variational parameters for each layer.
Revised: They are followed by MLPsfor obtaining the variational parameters for each layer.

Original: The decoder that obtains p θ ( x | h 1 ) is thesymmetric convolutional network.
Revised: The decoder that obtains p θ ( x | h 1 ) is thesymmetric convolutional network.

Original: All the neural networks employed in the models trained with
Revised: All the neural networks employed in the models trained with

Original: UCI datasets are one single layer MLPs with 256 hidden units.
Revised: UCI datasets are one single layer MLPs with 256 hidden units.

Original: The noise variance for Gaussianlikelihoods is set up to 0 . 1 .
Revised: The noise variance for Gaussianlikelihoods is set up to 0 . 1 .

Original: We employ a learning rate of 1 × 10 − 3 for all the parameters except for the inflation parameter s ,where we increase to 1 × 10 − 2 for a faster convergence.
Revised: We employ a learning rate of 1 × 10 − 3 for all the parameters except for the inflation parameter s ,where we increase to 1 × 10 − 2 for a faster convergence.

Original: A batch size of 100 is used for all the modelsexcept for Yatch and Wine dataset, where we use 20 samples per batch.
Revised: A batch size of 100 is used for all the modelsexcept for Yatch and Wine dataset, where we use 20 samples per batch.

Original: The number of training stepsis 20 × 10 3 for Boston, Energy, Wine, Yatch, Concrete, Diabetes and Yatch, and 50 × 10 3 for Naval,
Revised: The number of training stepsis 20 × 10 3 for Boston, Energy, Wine, Yatch, Concrete, Diabetes and Yatch, and 50 × 10 3 for Naval,

Original: Avocado, Bank, and Insurance.
Revised: Avocado, Bank, and Insurance.

Original: For MNIST and Fashion-MNIST, we have 100 × 10 3 training steps.
Revised: For MNIST and Fashion-MNIST, we have 100 × 10 3 training steps.

Original: For the marginal VAEs stage, we employ 1 × 10 3 training steps.
Revised: For the marginal VAEs stage, we employ 1 × 10 3 training steps.

Original: The dimension of the latent variablesis [ d 1 = 10 , d 2 = 5] for Boston, Energy, Wine, Naval Avocado, Bank and Insurance [ d 1 = 4 , d 2 = 2]for Concrete, Yatch and Diabetes and [ d 1 = 20 , d 2 = 10] for MNIST and Fashion-MNIST.
Revised: The dimension of the latent variablesis [ d 1 = 10 , d 2 = 5] for Boston, Energy, Wine, Naval Avocado, Bank and Insurance [ d 1 = 4 , d 2 = 2]for Concrete, Yatch and Diabetes and [ d 1 = 20 , d 2 = 10] for MNIST and Fashion-MNIST.

Original: We use L = 5 Leapfrog steps, chains of T = 10 for Boston, Energy, Wine, Naval Avocado, Bank,
Revised: We use LF = 5 Leapfrog steps, chains of T = 10 for Boston, Energy, Wine, Naval Avocado, Bank,

Original: Insurance, MNIST and Fashion-MNIST, and T = 5 for Concrete, Yatch and Diabetes.
Revised: Insurance, MNIST and Fashion-MNIST, and T = 5 for Concrete, Yatch and Diabetes.

Original: The SKSDfunction is estimated using 30 HMC samples.
Revised: The SKSDfunction is estimated using 30 HMC samples.

Original: For the MNIST datasets, the use of Nvidia P100 GPU with Pascal architecture sped up the trainingwith the CNN-based architecture.
Revised: For the MNIST datasets, the use of Nvidia P100 GPU with Pascal architecture sped up the trainingwith the CNN-based architecture.

Original: For the UCI datasets, due to the use of small networks, thedifferences using CPU or GPU are almost imperceptible.
Revised: For the UCI datasets, due to the use of small networks, thedifferences using CPU or GPU are almost imperceptible.

Original: A.2 Balancing the KLs

Original: Following [46], we define a short initial warming stage (10% of the total training steps) during theoptimization where the KLs are balanced according to their magnitude and the corresponding latentdimension, preventing the model for falling into posterior collapse.

Original: A factor is applied to each KL,following

Original: [Equation]

Original: The factors penalises the fact that a layer might be ignored by making the KL smaller when itsmagnitude is small compared to the rest layers.

Original: A.3 Datasets information
Revised: A.2 Datasets information

Original: [Figure]

Original: Apart from the MNIST and Fashion-MNIST datasets, a total of 10 UCI datasets have been employedin this work including: Bank Marketing, Insurance Company Benchmark, Avocado sales, Naval
Revised: Apart from the MNIST and Fashion-MNIST datasets, a total of 10 UCI datasets have been employedin this work including: Bank Marketing, Insurance Company Benchmark, Avocado sales, Naval

Original: Propulsion Plants, Yatch Hydrodynamics, Diabetes, Boston Housing, Wines, Energy efficiency and 
Revised: Plants, Yatch Hydrodynamics, Diabetes, Boston Housing , Energy efficiency and A.3 Deterministic imputation metrics

Original: Bank Marketing.
Revised: We include in Table 6 results on the RMSE obtained with other discriminative validated predictors,using mean imputation under the same missing rates. Additionally [Table] we include here the missForest inthe baselines, a wide-spread method for missing data imputation using a Random Forest approach[66].

Original: A.4 Convergence of HMC optimization
Revised: For classification tasks [Table] the error rate is considered. In almost all cases, HH-VAEM outperformsthe baselines.

Original: [Figure]
Revised: [Table]

Revised: A.4 Efficacy of training HMC

Revised: We include in this experiment results that demonstrate the efficacy of training the hyperparameters of

Revised: HMC using the gradient-based strategy on several 2D densities.

Original: We face the computational cost of running HMC by defining a small percentage of training steps forthe last stage in Algorithm 1.
Revised: We face the computational cost of running HMC by defining a small percentage of training steps forthe last stage in Algorithm 1.

Original: A 10% of the total training steps for T HMC is sufficient for obtainingthe convergence.
Revised: A 10% of the total training steps for T HMC is sufficient for obtainingthe convergence.

Original: In Figure 6 we include the validation metrics obtained during the optimization of
Revised: In Figure 7 we include the validation metrics obtained during the optimization of

Original: Yatch dataset, where T HMC = 2 e 3 .
Revised: Yatch dataset, where T HMC = 2 e 3 .

Revised: In figure 8 (a), the mean acceptance rate of the HMC sampler over the training steps and the step sizes(b) are included.

Revised: The steps are initialized from U (0 . 05 , 0 . 2) .

Revised: After 2 × 10 3 steps, the mean acceptancerate converges to a value closer to ¯ p a = 0 .

Revised: 65 , which is defined as the optimal desired acceptanceprobability [51].

Revised: This empirical result provides evidence that, apart from reducing the computationalcost, reducing the HMC training step to this value is sufficient for achieving convergence.

Original: [Figure]
Revised: We include in this section results on the negative log likelihood of the reconstructed observed features. In almost all the cases, we confirm the incremental superiority when adding each part of our proposeddesign. [Table]

Revised: A.7 Heterogeneous likelihoods

Original: A.5 Outlier Detection
Revised: In experiment 5.1 we reported an average likelihood across heterogeneous variables. Althoughthis quantity is not a valid joint likelihood probability, we employed this average to provide a faircomparison on models that have been trained on the same heterogeneous likelihoods. In this section,we show the comparison on averaging separately the three considered marginal likelihoods (Gaussian, Bernoulli and Categorical) for two of the biggest datasets considered on Tables 8-10.

Revised: Again, we showthe incremental superiority when adding the different design choices of our model.

Revised: [Figure]

Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: The unsupervised task of Outlier Detection (OD) or Identification can be naturally handled with VAEs by using the reconstruction term E q z [log p ( x | z )] . Nonetheless, they are not always robustagainst outliers, due to their high flexibility [18]. All the existing approaches approximate this scoreby sampling from the Gaussian variational approximation of the true posterior [1, 14, 36, 48].
Revised: [Figure]

Original: [Figure][Table] [Figure]
Revised: [Figure]

Original: [Table] [Figure] [Table] [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure] [Table] [Figure] [Table]
Revised: A.8 Balancing the KLs

Original: [Figure] In this experiment, we evaluate our model in detecting Out ofDistributions (OoD) samples. We usethe metric in equation (14) on the observed data as score. Results for this evaluation on the MNISTdatasets are included in Table 8, where the second dataset within each line is the OoD dataset.
Revised: Following [71], we define a short initial warming stage (10% of the total training steps) during theoptimization where the KLs are balanced according to their magnitude and the corresponding latentdimension, preventing the model for falling into posterior collapse.

Original: Thehierarchy plays a vital role due to the increased flexibility of the model, while the gain provided byincorporating HMC is not considerably big. HMC is expected to increase the performance of themodel in reconstructing data, which affects both in-domain and out-of-domain data. In Figure 7, histograms of the scores provided by HH-VAEM and the Gaussian standard versionwithout HMC (H-VAEM) are included. As one may appreciate, HMC improves the reconstruction inboth the domain and out-of-domain data, which shifts both histograms to the right.
Revised: A factor is applied to each KL,following

Original: [Figure]
Revised: [Equation]

Revised: The factors penalises the fact that a layer might be ignored by making the KL smaller when itsmagnitude is small compared to the rest layers.

Original: A.6 Deterministic baselines
Revised: A.9 SAIA log-likelihoods

Original: We include in Table 7 results on the RMSE obtained with other discriminative validated predictors,using mean imputation under the same missing rates. In all cases, HH-VAEM outperforms thebaselines.
Revised: In order to extend the results provided in Section 5.3, we include here the log-likelihoods curveswhen dynamically selecting features using the same procedure (Figure 9).

Original: A.7 SAIA times
Revised: A.10 SAIA times

Original: The times for obtaining the SAIA metric curves in Section 5.3 are included in Figure 8.
Revised: The times for obtaining the SAIA metric curves in Section 5.3 are included in Figure 10.

Original: Although theperformance is improved with HH-VAEM, it requires considerably higher time than the baselines toevaluate the reward, due to the HMC algorithm for sampling from the true posterior.
Revised: Althoughthe performance is improved with HH-VAEM, it requires considerably higher time than the baselinesto evaluate the reward, due to the HMC algorithm for sampling from the true posterior.

Original: Future workmight be oriented in proposing ways to measure and reduce this gap.
Revised: Future workmight be oriented in proposing ways to measure and reduce this gap.

Original: A.8 SAIA log-likelihoods
Revised: A.11 Training times

Revised: Table 11 shows the average training time in minutes for each model in the experiments for Tables

Revised: 1and 2.

Original: In order to extend the results provided in Section 5.3, we include here the log-likelihoods curveswhen dynamically selecting features using the same procedure (Figure 9)
Revised: The ratio between training times for our method and the Gaussian baselines is approximatelybetween 5 and 10. [Figure] [Figure] [Figure]

Revised: [Table]

Original: [Figure]
Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]