  Learning from heterogeneous data poses challenges such as combining data from
various sources and of different types. Meanwhile, heterogeneous data are often
associated with missingness in real-world applications due to heterogeneity and
noise of input sources. In this work, we propose the variational selective
autoencoder (VSAE), a general framework to learn representations from
partially-observed heterogeneous data. VSAE learns the latent dependencies in
heterogeneous data by modeling the joint distribution of observed data,
unobserved data, and the imputation mask which represents how the data are
missing. It results in a unified model for various downstream tasks including
data generation and imputation. Evaluation on both low-dimensional and
high-dimensional heterogeneous datasets for these two tasks shows improvement
over state-of-the-art models.
