A2A. It sounds like you're getting inference and generation mixed up. If we think about the VAE as probabilistic model we are actually trying to learn a directed generative model (think top-down the decoder is in some sense this model when we account for the latent variables). The encoder is really a tool to help us infer the latent variables when we have an observation (a data point). So with this in mind it is critical that we use the inference network (encoder) to figure out the latent variable quickly when we have data (i.e. my colleague Iulian and I focus on but images are equally applicable) otherwise we would have to sample the directed generative model a lot to find if it can generate the test sample. Again the inference network is a shortcut (though we might still draw several samples to get a Monte Carlo estimate) but it is principled shortcut since we are optimizing a variational lower bound on the log likelihood (the goal is to make this bound as tight as we can). However generation doesn't necessarily require the inference network. Thats why the VAE isn't really an autoencoder it's really two models (one bottom-up and one top-down) trained jointly. So with this in mind you can run the directed generative model you learned in free-running mode and since your prior is a fixed Gaussian it's pretty easy to do. (Note that work such as mine and Iulian shows that actually it's better to learn the prior too but again the VAE framework largely remains the same).