Does pytorch kl_divergence take care of reparameterization and ELBO?

jwillette: <p>I’m used to reading papers and seeing code (like <a href="https://github.com/pytorch/examples/tree/master/vae" rel="nofollow noopener">https://github.com/pytorch/examples/tree/master/vae</a>) with KL divergence which is minimized via ELBO, but recently I looked at an implementation of neural processes which does the same thing, and an implementation just uses the <code>kl_divergence</code> from pytorch (<a href="https://github.com/EmilienDupont/neural-processes/blob/master/training.py#L130" rel="nofollow noopener">https://github.com/EmilienDupont/neural-processes/blob/master/training.py#L130</a>)</p>
<p>I took a look at the source code and docs and there wasn’t much information there besides the basics of what kl is. It looks like the source code might be sampling the distribution somehow…</p>
<p>Does computing loss with kl divergence and then back propagating it remove the need to derive ELBO and reparameterize altogether?</p>

kirk86: <p>I’d like to know the answer as well. One thing that pops up is that VAEs use reparameterisation in order to avoid the autoencoder to learn the identity function, I’m not sure if neural processes suffer from the same problem (I haven’t read in detail)?</p>
<p>Another aspect is that VAEs are doing something like binary classification but on the distribution level not sure if neural process try to tackle the problem from the same perspective.</p>
<p>Finally, I’ve also encountered numerous papers that use the KL and FWIW I’m not fully convinced that empirically at least KL offers anything beyond what you’ll get with cross entropy.</p>

jwillette: <p>IDK what I was really asking in the original question. KL is just a autograd enabled version of the KL divergence for the given distributions. the kl_divergence function does nothing for reparameterization. Reparameterization happens by manually computing it or using the <code>rsample()</code> function which is present on many of the distributions in Pytorch.</p>
<p>Also, KL doesn’t replace CE, its just an added regularization term on whatever distribution is being reparameterized to keep it from drifting too far from the prior.</p>

kirk86: <aside class="quote no-group" data-username="jwillette" data-post="3" data-topic="60439">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jwillette/48/17366_2.png" class="avatar"> jwillette:</div>
<blockquote>
<p>KL doesn’t replace CE, its just an added regularization term on whatever distribution is being reparameterized to keep it from drifting too far from the prior.</p>
</blockquote>
</aside>
<p>Thanks, I agree that KL doesn’t replace CE, I’m just questioning whether “empirically” is really contributing to the final model? E.g. have you seen any results/differences between a vae with and without a KL regularisation term? Is the final model with kl regularisation orders of magnitude better than the one without or is it just negligible difference?</p>

jwillette: <p>When I make a model which requires KL, I usually run it once while testing without the KL to make sure everything is hooked up and working correctly. What I have seen there is that without the KL, the variance of whatever distribution that is being reparameterized collapses to look more deterministic.</p>
<p>It probably depends on the specific data and problem as to whether this would help or hurt. But if the variance does go to zero and it helps, then you probably don’t need a reparamterized model in the first place would be my guess.</p>