Before revision:  We show that the vanilla VAE's generative capacity lies between the pairwise Potts and site-independent models.
After revision:  We show that the generative capacity of the Potts Hamiltonian model is the largest, in that the higher order mutational statistics generated by the model agree with those observed for natural sequences. In contrast, we show that the VAE's generative capacity lies between the pairwise Potts and site-independent models.