Variational autoencoders (VAEs) have recently gained popularity as generative protein sequence models (GPSMs) to explore fitness landscapes and predict the effect of mutations. Despite encouraging results, a quantitative characterization of the VAE-generated probability distribution is still lacking. In particular, it is currently unclear whether or not VAEs can faithfully reproduce the complex multi-residue mutation patterns observed in natural sequences arising due to epistasis. In other words, are frequently observed subsequences assigned a correspondingly large probability by the VAE? Using a set of sequence statistics we comparatively assess the accuracy, or "generative capacity" , of three GPSMs : a pairwise Potts Hamiltonian, a vanilla VAE, and a site-independent model, using natural and synthetic datasets. We show that the vanilla VAE's generative capacity lies between the pairwise Potts and site-independent models. Importantly, our work measures GPSM generative capacity in terms of higher-order sequence covariation  and provides a new framework for evaluating and interpreting GPSM accuracy that emphasizes the role of epistasis.
Potts models and variational autoencoders (VAEs) have recently gained popularity as generative protein sequence models (GPSMs) to explore fitness landscapes and predict the effect of mutations. Despite encouraging results, quantitative characterization and comparison of GPSM-generated probability distributions is still lacking. It is currently unclear whether GPSMs can faithfully reproduce the complex multi-residue mutation patterns observed in natural sequences arising due to epistasis. We develop a set of sequence statistics to assess the "generative capacity" of three GPSMs of recent interest: the pairwise Potts Hamiltonian, the VAE, and the site-independent model, using natural and synthetic datasets. We show that the generative capacity of the Potts Hamiltonian model is the largest, in that the higher order mutational statistics generated by the model agree with those observed for natural sequences. In contrast, we show that the VAE's generative capacity lies between the pairwise Potts and site-independent models. Importantly, our work measures GPSM generative capacity in terms of higher-order sequence covariation statistics which we have developed, and provides a new framework for evaluating and interpreting GPSM accuracy that emphasizes the role of epistasis.
R
Variational
Potts models and variational
R
a quantitative characterization of the VAE-generated probability distribution
quantitative characterization and comparison of GPSM-generated probability distributions
R
In particular, it
It
R
or not VAEs
GPSMs
R
In other words, are frequently observed subsequences assigned a correspondingly large probability by the VAE? Using a
We develop a
R
we comparatively assess the accuracy, or
to assess the
D
,
R
: a
of recent interest: the
R
a vanilla
the
R
a
the
R
vanilla
generative capacity of the Potts Hamiltonian model is the largest, in that the higher order mutational statistics generated by the model agree with those observed for natural sequences. In contrast, we show that the
A
statistics which we have developed,