Variational Autoencoder with Embedded Student-t Mixture Model for Authorship Attribution (2020)

Abstract: Traditional computational authorship attribution describes a classification task in a closed-set scenario. Given a finite set of candidate authors and corresponding labeled texts, the objective is to determine which of the authors has written another set of anonymous or disputed texts. In this work, we propose a probabilistic autoencoding framework to deal with this supervised classification task. Variational autoencoders (VAEs) have had tremendous success in learning latent representations. However, existing VAEs are currently still bound by limitations imposed by the assumed Gaussianity of the underlying probability distributions in the latent space. In this work, we are extending a VAE with an embedded Gaussian mixture model to a Student-t mixture model, which allows for an independent control of the “heaviness” of the respective tails of the implied probability densities. Experiments over an Amazon review dataset indicate superior performance of the proposed method.

## conclusion variational autoencoders (vaes) have proven their benefit in many tasks. they provide an attractive machine-learning framework that combines the strengths of neural-network training with the power of uncertainty metrics derived from statistical models. they can learn a low-dimensional manifold to summarize the most salient characteristics of data and provide a natural, statistical interpretation, both in the latent as well as in the observation space. in our work, we are addressing the question of whether vaes can benefit from a statistical model that allows for more heavy-tailed distributions. the promise of heavy-tailed distributions is that susceptibility to outliers is curtailed and that the overall capacity of the model is improved. towards that goal, we have proposed and evaluated a vae that is equipped with an embedded student-t mixture model. it incorporates an assumption of student-t distributed data into the joint learning mechanism for the latent manifold and its statistical distribution. variational inference is performed by trying to simultaneously solve both tasks: jointly learning a nonlinear mapping to transform a given dataset of interest onto a (lower-dimensional) subspace and grouping the latent representations into meaningful categories. we have derived a variational learning algorithm to accomplish this goal and we have shown its benefit for learning latent representations, both on synthetic data as well as a real-world authorship attribution task. our more flexible model provided a capacity to obtain better results than an svm-based classifier as well as a standard gaussian vae.