A Disentangled Adversarial Neural Topic Model for Separating Opinions from Plots in User Reviews (2021)

Abstract: The flexibility of the inference process in Variational Autoencoders (VAEs) has recently led to revising traditional probabilistic topic models giving rise to Neural Topic Models (NTM). Although these approaches have achieved significant results, surprisingly very little work has been done on how to disentangle the latent topics. Existing topic models when applied to reviews may extract topics associated with writers’ subjective opinions mixed with those related to factual descriptions such as plot summaries in movie and book reviews. It is thus desirable to automatically separate opinion topics from plot/neutral ones enabling a better interpretability. In this paper, we propose a neural topic model combined with adversarial training to disentangle opinion topics from plot and neutral ones. We conduct an extensive experimental assessment introducing a new collection of movie and book reviews paired with their plots, namely MOBO dataset, showing an improved coherence and variety of topics, a consistent disentanglement rate, and sentiment classification performance superior to other supervised topic models.

## introduction variational autoencoders (vaes) @xcite allow to design complex generative models of data. in the wake of the renewed interest for vaes, traditional probabilistic topic models @xcite have been revised giving rise to several neural topic model (ntm) variants, such as nvdm @xcite , prodlda @xcite , and ntm-r @xcite . although these approaches have achieved significant results via the neural inference process, existing topic models when applied to user reviews may extract topics with writers' subjective opinions mixed with those related to factual descriptions such as plot summaries of movies and books @xcite . yet surprisingly very little work has been gandalf frodo jackson tolkien dwarf fellowship . . . watch sure hobbit trilogy download purchase . . . silly dwarf lotr jackson shame remove . . . . . . . . . ## diatom architecture our proposed diatom model is shown in figure 2. assuming a document x is associated with a sentiment label y s , and each document can be represented by latent topics associated with senti-ments (z s ) and plots foot_1 (z a ), we aim to learn a model maximizing the joint data-label log-likelihood, log p(x, y s ): inspired by @xcite and @xcite , we assume the document-level topic distribution for plots can be approximated by a multilayer perceptron (mlp) taking as input a multivariate gaussian distribution, and similarly for the topic distribution for sentiments. the multinomial distribution over words under a plot topic and an opinion topic can be parametrised by a weight matrix w . the generative process is shown below. the outputs of both networks are vectors in r k . here, 'diag' converts a column vector to a diagonal matrix. for a document x, q(φ) ln (µ φ , σ φ ). with such a formulation, we can generate samples from q(φ) by first sampling ∼ n (0, i) and then computing φ = σ(µ φ + σ φ 1/2 ). ## experimental setup we conduct thorough experimental evaluations to assess the quality and disentanglement rate of extracted topics. to assess the quality of topics, we compute their topic coherence @xcite coupled with their topic uniqueness. then, we additionally look at the discriminative power of the disentangled features on the sentiment classification task. to fully assess the disentanglement rate of different methods, we perform topic labeling to compute the sentiment polarity of each topic (if any) and then measure the overall disentanglement rate ( §5.3). as a result, we obtain an estimate of the extent to which different models can accurately control the topic disentanglement rate. we introduce and use a new dataset, named the mobo dataset, pairing movie/book plots with their users' reviews, and including human-annotated sentences. ## experimental results we report the results in terms of topic coherence/uniqueness, sentiment classification and topic disentanglement rate. we also perform ablation studies to gain more insights into our model. ## conclusions we have described diatom, a new neural topic model to generate disentangled topics through the combination of vae and adversarial learning. the results on the novel mobo dataset show that di-atom generates better topics in terms of both topic coherence and topic uniqueness, and can disentangle opinion-bearing topics from plot/neutral ones. finally, we have identified some existing limitations and provided viable solutions to be explored in the future.