Unsupervised Discovery of Interpretable Latent Manipulations in Language VAEs
Authors: Max Ryabinin
Artem Babenko
Elena Voita
Venue:  (2021)
Keywords: interpretability
unsupervised interpretable directions
controllable text generation

Abstract: Language generation models are attracting more and more attention due to their constantly increasing quality and remarkable generation results. State-of-the-art NLG models like BART/T5/GPT-3 do not have latent spaces, therefore there is no natural way to perform controlled generation. In contrast, less popular models with explicit latent spaces have the innate ability to manipulate text attributes by moving along latent directions. For images, properties of latent spaces are well-studied: there exist interpretable directions (e.g. zooming, aging, background removal) and they can even be found without supervision. This success is expected: latent space image models, especially GANs, achieve state-of-the-art generation results and hence have been the focus of the research community. For language, this is not the case: text GANs are hard to train because of non-differentiable discrete data generation, and language VAEs suffer from posterior collapse and fill the latent space poorly. This makes finding interpetable text controls challenging. In this work, we make the first step towards unsupervised discovery of interpretable directions in language latent spaces. For this, we turn to methods shown to work in the image domain. Surprisingly, we find that running PCA on VAE representations of training data consistently outperforms shifts along the coordinate and random directions. This approach is simple, data-adaptive, does not require training and discovers meaningful directions, e.g. sentence length, subject age, and verb tense. Our work lays foundations for two important areas: first, it allows to compare models in terms of latent space interpretability, and second, it provides a baseline for unsupervised latent controls discovery.

Under review as a conference paper at ICLR 2021
UNSUPERVISED DISCOVERY OF INTERPRETABLE LATENT MANIPULATIONS IN LANGUAGE VAES
Anonymous authors
Paper under double-blind review
ABSTRACT
Language generation models attracting more and more attention due their constantly increasing quality and remarkable generation results. State-of-the-art NLG models like BART/TS/GPT-3 do not have latent spaces, therefore there is no natural way to perform controlled generation. In contrast, less popular models with explicit latent spaces have the innate ability manipulate text attributes by moving along latent directions. For images, properties of latent spaces are well- studied: there exist interpretable directions (e.g. zooming, aging, background re- moval) and they can even be found without supervision. This success is expected: latent space image models, especially GANs, achieve state-of-the-an generation results and hence have been the focus of the research community. For language, this is not the case: text GANs are hard to train because of non-differentiable discrete data generation, and language VAEs suffer from posterior collapse and fill the latent space poorly. This makes finding interpetable text controls chal- lenging. In this work, we make the first step towards unsupervised discovery of interpretable directions in language latent spaces. For this, we turn to methods shown work in the image domain. Surprisingly, VAE representations of training data consistently outperforms shifts along the co- ordinate and random directions. This approach is simple, data-adaptive, does not require training and discovers meaningful directions, e.g. sentence length, subject age, and verb tense. Our work lays foundations for two important areas: first, it allows to compare models in terms of latent space interpretability, and second, it provides baseline for unsupervised latent controls discovery.
find that running PCA
INTRODUCTION
Transformer- based models yield state-of-the-ar results number of tasks, including represen- tation learning (Devlin et al.] 2019 Liu et al.) 2019 Clark et al.) (020) and generation (Radford) al. Raffel al. 2019 Lewis et 2020). Notably, large language models have been reported to
produce outputs nearly indistinguishable from human-written texts (Brown et al. (020)..
Although the predictions of autoregressive language models fluent and coherent, it is not clear how to manipulate the model to get samples with desired properties. For example, make them shorter, more formal or more positive, or, alternatively, use the same model to rewrite texts in a different tone. Current approaches often rely on external labels of target attributes and require modifications to the model. This involves retraining for new attributes or changing the decoding procedure, which is usually expensive.
In contrast, models with explicit latent spaces have the innate ability to manipulate text attributes by moving along latent directions. They, however, gained limited traction. One reason is that training VAE on text data poses a number of optimization challenges, which have been tackled with a varying degree of success (He et al., 2019: Fu et al. 2019 Zhu et al.) 2020). Additionally, language VAEs are mostly small LS -SST--based models which goes against the current trend using large pretrained Transformers. The first large-scale language VAE model is the recently introduced OPTIMUS (Li et al., 2020): it uses BERT as the encoder and GPT-2 as the decoder, and sets a new record on benchmark datasets.
Differently from texts, latent space models for images, especially GANs, achieve tate-offthe-aa generation results. Therefore, these models have been the focus of the research community, and the

---

Under review as a conference paper at ICLR 2021
properties of latent spaces are well-leamed. For example, even early works generative adver- sarial networks for images report that is possible to have smooth interpolations between images al.) 2014). More recent studies show that the latent space di-
in the latent space (Goodfellow rections corresponding to human-interpretablle image transformations (from now on, "interpretable
directions") can be discovered in an unsupervised way (Härkönen et al. 2020 Voynov & Babenko,
2020 Peebles et al. 2020).
In this paper, we show that for the language domain, much alike the well-studied visual domain, sufficiently "good" latent space allows to manipulate sample attributes with relative ease. To avoid the known difficulties associated with training language GANs, we experiment with VAEs; more specifically, with the current state-of-the of--the-art art model OPTIMUS. We show that for this model, not only is possible to produce meaningful and "smooth" interpolations between examples and transfer specific properties via arithmetic operations in the latent space, but it is also possible to discover the interpretable latent directions in an unsupervised manner. We propose a method based on the PCA of latent representations of the texts in the training dataset. According to human evaluation, the proportion of interpretable directions among the ones found by our method is consistently larger than the proportion of interpretable directions among canonical co-ordinates or random directions in the latent space. The meaningful directions found by this method include, for example, subject age, subject gender, verb tense, and sentence length. Some of the directions, e.g. sentence length, potentially useful: the ability to expand or shrink a text while preserving its content may be useful for tasks like summarization.
Note that the proposed method is simple and fast. The method is simple because it requires only the forward pass of the encoder, without backpropagating through decoding steps. This is very important for the language domain, where backpropagation through samples is significantly more difficult than for images. Namely, generation is oondiffeeretiable, and previous attempts to overcome this issue relied on noisy or biased gradient estimates, which is less reliable than the standard MLE training. Instead, we do not rely on generated samples at all: we operate directly the latent space. Additionally, since sampling directly from the prior does not yield diverse samples in case of OPTIMUS, we use the representations of the training data without running a decoding procedure- this maked the method fast.
To summarize, our contributions are as follows:
1. We propose the first method for unsupervised discovery of interpretable directions in latent spaces of language VAEs.
2. This method is simple and fast: it is based PCA of latent representations for texts in the training dataset.
3. This method is effective: the proportion interpretable directions among the ones found by our method is consistently larger than that of canonical co-ordinates or random directions in the latent space.
4. Our work lays foundations for two important areas: first, it allows to compare models in terms of latent space interpretability, and second, it provides a baseline for unsupervised latent controls discovery.
2 RELATED WORK
Finding interpretable directions in latent spaces of language VAEs is related to three lines of work. First, latent variable models for text and, more specifically, properties of latent spaces: for inter- pretable directions to exist, latent space has to be smooth (i.e. allow coherent interpolations). Then, since great part of the motivation for finding interpretable directions is manipulating generated texts, we discuss works controllable text generation for different types of models, both VAE and stan- dard autoregressive. Finally, we mention recent works trying to discover interpretable directions in image GANs.
2.1 LATENT VARIABLE MODELS FOR TEXT
Latent variable models encode information about text into a probability distribution. In addition to sampling new sentences from the prior distribution, they potentially allow to explicitly encode
2

---

Under review as a conference paper at ICLR 2021
specific properties of text, such as sentiment or style. Even early works on VAEs show that latent space obtained with the VAE objective can result coherent interpolations [owmman et al. 2016).
While this is encouraging, training good VAEs with smooth and expressive latent spaces is chal- lenging. Specifically, for interpretable directions to exist, we need a model which (i) does not ignore latent variable to produce good samples, (ii) has continuous latent space to allow controllable manipulation.
Ignoring latent variable is known problem of VAEs. It arises because of the the KL vanishing problem: over the course of training, the KL divergence part of the loss may drop to O, which indi- cates that the model ignores the latent variable. There exist many ways to alleviate this issue (Yang) et al. 2017 Fang et al. 2019 Fu et al.) 2019 Zhu et al. 2020); one of the simpler ways is adjusting the weight of the KL loss component according to specific schedule.
Another problem is the latent vacancy problem: differently from images, not all regions of the latent space are occupied by the posterior distribution (Xu al., 2020). In simple words, text latent spaces tend to have "holes" where the decoding network fails to generalize. As result, when the latent codes are manipulated, the modified codes often land in these holes or vacant regions the posterior latent space. If this happens, model can not decode properly.
In light of the above, discovery of interpretable directions in text latent spaces is possible only with a strong model. Therefore, we use the current state-of-the-ar model OPTIMUS (Li et al. 2020). a recent large-scale variational autoencoder which initializes the encoder with BERT (Devlin et al., 2019) and the decoder with GPT2 (Radford et al.). In addition to the model's high capacity, we use it because of the available checkpoints and reported results on latent space manipulation.
2.2 CONTROLLABLE GENERATION FOR TEXT DATA
Latent variable models. natural way to achieve text generation with required attributes is us- ing latent variable text generation models. The idea is that information about the attribute value encoded in the latent code, and obtain samples with the desired property one has fix the corresponding component (direction) of the code.
For example, several works learn latent spaces with disentangled representations of content and style (Hu et al., 2017 Logeswaran et al. 2018 Lample et al. 2019; Yang et al., 2018, [Shen et al., 2017 John et al., 2019). After that, to generate sentences Depending on the approach, this style vector can either be estimated by encoding sentences with the desired attribute or directly produced by specifying the structured latent code (e.g. one-hot encoding of an attribute).
specific style, the style vector fixed.
line of research shows that possible to achieve attribute manipulation by moving in the latent space along specific vectors. These vectors, however, are found using data labelled with the attribute, i.c. with supervision. For example, (Shen et al. (2020) change tense of sentence by adding to its latent representation "tense vector" computed as difference of averaged representations of sentences with different tenses; Wang et al. (2019) use gradients of the attribute classifier. One of the first successful methods that learns disentangled latent space is the work by Xu et al.) (2020): they use basis vectors in the constrained latent space; however, this involves training a model with structured latent space, which rather complicated.
Another
Autoregressive models. Controllable generation for standard autoregressive language models is usually achieved by either prepending an attribute to the input sequence as prompt (Keskar al, 2019) training an additional component of the model (Chan et al., 2020) or adjusting the decoding result with additional attribute-specific language models (Dathathri et al. 2020). A more thorough comparison of approaches to controlled text generation can be read in Prabhumoye et al. (2020).
Note that all these approaches require supervision and substantial changes to either training or gen- eration procedures, whereas approach is applicable to any variational autoencoder.
2.3 INTERPRETABLE DIRECTIONS MINING IN IMAGE GANS
To the best of our knowledge, there are only three works which discover interpretable latent direc- tions in unsupervised way, and all of them operate with GANs for images.
3

---

Under review as a conference paper at ICLR 2021
Two of them are not applicable to texts directly. In Voynov & Babenko (2020), the interpretable directions are trained. These directions are the ones which can be recognized by a separate recon- structor based on two samples, from the original and shifted latent vector. [Peelles al. (2020) propose to learn disentangled attributes by minimizing the sum of squared off-diagonal terms of the generator Hessian matrix. Both approaches require backpropagation through sampling and there- fore are not applicable directly for texts: unlike images, generated texts are not differentiable with respect their latent representations.
The last approach, Härkönen et al. (020), show that interpretable controls for image synthesis can be identified by finding principal components from layer outputs of the generator network for several samples from the prior. In our more challenging language domain, instead of sampling from the generator distribution, we take advantage of the availability of the encoder VAEs and perform PCA on training data representations.
BACKGROUND
3.1 VARIATIONAL AUTOENCODERS FOR TEXT DATA
Variational autoencoders (Kngmma& & Welling 2013) are class of generative models. They consist of an encoder and a decoder. For a given text, the encoder outputs a latent code: a d-dimensional probability distribution (usually a gaussian with diagonal covariance matrix). The decoder receives a vector sampled from some distribution (in training, from the encoder output, in inference, from the prior) and generates an output text.
VAES are trained to minimize the sum of the reconstruction loss (for language, usually it is the cross-entropy loss) and the KL divergence loss, which forces the encoder outputs stay close to the prior distribution. To avoid the KL vanishing problem (see section the KL term of the loss is weighted with 3, which is annealed according predefined schedule (Bowman et al. 2016). Formally, the loss function is as follows:
arg min L3 arg min (Cree + (BKL))
0.0
6.0
For more details on the specifics of loss and training, see e.g. Li et al. 2020
3.2 OPTIMUS: THE FIRST LARGE-SCALE TEXT VAE
The current Bate-of-the-are variational autoencoder is OPTIMUS (Li et al. 2020 This the first large-scale VAB: it initializes encoder and decoder networks with pretrained weights of BERT (De- vlin et al., 2019) and GPT-2 (Radford et al.) respectively. There are two strategies for including the latent vector in the generative process of the Transformer decoder: memory and embedding. In the memory strategy, latent vector is concatenated with the prefix tokens; in the embedding strategy, this latent vector is added to every token representation in the decoder.
We use OPTIMUS because for our study, we need a strong model with a good latent space. For a detailed explanation, see section 2.1
4
OUR METHOD
Each method for discovery of interpretable directions is inevitably paired with its specific underlying definition of the vague notion of "interpretability"": the former defines the latter, and vice versa. For example, in Voynov & Babenko 2020)). interpretable directions are the ones that are easy to distinguish from each other by observing results of manipulations; in Peebles al. (2020) the ones that change the output image independently from each other.
To start, we formulate the desired properties we believe interpretable directions should have. In our setting, interpretable directions are the ones that:
are orthogonal: this is a weaker form of the standard attribute disentanglement requirement, which states that changing one attribute of the sentence should not affect others;
4

---

Under review as a conference paper at ICLR 2021
maximize variance, i.e. they "explain" the data distribution well, that its most distinctive features are exposed as controllable attributes. This requirement might be formalized as follows: given the d-dimensional latent representations of samples X E Baxds interested in finding a set of vectors Wy that maximize the sum of [Xwxl]?
4.1 PCA FOR ENCODED TRAINING DATA
Luckily, common statistical technique Principal Component Analysis (PCA) satisfies both of these requirements by construction. This technique obtains principal components (vectors with highest variance) as cigenvectors of the covariance matrix X X corresponding to the largest eigenvalues. Although PCA is mostly used dimensionality reduction method, use it to obtain linear transformation that corresponds to a set disentangled controllable text attributes.
1. Encode unlabeled training samples
2. Compute PCA
3. Use principal components change sentence attributes boxt
of encoded samples
Usee buys ure on wates, and one of tler Loids our tie waters
millary offict manurbkko a bust laye samy fay the background.
PC2
milltar
pins
wamee is praciciuu Let youa posdiooss
militarr man dhrvo.
min driving
Beiil af werkiing
Real sentences
Samples from the VAE
Figure 1: The illustration of our method; PC stands for "Principal Component".
The resulting method is straightforward procedure (see Figure
Encode examples from the training dataset (since only encoder used, no generation required). For each example, we will have a probability distribution.
2. Take the expectation vector of each distribution, stack them to form matrix.
3. Compute the principal components of the resulting matrix.
4. Take top-N principal components with the highest explained variance.
Note that our method is very simple: it does not involve neither training any additional models generation from the model (which is slow for texts). Despite its simplicity, this approach dis- covers meaningful and often non-trivial latent manipulations (section 5.2) and the proportion of interpretable directions is greater compared several baselines (section 5.3
4.2 NOTES ON DESIGN CHOICES
In this part, explain some of our design choices in light of related works interpretable di- rections mining for image GANs (Voynov & Babenko| 22020; PPeelles et al.) 22200: Härkönen et al.) 2020) and highlight why the fist two of these three methods are not directly applicable for texts.
We do not sample from the prior. An important distinction of our approach from prior work on latent discovery in image GANs that do not use sampling from the prior distribution of the trained model. This has several reasons. First, while image generators are convolutional and the entire image is generated one pass, text decoders produce sentences token by token, which significantly slows down training. Second, we observed that sampling from the prior distribution of OPTIMUS does not result in high-quality samples. For example, when latent vectors are sampled from (00, Ia), the resulting texts are highly repetitive. Namely, average over 40% of tokens in a generated text have the same token type. To achieve higher generation quality, we could estimate the variance (22) from training data and sample latent vectors from (000, (=))) instead of A/ Ig). However, this requires mapping all training sentences to the latent space; at that point, one already has latent vectors encoding natural texts, so further sampling redundant.
5

---

Under review as a conference paper at ICLR 2021
Two of the methods are not directly applicable for texts. While our method is similar to the one proposed in Härkönen et al. (2020) for images, there exist two other methods that show promising results. Here, would like to stress that approaches by Voynov & Babenko (2020) and [Peebles) et al. (2020) for the visual domain are not directly applicable to language generation: they require backpropagation through discrete sampling. This problem has previously been studied in the context of language GAN training, and there is an evidence that the community has not yet proposed solution that would consistently outperform maximum likelihood training (Caccia et al. 2020).
Other matrix decompositions. There exist numerous ways to decompose the latent representation matrix into several orthogonal components. However, our preliminary evaluation of Independent Component Analysis (Hyvärinen & Oja 2000), which is one of the most widely used alternatives to PCA, did not yield substantially better results in terms direction interpretabilityy
5 EXPERIMENTS
In this part, we first show qualitative results: categories of the interpretable directions found by our method along with some examples. Next, we show that the success of our method can not be attributed solely to the properties of the latent space. Namely, the proportion of interpretable directions among the ones found by our method is larger than that of the baselines. Since, as we will see, types of the revealed directions can be quite diverse, developing metric for automatic evaluation of interpretability is challenging. Therefore, we resort to human evaluation.
5.1 SETUP
Models. We evaluate three publicly available OPTIMUS checkpoints: one of the model trained on a dump of Wikipedia with 3 = 0.5 and latent vector size 32, and two of the models finetuned on the SNLI dataset (Bowman et al. 2015 with / and and latent vector size 768]
In preliminary experiments, we also examined other models from prior work. As expected, sim- ple attributes are discoverable even in lower capacity models, but the overall reconstruction and generation quality is worse.
We compute the PCA for latent representations on a subsample of 50000 training examples for each model; larger subsample sizes did not improve quality of the attributes.
Finding directions.
Baselines. Since mining of interpretable latent manipulations for texts is new task, there are no estabilished methods. Therefore, we implement two simple but reasonable baselines.
Random directions: vectors sampled from the standard normal distribution and normalized unit length. This is the easiest way to find meaningful directions in the latent space.
Coordinate directions: coordinate vectors in the latent space. To choose the "best" di- rections, we pick coordinates with the highest variance over the training dataset repre- sentations. This goes in line with the motivation of PCA: out of all possible coordinate directions, these are responsible for most of the variance in data.
Note that the coordinate baseline can be potentially quite strong. The standard Gaussian distribution, which the prior for the OPTIMUS model, has independent components. Since OPTIMUS is trained with KL divergence between the posterior and the prior distributions, canonical coordinates its latent space are likely to be disentangled.
Evaluating directions. To evaluate the directions, sample sentences from the corresponding training set and apply the latent shift to the encoder output. Namely, we increase or decrease the
"Ideally would examine the properties of model with tighddimensional latent space trained on general corpus. Unfortunately, this was not possible: training such model from scratch prohibitively expensive, and OPTIMUS checkpoints from this kind models are not publicly available.
AAdiiiionally, in preliminary experiments we also looked at top coordinates with the lowest variance, well randomly chosen ones. Coordinates with the highest variance tend he more reasonable and interpretahle.

---

Under review as a conference paper at ICLR 2021
attribute presence by multiplying the shift vector by scalar constant and adding it to the mean vector of the distribution. The value the scalar varies from -m to m; each case, we take points, {-m, -0.5m, 0, 0.5m, m). so that observe the unchanged sentence and its 4 modifi- cations with a varying direction and degree of change. For the PCA directions, m = 5; for the 10. These values of in model output, higher lead to degenenerate sequences with repeated tokens. For generation, we use nucleus sampling (Holtzman al. 2020) with p 0.9; in preliminary experiments, we also found the results do not change when using greedy decoding.
baselines,
found empirically: lower result in mostly changes
5.2 QUALITATIVE RESULTS
First, we observe that almost all variance in latent representations of the SNLI is covered by a small fraction of principal components (see Appendix A This hints at the presence of directions that capture the most important features of these texts.
Next, we look how the shifts along the top components influence generated texts. Figure shows examples of directions discovered by our method for the model trained on the SNLI dataset. The discovered directions can be roughly categorized into four types:
1. Basic sentence attributes, such as length: varying the length attribute produces expanded or contracted versions of the original sentence with minor content alterations. This might be useful for sentence summarization or the reverse task of enriching a text with more details.
2. Word-level altributes: similarly to image models, OPTIMUS has directions that can affect the age of the subject. transform nouns from singular to plural, etc. One of these direc- tions transforms nouns from gender-specific to gender-neutral (c.g. "a man" "a woman" changes "a person"); this can also find practical applications given the known issue of unintended biases in language models (Dinan et al. 2020 Florez, 2019).
3. Insertion of particular words: other than changing the existing words in the sentence, some directions force the model to add specific tokens. They range from content words, such as nouns, to function words, such prepositions.
4. Enforcing specific structure: a few directions change the input sentence to adhere to certain syntactic structure. In case of SNLI, this may be explained by the declarative nature of most sentences in the dataset with several frequent verb phrases.
1. Basic sentence attributes
2. Changes of word-level attributes
woman dreams.
two men are having two men having a conversation
girl dreams.
two men huving a conversution while having u briefcase
3 people with umbrellas street next park
3 children with a umbrella park
3. Insertion of particular words
4. Enforcing specific structure
a woman dreums. there is man celebrating with friends there is man celebrating in group
friends playing football together that team is playing football
a woman in dreams.
sitting the ground playing drums. the man playing drums.
a man in jeans runs through park.
a mun in jeans runs a purk.
Figure 2: Examples of the directions discovered by our method for the SNLI model.
5.3
HUMAN EVALUATION
5.3.1 PROCEDURE
We evaluate the three available checkpoints of OPTIMUS and the three methods (PCA with two baselines) described above. For each model, we sample 20 sentences from the corresponding train- ing dataset. Then, for each method take 20 "best" directions and apply them to the gathered

---

Under review as a conference paper at ICLR 2021
sentences. Note that some directions applicable only to fraction of sentences (e.g., gender- specific attributes require a human subject). If after applying direction shifts to sentence at least three of the 5 versions
the same, do not use this sentence for this direction. To form an annotation task, we randomly choose 5 sentences from the filtered results. For a single combination of model, method, and direction, generate such tasks.
The evaluation protocol for one annotation task is as follows:
The annotator sees sentences along with their modifications. They have to answer whether the direction corresponding to that example is interpretable
2. If the direction is interpretable, the annotator has to specify its category: either choose one the described in section [52) or enter description manually.
3. If the direction is partially interpretable (e.g. content preservation is lacking or the results are "visible" only in part of the examples), the annotator has to indicate this.
Our annotators are 12 people with the background in machine learning. On average, we give each participant about 75 annotation tasks. For a single combination of model, method, and direction, we aggregate the results with the majority vote.
Method
SNLI, 8 0 SNLI, B Wiki
Method
Basic
Word Insertion Structure
Random 0.2
0.3
Random
0,41
0.35
0.09
0.12
Coordinate 0.2
0.15
0
Coordinate 0.63
0.23
0.06
0.06
PCA
0.6
0.4
0.05
PCA
0.25
0.6
0.06
0.06
Table Proportion of interpretable directions found by each method.
Table 2: Proportion of direction categories found by each method for the SNLI model.
5.3.2 RESULTS AND DISCUSSION
The results are shown in Tables I and 2 more detailed statistics are given in Appendix C Our method outperforms the baselines for three models, being the only one that achieves more than 50% of output direction interpretability. Apparently, the Wikipedia model exhibits too low recon- struction quality to be interpretable; this is most likely because its latent space dimension is 32, which is very small.
Interestingly, for the SNLI model with 1, coordinate directions perform worse than random direction or even coordinate directions for the model with , = 0. Let recall that / affects the regularization strength: higher 3 forces the latent distribution be closer to the prior, in this case, an isotropic Gaussian. This has two implications. First, the model with the stronger regularization (8 1) has weaker representation capabilities. Second, differences variance the output distribution become less pronounced: such, the fraction of explained variance becomes non-informative coordinate selection criterion.
If we now look at the specific categories of the directions discovered by different methods (Table 2). we will see an interesting pattern: the baseline methods discover mainly basic text properties (e.g. length), but our method identifies mostly changes of word-level attributes (subject gender, age, num- ber, etc.) Note that these properties are more useful for the SNLI task than hasic sentence attributes. This suggests that the directions discovered by our method are more reasonable for given model.
CONCLUSION
We propose the first method for unsupervised discovery of interpretable attribute manipulations in text variational autoencoders. This method involves computing the principal components of training data representations. It is very simple, fast, and outperforms the basclines by large margin. Future work may investigate the locality of directions in text: many modifications (e.g. age) applicable to only a fraction of sentences, and this might influence the results in Table
"The instruction shown the annotators he seen Appendix
8

---

Under review as a conference paper at ICLR 2021
We believe this work will encourage further research in two important areas. First, approach may serve as a baseline for the new task of unsupervised discovery of interpretable latent controls in generative models. Second, it can be used to compare models in terms of latent space interpretability.
REFERENCES
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. A large an- notated corpus for learning natural language inference. In Proceedings of the 2015 Confer- ence on Empirical Methods in Natural Language Processing, pp. 632-642, Lisbon, Portugal, September 2015. Association for Computational Linguistics. doi: 10.18653/v1/D15--007. URL https: www. ac acwweblwwbb..... orq/ www.aaaooogooiii-aoaaa
Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew Dai, Rafal Jozefowicz, and Samy Ben- gio. Generating sentences from continuous space. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pp. 10-21, Berlin, Germany, Au- gust 2016. Association for Computational Linguistics. doi: (0.18633/1/K11--1002 URL
https: www www.aclweb.orr//atttooo...
1002
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. arXiv preprint arXiv:2005. 14165, 2020.
Massimo Caccia, Lucas Caccia, William Fedus, Hugo Larochelle, Joelle Pineau, and Laurent Char- lin. Language gans falling short. In International Conference on Learning Representations, 2020.
URL httph:tts//seeeeeeeeeee httpaeeeeeeeeeeeeeeeoeooooooiiea
net htte:/oooommidieeeeee (irruum2ddBJJ
Alvin Chan, Yew-Soon Ong, Bill Pung, Aston Zhang, and Jie Fu. Cocon: A self-supervised ap- proach for controlled text generation, 2020.
Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. Electra: Pre-training text encoders as discriminators rather than generators. In International Conference on Learning Representations, 2020. URL https: trstps://openee httpp:::/ppeeeeeeeetee: http:///ppeeeeeeeee//ee/ net worum?ic-rrirree B1MHBBBt
Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. Plug and play language models: simple approach to con- trolled text generation. In International Conference Learning Representations, 2020. URL net. forum?iiiio (oorumi1d=H1eedyyRRR
https::/cpperreiiie htt.://wwweee.eett.oomodee.d.ee.
Jacob Devlin, Ming- Weig Chang. Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chupter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Pupers), pp. 4171-4186, 2019.
Emily Dinan, Angela Fan, Adina Williams, Jack Urbanek, Douwe Kiela, and Jason Weston. Queens are powerful too: Mitigating gender bias in dialogue generation, 2020.
Le Fang, Chunyuan Li, Jianfeng Gao, Wen Dong, and Changyou Chen. Implicit deep latent variable models for text generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3937-3947, 2019.
Omar U. Florez. On the unintended social bias of training language generation models with data from local media, 2019.
Hao Fu, Chunyuan Li, Xiaodong Liu, Jianfeng Gao, Asli Celikyilmaz, and Lawrence Carin. Cyclical annealing schedule: A simple approach to mitigating KL vanishing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers), pp. 240-250. Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10. 10.18653/v1/N19. 1021.
URL https:: www. a c. web. web.orr//nttho. ogy /NI 9-10

---

Under review as a conference paper at ICLR 2021
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farles Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural infor- mation processing systems, pp. 2672-6622266 2680, 2014.
Junxian He, Daniel Spokoyny, Graham Neubig, and Taylor Kirg-Kirkpatrick. Lagging inference networks and posterior collapse in variational autoencoders. In Proceedings of ICLR, 2019.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. In International Conference on Learning Representations, 2020. URL https:
7/openzeview. http//oowmidd-yggoppp
Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. Toward con- trolled generation of text. volume 70 Proceedings of Machine Learning Research, pp. 1587- 1596, International Convention Centre, Sydney, Australia, 06-11 Aug 2017. PMLR. URL http:///rroceedings mlr.
http://..........rroeooe///o//// pwwss//70/huul html
Aapo Hyvärinen and Erkki Oja. Independent component analysis: algorithms and applications.
Neural networks, 13(4-5):111-33, 2000.
Erik Härkönen, Aaron Hertzmann, Jaakko Lehtinen, and Sylvain Paris. Ganspace: Discovering interpretable gan controls, 2020.
Vineet John, Lili Mou, Hareesh Bahuleyan, and Olga Vechtomova. Disentangled representation learning for non-parallel text style transfer. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 424-434, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10. 10.18653/v1//19-1000 URL httph:ttp//// www. paclwweb
atr//anthollogy/119-1000
Nitish Shirish Keskar, Bryan McCann, Lav Varshney, Caiming Xiong, and Richard Socher. CTRL
A Conditional Transformer Language Model for Controllable Generation. arXiv preprint
arXiv:1909.05855 2019.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv:1312.61 2013.
arXiv preprint
Guillaume Lample, Sandeep Subramanian, Eric Smith, Ludovic Denoyer, Marc' Aurelio Ranzato, and Y-Lan Boureau. Multiple-attribute text rewriting. In International Conference on Learning Representations, 2019. / www.:eee/oormmiccommmm Hit.ae/eoooommiLLHHHH2HH H1ggh-C5Kg
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy. Veselin Stoyanov, and Luke Zettlemoyer. BART: Denoising sequence-to-sequence pre- training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, July 2020. Asso- ciation for Computational Linguistics.
Chunyuan Li, Xiang Gao, Yuan Li, Xiujun Li, Baolin Peng, Yizhe Zhang, and Jianfeng Gao. Opti- mus: Organizing sentences via pre-trained modeling of a latent space. In EMNLP, 2020.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach, 2019.
L. Logeswaran, H. Lee, and S. Bengio. Content preserving text generation with attribute controls. ArXiv, abs/1811 001 135, 2018
William Peebles, John Peebles, Jun-Yan Zhu, Alexei A. Efros, and Antonio Torralba. The hessian penalty: A weak prior for unsupervised disentanglement. In Proceedings of European Conference Computer Vision (ECCV), 2020.
Shrimai Prabhumoye, A. Black, and R. Salakhutdinov. Exploring controllable text generation tech- niques. ArXiv, abs/2005.01 0182 2020.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners.
10

---

Under review as a conference paper at ICLR 2021
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv e-prints, 2019.
Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. Style transfer from non-parallel text by cross-alignment. In Advances neural information processing systems, pp. 6830-6841, 2017.
Tianxiao Shen, Jonas Mueller, Regina Barzilay, and Tommi Jaakkola. Educating text autoencoders: Latent representation guidance via denoising, 2020.
Andrey Voynov and Artem Babenko. Unsupervised discovery of interpretable directions in the gan latent space, 2020.
Ke Wang. Hang Hua, and Xiaojun Wan. Controllable unsupervised text attribute transfer via edit- ing entangled latent representation. In Advances in Newral Information Processing Systems, pp- 11036-11046, 2019.
Peng Xu, Jackie Chi Kit Cheung, and Yanshuai Cao. On variational learning of controllable repre- sentations for text without supervision. In The 37th International Conference on Machine Learn- ing (ICML 2020). PMLR, July 2020.
Zichao Yang. Zhiting Hu, Ruslan Salakhutdinov, and Taylor Berg-Kirkpatrickk Improved varia- tional autoencoders for text modeling using dilated convolutions. In International Conference Machine Learning, pp. 3881-3890, 2017.
Zichao Yang, Zhiting Hu, Chris Dyer, Eric P Xing. and Taylor Berg-Kirkpatrick. Un- supervised text style transfer using language models as discriminators. In S. Ben- gio, H. Wallach, H. Larochelle, K. Grauman, N. Bisnchia and Garnett (eds.), Advances in Neural Information Processing Systems 31, Pp. 7287-7298. Curran Associates, Inc., 2018. URL http::/papers ni nips. cctppacapppseccpppepppppppppppp. 7959 1959-unsupervine wtw.rrewww...ttttttttttaaeeeerer pdf
Qile Zhu, Wei Bi, Xiaojiang Liu, Xiyao Ma, Xiaolin Li, and Dapeng Wu. A batch normalized inference network keeps the KL. vanishing away. In Proceedings of the 58th Anmual Meeting of the Association for Computational Linguistics, pp. 2636-2643 Online, July 2020. Association for Computational Linguistics. doi: 10.18653/w1/220... acl- URL https://
acl solweb.cc org/ antho aetthology20000 aacl 2ttp:/000....lmmaa..222...
A VARIANCE EXPLAINED BY PRINCIPAL COMPONENTS
To verify that small number of directions found by our method can explain the large proportion of variance in SNLI, plot the cumulative explained variance ratio for each coordinate, ordered by explained variance in decreasing order. It can be seen that 70 latent directions are responsible for most of the variance in the training dataset: given good reconstruction capabilities of OPTIMUS, this means that almost all sentences of SNLI can be described terms of these 70 orthogonal vectors.
B HUMAN EVALUATION INSTRUCTION
B. TASK DESCRIPTION
In each assignment, you will see 5 sentences with a certain textual characteristic (or attribute) changed by the model. There degrees of change of different intensity on the scale of "_" to "++", the unchanged sentence is given in the middle.
1. You need to answer whether it is possible to interpret the changing attribute from given amples. If the atribute is partially interpretable (for instance, because the nature of changes is the same in only 2 of sentences), you should also state this.
11

---

Under review as a conference paper at ICLR 2021
Cumulative explained variunee
0.8
0.6
0.4
0.2
200
400
600
768
Coordinate index
Figure Cumulative explained variance with increasing the number components.
2. If you can interpret the attribute (even partially), you need to assign to one of the attribute categories specify your own. Do not hesitate give your name to the attribute in the corresponding field: the model might change sentence properties that are not given in the list or do not fit the provided categories
3. If attribute interpretation was challenging or only partial, choose the option "Partially inter- pretable" and give the exact reason: whether this because of several properties changing at once, one property being interpretable in only 1-2 sentences out of 5, or too drastic content changes. You can also explain partial interpretability using the text field.
4. If the attribute is not interpretable because you cannot unambiguously formulate its essence so that it would fit all sentences, you should choose the option "No". changing this attribute leads to sentences becoming meaningless character sequences repetitions of the same words, you should choose the option "Incorrect text""
B.2 EXAMPLES WITH DIFFERENT INTERPRETABILITY
Example 1 (correct changes the attribute):
there is man celebrating
there is man celebrating
there is a man celebrating with friends
there is man celebrating with friends and celebrating in front of a red building
++ there is man celebrating with friends celebrating with red
flag about to celebrate with other peoples in <unk>
a man is walking.
a man in jeans runs.
a man in jeans runs through a park.
a man in jeans runs through a park through jean shorts.
++ a man in jeans jeans runs through a park in a gray t-shirt through woods with runners past.
In this case, for both sentences changing the attribute leads only to changes in the sentence length, so the appropriate answer is "Yes".
Example 2 (content not preserved):
In the aftermath of the war, the local population of the area, including the local inhabitants of the area, who had been evacuated from the area by the Germans.
Despite this sentence, the option of custom attribute type mostly unused by all annotators.
12

---

Under review as a conference paper at ICLR 2021
In the aftermath of the waz, the "Carnivores" were sent to the "Carnivores" camp in the "Carnivores" area, where they were killed.
The remaining members of the "Cavaliers" were killed in the battle, and the remaining members of the "Cavaliers" killed in the battle.
The remaining were:
++
In the early 1990s, after series of interviews with the author of "The Adventures of Dr. John D. MacDonald", who had been diagnosed with Parkinson's disease, and his wi wife, Dr. Louise MacDonald,
In "The Adventures of Doctor Who", he was introduced young man who was sent to the Earth by the Doctor.
She played by John Hurt in the first film, "The Man Who Wasn't Born".
She is played by John Reilly.
++
In addition to her high school education, she received $1, 000 grant from the state New York to provide a $1, 00,000 grant to the New York State Department of Education
In addition to her high school diploma, she received a bachelor's degree in business administration from the University of Michigan, and a master's s degree in business administration from the University of Michigan
She earned a bachelor's s degree in business administration, and a master's degree in business administration. 5.
In this case the sentence length also changes, but it also leads to drastic change in content (it is not possible to say that all sentences were obtained from the one in the middle by adding removing words). In such cases, the expected answer is "Partially" and the reason is "Too drastic content changes".
Example 3 (several attributes are changed at a time):
a couple are embracing.
two people are embracing.
two people are embracing each other.
two people are embracing each other across the surface.
++ two humans glide across each other as each other are embraced by the other.
couple with umbrella
2 people with a umbrella on a beach
3 people with umbrellas on street next a park
3 people with umbrellas on a street next to some pedestrians outside a park
++ 5 pedestrians all on green spots along side a cur with street in front of some pedestrians in an area of nyc
two men and is a birthday.
two men and three children are at the beach.
two men and three children are at the beach.
two men and three children are at the beach three.
++ three men and three children all sit at the side of the road in green and yellow.
13

---

Under review as a conference paper at ICLR 2021
In this case, the meaning is mostly preserved, but there two changes the time: sentence length and the number of subjects. Because this, you should choose "Partially interpretable More than attribute changes".
Example 4 (non-interpretable changes):
three girls watch ballet dance. four women watch a ballet practice from their chair. four women watch ballet practice from their chairs. four women watch a ballet from their practice chairs. women watch ballet during their breaks at the computers.
young boy using a vacuum cleaner. young boy using a vacuum cleaner on rug. young boy using a vacuum cleaner on rug. young boy using a vacuum cleaner on a rug. dry rug.
+ young boy using a vacuum
a boy is jumping from a plank.
a boy jumps from plank high.
a boy jumps from a plank up high.
a boy jumps from a plank up high.
++ a boy jumps from a plank up over the water.
Here (at least, for the task authors) it is too hard to briefly explain the kind of changes, hence the answer should be "No".
Example 5 (incorrect text):
"The Merry Christmas" was signed by the London Company and sold to the Royal Opera Company in 1885 for 000, and was the first of the "Merry Christmas" to be sold to the Royal Opera Company.
The casualties were estimated at 1, 000, 000 men, and the British were estinated at 100, 000.
There are casualties.
++
In this case significant changes to the attribute ("-" and "++") lead to the sentence longer being readable. You may choose "The text incorrect" if it is impossible to understand the interpretability of attribute in this case.
B.3 ATTRIBUTE TYPES
Each attribute may be connected with simple sentence features (e.g length), presence of certain words in the sentence with sentence sentiment. You can examples of different attributes below:
Length:
3 people are outside
3 people with a umbrella on street
street next to a park 3 umbrellas with a person on top of a street for umbrellas outside park
3 people with umbrellas
14

---

Under review as a conference paper at ICLR 2021
++ 3 unbrellas with umbrellas street corner with lot of people next to a green parka with a street sign on it in front of it for an area of city in asian city
Singular/plural number:
worker standing
high scaffold.
a worker atanding on a high scaffold. worker standing
high scaffold.
a worker standing on high scaffolding.
++ several workers standing high on the scaffold.
Changing the order of words in a sentence (also, the number is changed);
the are standing and reading newspaper.
the man is standing and reading a newspaper.
a man is standing and reading a newspaper.
a man is standing and reading in a newspaper.
is standing and running in newspaper.
the naked men are holding the drum
the naked man is playing drums.
the naked man is playing drums. a naked man is playing drums. a naked man is playing in wate Adding the word "in":
a man runs through jeans.
a man in jeans runs through a park. a man jeans runs through a park. man in jeans runs through park. a man in jeans runs in a park.
C ADDITIONAL TABLES FROM THE HUMAN STUDY
Here, we provide full data the results of the interpretability evaluation without aggregating results over different interpretability categories (Table or different models for direction types (Table 4
D EXAMPLES OF FOUND DIRECTIONS
Below we give additional examples of latent directions discovered by our method on the SNLI dataset (8 0);
1. Basic sentence attributes:
children are playing a game.
children are playing game.
children are playing a game together.
children are playing game together as ball game. ++ children playing game of board war together are playing game of snowball together after birthday.
man is looking down
a fireman is looking down. fireman is looking down at the ground. a fireman looking down at the ground is still raining down.
++ a firetruck driver looking down at the ground about to fall down from the ground is still burning up smoke.
15

---

Under review as a conference paper at ICLR 2021
Method
Model
Interpretable?
Proportion
Coordinate
sali_betaO
0.80
yes partial seli_ no partial yes wiki not coherent
0.15
0.05
0.85
0.10
0.05
0.90
no
0.10
PCA
snli_beta0 partial no sali_betal yes
0.45
0.40
0.15
0.60
yes partial wiki no not_coherent
0.25
0.15
0.80
0.15
partial
0.05
Random snli
0.80
partial yes no partial yes not.coherent
0.15
0.05
snli beti_betal
0.70
0.20
0.10
0.90
no
0.10
Table 3: Evaluation data for all interpretability categories
2. Word-level attributes:
Singular/plural noun:
family at a wedding.
family at a wedding.
family at a wedding.
family at a wedding.
++ families at wedding.
person performing a a guitar for a little crowd. band performing for a little crowd. band performing for a little crowd. the band performing for some big crowd. ++ the bands performing just for friends.
child is playing game ball in room. children are playing a game together. children are playing game together. children are playing a game together.
children are playing game together. ++
Age:
the man is is important.
the man is athletic.
the man is athletic.
the man is athletic.
++
the athletic boy.
a woman is sitting at her positions to practice their positions.
16

---

Under review as a conference paper at ICLR 2021
Method
Model
Direction type
Proportion
Coordinate
beti
basic-properties
0.86
wwwpgeesoojcc.propeeee insert_word _ord
0.07
0.07
snli
basic_ pespcpproperties changes_obye specific_structtre
0,44
0.38
0.12
insert word
0.06
PCA
snli_betaO
changes changes_object.ppn
abject_properticc
0.68
prope_propesties
0.16
specific structure insert word
0.11
0.05
sali
bbangess.bjeet.proprriis
0.47
basic-properties
0.37
insert_word
0.1
specific_strucctre specific strrccure
0,05
wiki
0.80
properpropesties
0.20
Random
buli_betaO
changes changes_ebject__pooprtis
0.53
basic. prope-propssties
0.29
ppecific_structuie
0.12
insert_word word
0,06
snli betal
basic properties changes object properties
0.58
0.17
inse
0.17
specific
0.08
wiki
basic_propertic
0.50
specific_struucure
0.50
Table 4: Evaluation data for all direction types, grouped by model and method
woman is practicing her positions at an yoga center.
a woman is practicing her yoga positions.
female is practicing her yoga positions.
++ a yoga girl practicing her knees.
the woman woman patting volleyball, the women are playing volleyball in the park.
the women are playing volleyball in the park.
the women are playing volleyball in the park.
++ the girls are playing volleyball in the park. Genderieutallgeedersppiiff::
children are playing a game together. children are playing a game together. children are playing a game together. children are playing game together.
++ boys are play game together.
person is practicing her standing muscles. woman is practicing her positions yoga. woman is practicing her yoga positions. woman is practicing her yoga positions. ++ two women practices her yoga positions.
3. Insertion of particular words:
"In":
children are playing game to celebrate. children are playing a game together. children are playing game together.
17

---

Under review as a conference paper at ICLR 2021
children are playing game together. ++ two children in game are sitting together.
the women are practicing volleyball playing volleyball.
the women are playing volleyball during the beach.
the wonen are playing volleyball in the park.
the women are playing volleyball in the park.
++
the two women in volleybal1 are sitting in a park.
"Man":
children are playing a game in full. children are playing game together. children are playing a game together.
two childres are playing game together.
++ two men are playing a game together.
person pouring cup of concrete sidewalk in window display.
person pouring wheelbarrow of cement a man pouring a wheelbarrow of cement on a sidewalk. pulling wheelbarrow of cement his bicycle. man pulling a wheelbarrow of two men and hanmer on the rail.
sidewalk.
++
the person is athletic.
the man is athletic.
the man is athletic.
the man is athletic.
++ the man and his man are athletic. 4. Enforcing specific structure:
"A B":
children playing game together outside house. children are playing a game together. children are playing
game together.
+ children are playing a game together.
++ the child is game having winning.
women playing volleyball in the park. several women are playing volleyball in the park. the women are playing volleyball in the park. the women are playing volleyball in the park. ++ the woman is the volleyball player.
iA C"
the women are playing the volleyball bar. the women are playing volleyball in the park.
the women are playing volleyball in the park. the woman are playing volleyball ++ woman is playing volleyball in city.
the park.
the fireman are looking down the table. the fireman is looking down the ground. fireman is looking down at the ground. firenan is looking down ground.
++ firefighter is looking down in fire.
18


Reviews:
[{"review_id": "hmHHN99TBv", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The author propose to use PCA-like method on latent space of VAE models to unsupervisedly detect interpretable direction. The idea is reasonable and practically useful for large-scale pretrained VAE model, i.e. OPTIMUS. This paper has a clear idea and a thorough discussion with related works. \n\nI have some concerns about the model. The proposed model seems requiring a large-scale pretrained model. If the VAE model is just trained on SNIL level, is method still valid?  From the PCA side, it does not require a Gaussian space. So why specifically targeting on VAE model, not just AE model is another confusion. Since the direction is computed based on training data, I kind of feeling of no need of using VAE model.", "extra_scores": {}, "extra_text": {}}, {"review_id": "ZF1DUm4mdXI", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "-------------------\nSummary\n-------------------\nThis paper proposes a simple approach to discover interpretable latent manipulations in trained text VAEs. The method essentially involves performing PCA on the latent representations to find directions that maximize variance. The authors argue that this results in more interpretable directions. The method is applied on top of a VAE model (OPTIMUS), and the authors argue that different directions discovered by PCA correspond to interpretable concepts.\n\n-------------------\nStrengths\n-------------------\n- The method is simple, and can be applied on top of existing text VAEs.\n- Learning interpretable and controllable generative models of text is an important research area, and this paper contributes to this important field.\n\n-------------------\nWeaknesses\n-------------------\n- There are only mostly qualitative results presented. While I agree that performing quantitative results is difficult with this style of work, the authors could have (for example) adopted methods from the style transfer literature to show quantitative results. These metrics include perplexity (to see how fluent the generations are), reverse perplexity, and style transfer accuracy (this may not be applicable since there is no ground truth \"style\" in this work, but the ground truth style could be heuristically defined for some transformations, e.g. for singular/plural transformations).\n- Human evaluation seems nonideal since it is only tested on 12 people.\n- The generations are actually not so good in my opinion? E.g. many of the generations in the appendix are ungrammatical and/or semantically nonsensical.  Again, metrics such as perplexity could quantify the fluency of generated text.\n- The method is only applied to one text VAE mode which specifically uses BERT/GPT-2 , so it is not clear if this will generalize to other models (e.g. models trained from scratch).\n\n-------------------\nQuestions/Comments\n-------------------\n- In Figure 2, are these the top 4 principal directions? If not, how were these directions discovered?\n- \"It is known that variational autoencoders trained with a schedule for the KL weight parameter (equation 1) obtain disentangled representations (Higgins et al., 2016; Sikka et al., 2019; John et al., 2019). Since OPTIMUS is also trained with KL annealing, canonical coordinates in its latent space are likely to be disentangled.\" I believe this is only valid for beta > 1 so it is not really applicable here.\n-----------------------\nEdit after rebuttal: Thank you for the rebuttal and clarifying some of my questions. I have decided to keep the original score.", "extra_scores": {}, "extra_text": {}}, {"review_id": "YA_fvk0vmP0", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a PCA-based latent variable language model for unsupervised latent variable interpretation.\n\nPros:\n1. The authors propose to use PCA to extract the principal components of the results and claim them to be interpretable latent variables.\n\nCons:\n1. The novelty is quite limited. Applying an existing well-known technique to obtain interpretable latent variables is not advancing this domain in the right direction.\n2. The explanation of latent variable in this paper is self-justified. The self-defined baselines cannot be convincingly conveyed that latent variable are interpreted. And the baselines are quite weak.\n3. In the quality evaluation, the authors do not show how clearly to modify the discovered latent variable to alter the sentences.\n\nQuestion:\n1. How do you encode a sentence in a two-dimensional space? Are both dimension probability?\n2. Other than the current quantitative and qualitative analysis, do you think any other quantitative evaluation will be helpful?", "extra_scores": {}, "extra_text": {}}, {"review_id": "MWu4hOJKhcx", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper studies latent manipulations in text autoencoders. The authors propose that compared to random and coordinate directions, moving in the PCA directions of encodings of training examples will produce more interpretable text manipulations.\n\nAs the idea is straightforward, I'd like to see more in-depth analysis and more solid evaluations. The authors characterize the effects of PCA directions into four types (length, word change, word insertion, and structure enforcement), but for each type only one example is provided. What are the changed/inserted words and what are the enforced structures? Can you give a comprehensive list of them? When are these latent directions applicable and when are they not? For sentences that are not applicable, what effects will they bring?\n\nThe only evaluation in the paper is human evaluation of whether a latent direction shift produces interpretable generations. It's conducted on 20 sentences, which is too small to draw any conclusions. The results on the Wikipedia dataset are very poor. You may test the success rate of manipulations in a specific direction (such as word insertion) through automatic evaluation. This can also reveal which manipulations are easier to implement and which are more difficult.\n\nI think with these changes, the paper will be more substantial, instead of spending 4 out of 8 pages on the background like in the current submission. Also, it's more suitable for NLP conferences than ICLR.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a simple method to discover latent manipulations in trained text VAEs. Compared to random and coordinate directions, the authors found that by performing PCA on the latent code to find directions that maximize variance, more interpretable text manipulations can be achieved. 

This paper receives 4 reject recommendations with an average score of 3.75. The reviewers have raised many concerns regarding the paper. (i) The idea is straightforward with limited novelty. (ii) There are only mostly qualitative results presented. More in-depth analysis and more solid evaluations are needed. (iii) Human evaluation is too small to draw any reliable conclusion. (iv) The proposed method is only tested on one text VAE, how well it can be generalized to other models remains unclear.

The rebuttal unfortunately did not address the reviewers' main concerns. Therefore, the AC regrets that the paper cannot be recommended for acceptance at this time. The authors are encouraged to consider the reviewers' comments when revising the paper for submission elsewhere. 
Decision: Reject