Title: Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks
Authors: Yossi Adi
Einat Kermany
Yonatan Belinkov
Ofer Lavi
Yoav Goldberg
Venue: ICLR 2017
Keywords: Natural language processing
Deep learning

Abstract: There is a lot of research interest in encoding variable length sentences into fixed
length vectors, in a way that preserves the sentence meanings. Two common
methods include representations based on averaging word vectors, and representations based on the hidden states of recurrent neural networks such as LSTMs.
The sentence vectors are used as features for subsequent machine learning tasks
or for pre-training in the context of deep learning. However, not much is known
about the properties that are encoded in these sentence representations and about
the language information they capture.
We propose a framework that facilitates better understanding of the encoded representations. We define prediction tasks around isolated aspects of sentence structure (namely sentence length, word content, and word order), and score representations by the ability to train a classifier to solve each prediction task when
using the representation as input. We demonstrate the potential contribution of the
approach by analyzing different sentence representation mechanisms. The analysis sheds light on the relative strengths of different sentence embedding methods with respect to these low level prediction tasks, and on the effect of the encoded
vector’s dimensionality on the resulting representations.

TL;DR: A method for analyzing sentence embeddings on a fine-grained level using auxiliary prediction tasks

Review discussion:
– Could you explain your motivation for testing your models only on their training examples? You explain it briefly at the start of 3.1, but I can't reconstruct the argument.
– Have you done significance testing of any kind on any of your results? The slight dip in ED performance in Figure 1a, for example, look like it could be a random fluctuation, but you mention it in your summary of the results.
– What do you think might cause the massive drop in performance for CBOW in Figure 1b?
Hey,
Thanks for the comment.
- Regarding the motivation for testing on training examples, in this work we are interested in analyzing to what extent a model can encode basic properties of sentences (content, order, length).
Hence, we wanted to reduce all the other “noise” and analyze the models on their “ideal environment”: encoding the sentences they were trained on encoding. This gives us an upper-bound on the model’s tendency to encode such values in its representation, without mixing in the effects of generalization.
It is true that additional experiments on unobserved sentences are also of interest, but we believe that the current setup is the more informative one. Note that we do also have experiments in which we encode sentences that the models were not trained on (all of the synthetic examples in which we used permuted sentences), and we observe similar behaviors.
- Yes, we did significance tests (using paired t-test and Wilcoxon test), and for all the results that we report in the summery of findings the score differences are all highly significant (p << 0.0001). We updated the appendix to include the paired t-test p-values for the pairs we tested.
- We are also puzzled by this, but do not currently have a good explanation. If you have convincing hypotheses we’ll be happy to test them.
Often it's desirable that a representation have some invariance to irrelevant changes in the input -- skip-thought vectors in particular aim to factor out certain types of phrasing variations that are irrelevant to prediction of surrounding context. In this sense, the ability to exactly reconstruct the input sentence could be seen as a detriment for a representation, since it is keeping around information that may not be helpful for future tasks. In this work, the ability to perfectly encode content seems to be presented as a mostly unqualified win. Have the authors considered auxiliary tasks to measure this sort of beneficial invariance to e.g. paraphrasing?
I'm very surprised that the auxiliary classifiers can not be overfit on the training set since they are so nonlinear, especially when training on e.g. skip-thoughts which do not explicitly throw away any information -- can the authors give some intuition as to why this is the case?
Hey,
Thanks for the comment.
We agree that quantifying generalization ability more directly is important and interesting, but we also find it beyond the scope of this work.
However, we would like to stress that we do not view higher scores for any of the tasks as reflecting of the corresponding model being *better*. A high score on an aux task for a given model implies in our view only that this model encodes more of the given property. If this is not a desired behavior for your end task, then take the high score as an indicator of a bad thing. We do note that both skip-thoughts and ED with dimension size 1000 achieve very similar results on the word content task.
We’re not sure we understood the overfitting question, but we try to answer the best we can. In order to avoid overfitting, we used a dropout layer after the non-linear function with a high dropout rate of 0.8 (smaller dropout rates indeed overfit), and implemented an early stopping criterion, meaning that we stopped training after 5 epochs in which we observed no loss improvement on a dedicated validation set.
Figure 2(b) shows the norm of CBOW embeddings decreasing very smoothly as the sentence length increases. Have you tried quantifying your explanation of this behavior based on the central limit theorem? If so, how well does it fit the curve?
What does the analogue of Figure 2(b) look like for the ED embeddings?
Hey,
Thanks for the comment.
We are not sure what you mean by “quantifying our explanation”, but we did perform two additional experiments in which we simulate the CBOW case using random vectors. We created 50k synthetic “sentences” of different lengths, using the average of random vectors, first from a uniform distribution in (-1,1) and second from a normal distribution with N(0,1).
Then, we plot the average norm of these vectors the same way we did in the CBOW case and both of the curves are very similar to the curve we got while using CBOW embeddings.
In the case of ED models, instead of decreasing, the norm is increasing as the sentence length grows higher. This makes intuitive sense as an RNN accumulates more information as it processes the sentence, and does not have the averaging property of the CBOW representation. Justifying this theoretically is less straightforward.
This paper presents a set of experiments investigating what kinds of information are captured in common unsupervised approaches to sentence representation learning. The results are non-trivial and somewhat surprising.
For example, they show that it is possible to reconstruct word order from bag of words representations, and they show that LSTM sentence autoencoders encode interpretable features even for randomly permuted nonsense sentences.
Effective unsupervised sentence representation learning is an important and largely unsolved problem in NLP, and this kind of work seems like it should be straightforwardly helpful towards that end.
In addition, the experimental paradigm presented here is likely more broadly applicable to a range of representation learning systems. Some of the results seem somewhat strange, but I see no major technical concerns, and think that that they are informative. I recommend acceptance.
One minor red flag:
- The massive drop in CBOW performance in Figures 1b and 4b are not explained, and seem implausible enough to warrant serious further investigation. Can you be absolutely certain that those results would appear with a different codebase and different random seed implementing the same model? Fortunately, this point is largely orthogonal to the major results of the paper.
Two writing comments:
- I agree that the results with word order and CBOW are surprising, but I think it's slightly misleading to say that CBOW is predictive of word order. It doesn't represent word order at all, but it's possible to probabilistically reconstruct word order from the information that it does encode.
- Saying that "LSTM auto-encoders are more effective at encoding word order than word content" doesn't really make sense. These two quantities aren't comparable.
Hey,
Thanks for the positive review,
Re. CBOW drop: we tried different configurations, different seeds and different hyper parameters, and the results still hold. You are right that it is surprising but we don't have a good explanation for it at the moment. We will be happy to discuss new ideas on how to investigate this.
Re. CBOW encoding order, and the comparison of order vs content: you are right, we changed our wording to be more precise about these issues and updated the paper.
This paper analyzes various unsupervised sentence embedding approaches by means of a set of auxiliary prediction tasks. By examining how well classifiers can predict word order, word content, and sentence length, the authors aim to assess how much and what type of information is captured by the different embedding models. The main focus is on a comparison between and encoder-decoder model (ED) and a permutation-invariant model, CBOW. (There is also an analysis of skip-thought vectors, but since it was trained on a different corpus it is hard to compare).
There are several interesting and perhaps counter-intuitive results that emerge from this analysis and the authors do a nice job of examining those results and, for the most part, explaining them.
However, I found the discussion of the word-order experiment rather unsatisfying. It seems to me that the appropriate question should have been something like, 'How well does model X do compared to the theoretical upper bound which can be deduced from natural language statistics?' This is investigated from one angle in Section 7, but I would have preferred to the effect of natural language statistics discussed up front rather than presented as the explanation to a 'surprising' observation. I had a similar reaction to the word-order experiments.
Most of the interesting results, in my opinion, are about the ED model. It is fascinating that the LSTM encoder does not seem to rely on natural-language ordering statistics -- it seems like doing so should be a big win in terms of per-parameter expressivity. I also think that it's strange that word content accuracy begins to drop for high-dimensional embeddings. I suppose this could be investigated by handicapping the decoder.
Overall, this is a very nice paper investigating some aspects of the information content stored in various types of sentence embeddings. I recommend acceptance.
Hey,
Thanks for the positive review,
We agree that more analysis of the effect of natural language properties in the encoded sentences is important
We will further explore it in future work.
The authors present a methodology for analyzing sentence embedding techniques by checking how much the embeddings preserve information about sentence length, word content, and word order. They examine several popular embedding methods including autoencoding LSTMs, averaged word vectors, and skip-thought vectors. The experiments are thorough and provide interesting insights into the representational power of common sentence embedding strategies, such as the fact that word ordering is surprisingly low-entropy conditioned on word content.
Exploring what sort of information is encoded in representation learning methods for NLP is an important and under-researched area.
For example, the tide of word-embeddings research was mostly stemmed after a thread of careful experimental results showing most embeddings to be essentially equivalent, culminating in "Improving Distributional Similarity with Lessons Learned from Word Embeddings" by Levy, Goldberg, and Dagan. As representation learning becomes even more important in NLP this sort of research will be even more important.
While this paper makes a valuable contribution in setting out and exploring a methodology for evaluating sentence embeddings, the evaluations themselves are quite simple and do not necessarily correlate with real-world desiderata for sentence embeddings (as the authors note in other comments, performance on these tasks is not a normative measure of embedding quality).
For example, as the authors note, the ability of the averaged vector to encode sentence length is trivially to be expected given the central limit theorem (or more accurately, concentration inequalities like Hoeffding's inequality).
The word-order experiments were interesting. A relevant citation for this sort of conditional ordering procedure is "Generating Text with Recurrent Neural Networks" by Sutskever, Martens, and Hinton, who refer to the conversion of a bag of words into a sentence as "debagging."
Although this is just a first step in better understanding of sentence embeddings, it is an important one and I recommend this paper for publication.
Hey,
Thanks for the positive review and helpful comments,
We’ve added the missing references and updated the paper.

Meta-review:
role: PC
data: comment: The area chair agrees with the reviewers and think this paper would be of interest to the ICLR audience. There is clearly more to be done in this area, but the authors do a good job shedding some light on what sentence embeddings can encode. We need more work like this that helps us understand what neural networks can model.

Decision: Accept (Poster)