Year: 2018

Abstract:
Sentence encoders are typically trained on generative language modeling tasks with large unlabeled datasets. While these encoders achieve strong results on many sentence-level tasks, they are difficult to train with long training cycles. We introduce fake sentence detection as a new discriminative training task for learning sentence encoders. We automatically generate fake sentences by corrupting original sentences from a source collection and train the encoders to produce representations that are effective at detecting fake sentences. This binary classification task turns to be quite efficient for training sentence encoders. We compare a basic BiLSTM encoder trained on this task with strong sentence encoding models (Skip-thought and FastSent) trained on a language modeling task. We find that the BiLSTM trains much faster on fake sentence detection (20 hours instead of weeks) using smaller amounts of data (1M instead of 64M sentences). Further analysis shows the learned representations also capture many syntactic and semantic properties expected from good sentence representations.

heading: 1 INTRODUCTION
text: Universal sentence encoding is a way to scale language processing tasks, especially when the target training data is limited. Solutions to sentence-level language processing tasks can be seen as consisting of two parts – one that creates a generic representation which approximates the meaning of the sentence, and another that uses this representations to make the target classifications. The idea behind universal sentence encoding is to learn generic sentence representations from unlabeled texts, which are often much larger and easier to obtain than the training data for the target tasks. A good universal encoding is one which is effective for training downstream target tasks.
The success of language modeling ideas for learning word representations has inspired similar ideas for learning universal sentence representations. Skip-thought model (Kiros et al., 2015) uses a Bidirectional LSTM (BiLSTM) encoder, which encodes a given sentence, and uses the encoding to generate neighboring sentences. Following up on this general idea, the FastSent model (Hill et al., 2016) simplifies the task and the encoder further by learning to output the average word embeddings so that they can predict words in the neighboring sentences (but not necessarily in a sequence).
This language modeling based training is undesirable in two respects. (1) Predicting neighboring sentences is a difficult generative training task. The task requires having a large output space, at least the size of the vocabulary. It also requires a complex decoding model such as an LSTM decoder with millions of parameters. Effective training requires a large amount of training data with long training cycles. Indeed Skip-thought requires tens of millions of training sentences and it takes more than a week to train. (2) Even minor changes to a sentence can drastically change its meaning, a phenomenon that needs to be modeled for many NLP tasks. Language model based training essentially relies on the training sample for this kind of generalization, i.e., the training text collection should include many instances of sentences that have only minor lexical differences but found in completely different contexts. Larger the dataset the more likely it is for the encoder to learn how to model these fine grained distinctions.
In this work, we introduce a discriminative training task fake sentence detection to address these challenges. The main idea is to generate fake sentences by corrupting an original sentence. We use two methods to generate fake sentences: word shuffling where we swap the positions of two words at random and word dropping, where we drop a word at random from the original sentence.
The resulting fake sentences are mostly similar to the original sentences—a fake sentence differs from its source in at most two word positions. We create the training corpus from a source corpus of unlabeled sentences. For each sentence in the source corpus we add multiple fake sentences by corrupting the source sentence.
This training task has three key advantages: (1) This binary classification task can be modeled with fewer parameters in the output layer and can be trained more efficiently compared to language modeling based training tasks. (2) From a language standpoint, the task forces the encoder to track both syntax and semantics. For instance, swapping words within a sentence can break the syntax (e.g., “John landed in Chicago on Friday” versus “John landed Chicago in on Friday”), and break or alter the semantics; it can lead to an incoherent or less plausible sentence (e.g., “John landed in Chicago on Friday” versus “Chicago landed in John on Friday”). (3) The task explicitly forces the encoder to capture big shifts in meaning that arise from small changes to a sentence. In particular, since the discrimination is really dependent on a small change to an original sentence, encoders cannot get away with simple aggregation—they need to model compositional aspects well enough to be able to detect small but semantically shifts in sentence constructions.
In overview, we train a bidirectional LSTM (BiLSTM) as our universal encoder and use a threelayer feed-forward network that uses the encoded sentence to predict if it is fake or real. We then evaluate this trained encoder without any further tuning on multiple sentence-level tasks and use probing tasks (Conneau et al., 2018) to study the syntactic and semantic properties in the encoded representations.
In summary, this paper makes the following main contributions:
1. Introduces fake sentence detection as an unsupervised training task for learning sentence encoders that can distinguish between small changes in mostly similar sentences.
2. Provides an empirical evaluation on multiple sentence-level tasks showing representations trained on the fake sentence tasks outperform a strong baseline model trained on language modeling tasks, even when training on small amounts of data (1M vs. 64M sentences) reducing training time from weeks to within 20 hours.
3. Demonstrates that the fake sentence training produces encoded representations that are better at capturing many linguistic properties compared to language model training.

heading: 2 MOTIVATION
text: Many sentence-level prediction tasks require learning a function to estimate the probability of a label y given a sentence x. This function can be parameterized as P (y|x, θ), and the parameter vector θ can be learned by minimizing the negative conditional log likelihood of the training data D, i.e., minθ − ∑ (x,y)∈D log(P (y|x, θ)). However, the amount of training data is limited in many cases, so learning θ by optimizing the conditional probability leads to overfitting, especially if one uses a deep neural network with millions of parameters.
One approach to address this problem is to break the task into two sub-tasks each with its own set of parameters: (1) sentence encoding—produce an effective low-dimensional representation of the sentence enc(x, θ1), (2) target prediction—estimate probability of the label y conditioned on the encoding P (y|x, enc(x, θ1), θ2). The dimension of θ2 is much smaller than the dimension of θ and it can be learned with the provided labeled training data D. For the encoder function enc(x, θ1), it is hoped that the parameter vector θ1 can be learned or pre-trained with a different sentence-level task using other data, which is possibly unlabeled, but can be easily collected in large quantity. The encoder can be trained on the auxiliary task as part of a prediction function f which has its own set of parameters θ3. The encoder parameters θ1 and the auxiliary task parameters θ3 are learned to minimize the loss on the auxiliary task: ∑ x∈U Laux(f(enc(x, θ1), θ3)).
In this work, we are interested in learning an universal sentence encoder that can be subsequently used for various downstream sentence-level tasks. One reasonable choice for the auxiliary loss function would be the negative log likelihood of the data, i.e., − logP (x|θ1, θ3) or equivalently − log(P (enc(x, θ1)|θ1, θ3)). Recall for the downstream task, we will seek the parameter vector θ2 to minimize the negative conditional log likelihood − log(P (y|enc(x, θ1), θ1, θ2)). The combination of two loss functions is: − log(P (y|enc(x, θ1), θ1, θ2)) − log(P (enc(x, θ1)|θ1, θ3)). This is
equivalent to − log(P (y, enc(x, θ1)|θ1, θ2, θ3)). In other words, we are maximizing the joint probability of observing the label y and the encoding vector enc(x, θ1). Optimizing the joint probability of the label and data corresponds to having a generative classifier, as opposed to the discriminative classifier that optimizes a conditional probability. As shown by (Ng & Jordan, 2002), a generative classifier converges faster to its asymptotic behavior than a discriminative classifier does. A generative approach is particularly useful when there is limited amount of training data. Thus, we propose to learn a sentence encoder by maximizing the data likelihood P (enc(x, θ1)|θ1, θ3), and we expect it to be an useful universal sentence encoder, especially when there is limited amount of labeled training data for downstream sentence-level task.
It is surprisingly simple to learn an encoder to maximize the data likelihood P (enc(x, θ1)|θ1, θ3), and this paper proposes exactly that. We can parameterize this probability using a simple function such as a log-linear model:
P (enc(x, θ1)|θ1, θ3) = 1
Z exp(−θT3 enc(x, θ1)) (1)
where the functional form of the encoder enc is an BiLSTM, and Z is a normalization constant to ensure a valid probability function.
Z = ∑ x exp(−θT3 enc(x, θ1)) = ∑ x∈U exp(−θT3 enc(x, θ1)) + ∑ x∈V exp(−θT3 enc(x, θ1)). (2)
where U is the set of real sentences, and V is the set of fake sentences. Our objective is to learn θ1, θ3 to maximize the quantity in Equation 1. This is equivalent to maximizing∑ x∈U exp(−θT3 enc(x, θ1)) while minimizing ∑ x∈V exp(−θT3 enc(x, θ1)). This can simply be done by training a classifier to separate real sentences from fake sentences.
In this work, we approximate the space of fake sentences by randomly swapping or dropping words from real sentences. Our method has some connection to the GAN framework (Goodfellow et al., 2014), where there is a generator that attempts to generate realistic data and a discriminator that distinguishes between real and generated data instances. As shown by Goodfellow et al. (2014), this approach will converge to a solution where the output of the discriminator is the estimated value for a data point to be real. In our case, we do not need to learn the generator and the discriminator in an adversarial manner because we already have a ‘strong’ generator that generates highly realistic sentences, simply by shuffling or dropping words. Furthermore, our method can be thought as a generative model, defined and trained using a discriminative function. This aspect is related to the recently proposed Introspective Neural Networks (Lazarow et al., 2017).
There are some similarities between the proposed method and Skip-thought sentence encoder. The training task of Skip-thought involves predicting the next or previous sentence z that follows or precedes the current encoded sentence x. This sentence language model is realized using an LSTM decoder, which sequentially generates a word at each position i conditioning on the previous encoded sentence enc(x), and the words that have been decoded so far (z1, · · · , zi−1). This model can be seen as maximizing the conditional probability:
P (z|enc(x, θ1), θ3) = |z|−1∏ i=1 P (zi|z1:i−1, enc(x, θ1), θ3). (3)
Interestingly, both Skip-thought and our proposed method model the unlabeled data U , a sequence of sentences u1, u2, · · · , um, and aim to optimize for the joint probability distribution: P (U|θ1, θ3). In the case of Skip-thought, there is an underlying Markovian assumption that the probability of a sentence only depends on the previous sentence. That is to approximate P (U|θ1, θ3) by∏ k P (u
k|enc(uk−1, θ1), θ3). In the case of the proposed method, we assume the sentences are independent of one another, so P (U|θ1, θ3) can be approximated as ∏ k P (enc(u
k, θ1)|θ1, θ3). Both methods are generative models of the data, and both are useful sentence encoders for the downstream tasks, as will be seen in the experiment sections.
However, Skip-thought model has several disadvantages compared to the proposed method: (1) The architecture of Skip-thought model is much more complex. Skip-thought model requires an LSTM decoder with millions of parameters. Furthermore, the need for computing the probability of individual words requires having a large output space, as large as the size of the vocabulary.
This further increases then number of parameters and the complexity of the overall architecture. (2) The modeling task of Skip-thought is much harder than that of the proposed method. Skipthought requires a probability model for both complete and incomplete sentences, and this much harder than modeling the probability of complete sentences alone (as proposed here). (3) Due to the complexity of the architecture and the difficulty of the modeling task, Skip-thought requires much more training data and longer training time. (4) Skip-thought is a purely generative model and it is only trained using the real data. Meanwhile, the proposed method is a generative model defined via a discriminative function. This function can be trained using both real and fake data, where the fake data can be selectively chosen to be very hard examples. Specifically, we use word shuffling and word dropping to generate hard examples that do not differ from the original sentences. This enables the sentence encoder to focus on the subtle semantics information.

heading: 3 RELATED WORK
text: Previous sentence encoding approaches can be broadly classified as supervised (Conneau et al., 2017; Cer et al., 2018; Marcheggiani & Titov, 2017; Wieting et al., 2015), unsupervised (Kiros et al., 2015; Hill et al., 2016) or semi-supervised approaches (Peters et al., 2018; Dai & Le, 2015; Socher et al., 2011; Clark et al., 2018).
The unsupervised approaches extend the skip-gram (Mikolov et al., 2013) to the sentence level, and use the sentence embedding to predict the adjacent sentences. Skip-thought (Kiros et al., 2015) uses a BiLSTM encoder to obtain a fixed length embedding for a sentence, and uses a BiLSTM decoder to predict adjacent sentences. Training Skip-thought model is expensive, and even one epoch of training on the Toronto BookCorpus (Zhu et al., 2015) dataset takes more than two weeks (Hill et al., 2016) on a single GPU. FastSent (Hill et al., 2016) uses embeddings of a sentence to predict words from the adjacent sentences. A sentence is represented by simply summing up the word representation of all the words in the sentence. FastSent requires less training time than Skip-thought, but FastSent has lower performance on most downstream tasks.
The supervised approaches train the encoders on tasks such as NLI and use transfer learning to adapt the learned encoders to different downstream tasks (Conneau et al., 2017).
Some semi-supervised approaches (Peters et al., 2018; Dai & Le, 2015; Socher et al., 2011) train sentence encoders on large unlabeled datasets, and do a task specific adaptation using labeled data, while some other approaches such as Cross-View Training(CVT) (Clark et al., 2018) jointly train the sentence encoder on the labeled and unlabeled data. CVT applies regular supervised learning on the labeled data, and for the unlabeled data, it enforces consistency across the outputs obtained for partial versions of the input sentence.
In this work, we propose an unsupervised sentence encoder that takes around 20 hours to train on a single GPU, and outperforms Skip-thought and FastSent encoders on multiple downstream tasks. Unlike the previous unsupervised approaches, we use the binary task of real versus fake sentence classification to train a BiLSTM based sentence encoder.

heading: 4 TRAINING TASKS FOR ENCODERS
text: We propose a discriminative task for training sentence encoders. The key bottleneck in training sentence encoders is the need for large amounts of labeled data. Prior work use language modeling as a purely generative training task that models the generation of unlabeled text data. The encoder is trained to produce sentence representations which are effective at either generating neighboring sentences (e.g., Skip-thought (Kiros et al., 2015) or to predict the words in the neighboring sentences (Hill et al., 2016). As we outlined in Section 2 this purely generative training raises multiple challenges.
Instead, we propose fake sentence detection, a task that targets the same overall generative model, one that generates unlabeled text, but is trained discriminatively. The task requires making a single prediction over an input sentence. In particular, we propose to learn a sentence encoder by training a sequential model to solve the binary classification task of detecting whether a given input sentence is fake or real. This real-fake sentence classification task would perhaps be trivial if the fake sentences look very different from the real sentences. We propose two simple methods to generate noisy
sentences which look mostly similar to real sentences. We describe the noisy sentence generation strategies in Section 4.1. Thus, we create a labeled dataset of real and fake sentences, and train a sequential model to distinguish between real and fake sentences, which results in a model whose classification layer has far fewer parameters than previous language model based encoders.
Our model architecture is described in Section 4.2.

heading: 4.1 FAKE SENTENCE GENERATION
text: For a sentenceX = w1, w2, . . . , wn comprising of nwords, we consider two strategies to generate a noisy version of the sentence: 1) WordShuffle: randomly sample two indices i and j corresponding to words wi and wj in X , and shuffle the words to obtain the noisy sentence X̂ . Noisy sentence X̂ would be of the same length as the original sentence X . 2) WordDrop: randomly pick one index i corresponding to word wi and drop the word from the sentence to obtain X̂ . Note there can be many variants for this strategy but here we experiment with this basic choice.

heading: 4.2 REAL VERSUS FAKE SENTENCE CLASSIFICATION
text: Figure 1 shows the proposed architecture of our fake sentence classifier with an encoder and a Multilayer Perceptron(MLP) with 2 hidden layers. We do not use any nonlinearity after these hidden layers, hence, these layers correspond to the single linear layer represented by θ3 in Equation 1. Another motivation behind not using the nonlinearity is to keep the classifier simple enought so that the encoder is forced to learn useful representation while training for the fake sentence classification task. The encoder consists of a bidirectional LSTM followed by a max pooling layer. Given a sentence X = w1, w2, . . . , wn consisting of n words, the forward LSTM of the encoder generates a hidden state vector −→ ht . Similarly, the backward LSTM generates the hidden state ←− ht . The forward and backward hidden states are concatenated to get ut = ( −→ ht , ←− ht). Processing the entire sentence results in n such vectors, and we apply max-pooling to these concatenated hidden states to obtain vector z. z serves as a fixed length encoding of the sentence X , which we then use as input to a MLP for classifying the sentence into real/fake classes.

heading: 5 EVALUATION SETUP
text: Downstream Tasks: We compare the sentence encoders trained on a large collection (BookCorpus (Zhu et al., 2015)) by testing them on multiple sentence level classification tasks (MR, CR, SUBJ, MPQA, TREC, SST) and one NLI task defined over sentence-pairs (SICK). We also eval-
uate the sentence representations for image and caption retrieval tasks on the COCO dataset (Lin et al., 2014). We use the same evaluation protocol and dataset split as (Karpathy & Fei-Fei, 2015; Conneau et al., 2017). Table 1 lists the classification tasks and the datasets. We also compare the sentence representations for how well they capture important syntactic and semantic properties using probing classification tasks (Conneau et al., 2018). For all downstream and probing tasks, we use the encoders to obtain representation for all the sentences, and train logistic regression classifiers on the training split. We tune the L2-norm regularizer using the validation split, and report the results on the test split.
Training Corpus: The FastSent and Skip-thought encoders are trained on the full Toronto BookCorpus of 64M sentences (Zhu et al., 2015). Our models, however, train on a much smaller subset of only 1M sentences.
Sentence Encoder Implementation: Our sentence encoder architecture is the same as the BiLSTM-max model (Conneau et al., 2017). We represent words using 300-d pretrained Glove embeddings (Pennington et al., 2014). We use a single layer BiLSTM model, with 2048-d hidden states. The MLP classifier we use for fake sentence detection has two hidden layers with 1024 and 512 neurons. We train separate models for word drop and word shuffle. The models are trained for 15 epochs with a batch size of 64 using SGD algorithm, when training converges with a validation
set accuracy of 89 for word shuffle. The entire training completes in less than 20 hours on a single GPU machine.
Baseline Approaches: We compare our results with previous unsupervised sentences encoders, Skip-thought (Kiros et al., 2015), FastSent (Hill et al., 2016), Quick-thought(MC-QT) (Logeswaran & Lee, 2018), and bag-of-words baseline NB-SVM (Wang & Manning, 2012). We use the FastSent and Skip-thought results trained on the full BookCorpus as mentioned in (Conneau et al., 2017). We also report Skip-thought results when trained on a smaller 1M sentence subset.

heading: 6 RESULTS
text: Classification and NLI: Results are shown in Table 2. Both fake sentence training tasks yield better performance on five out of the seven language tasks when compared to Skip-thought (full), i.e., even when it is trained on the full BookCorpus. Word drop and word shuffle performances are mostly comparable. Skip-thought (1M) row shows that training on a sentence-level language modeling task can fare substantially worse when trained on a smaller subset of data. FastSent, while easier to train and has faster training cycles, is better than Skip-thought (1M) but is worse than the full Skip-thought model.
Further gain in performance can be obtained by validating the performance of the encoder on downstream tasks after each epoch of training, and picking the encoder with best performance on the validation set for the downstream tasks. For MR and SUBJ tasks, this leads to accuracy of 80.3 and 93.3 respectively. For the remaining tasks, performance gain is less significant.
Image-Caption Retrieval: On both caption and image retrieval tasks (last 2 columns of Table 2), fake sentence training with word dropping and word shuffle are better than the published Skipthought results.
Linguistic Properties: Table 3 compares sentence encoders for various linguistic properties using the recently proposed probing tasks (Conneau et al., 2018). The goal of each task is to use the input sentence encoding to predict a particular syntactic or semantic property of the original sentence it encodes (e.g., predict if the sentence contains a specific word). Encodings from fake sentence training score higher in six out of the ten tasks.
A key property of the discriminative training is that it forces the encoder to be sensitive to small changes that can induce large meaning shifts. WordShuffle encodings turn out to perform significantly better on probing tasks that are related to the sensitivity property. Semantic odd man out requires identifying word insertions that are incompatible in the context, tracking word content requires knowing if a particular word is present or not, and bigram shift (BShift) requires knowing if words in adjacent positions are swapped (a subset of the training task for WordShuffle). Table 4 illustrates examples where fake sentence training is able to distinguish sentence meaning changes that results from a small change in its surface form.
Sentence Lengths We analyze the performance of fake sentence and Skip-thought models to understand how the models behave for sentences of different lengths. Figure 2 compares the performance of fake sentence and Skip-thought encoders binned by length. It turns out that fake sentence training performs better on longer sentences on MR but not so on the SST, even though both are sentiment tasks. The analysis for other tasks (not shown here) also indicate that fake sentence is
better for most sentence lengths but do not indicate a clear trend with respect to increasing sentence lengths.
Relation between Fake Sentence Classification and Downstream Tasks In figure 3, we compare the test performance on fake sentence classification task(WordShuffle) with that on multiple downstream tasks. We notice that the encoder resulting in good performance on the fake sentence classification task results in good performance on the downstream task.

heading: 7 CONCLUSIONS
text: The effectiveness of universal sentence encoders depends both on the architecture of the encoders as well as the training task itself. Language modeling is a suitable task as it fits the generative goal of modeling the distribution of language (text data). However, tackling this in a purely generative fashion (i.e., actually generating sentences) requires large models and long training times.
Instead, we introduced a discriminative formulation of the generative task called fake sentence detection. The sentence encoders are trained to produce representations which are effective at detecting if a given sentence is an original or a fake. This leads to better performance on downstream tasks and is able to represent semantic and syntactic properties, while also reducing the amount of training needed. As future work, the discriminative setup opens up the possibility for the training to be influenced by a specific downstream target task. In particular, we can create negative samples (fake sentences) that are focused towards those specific phenomena relevant to the target task.

Review summaries:
Derive sentence representations from a bidirectional LSTM encoder trained to distinguish real sentences from fake ones. Fake sentences are derived from real ones by swapping two words or dropping a single word (yielding two different models). The resulting representations are applied to various sentence classification tasks by using them as input to a logistic regression classifier trained for the task. Results are generally better than similar experiments performed with SkipThought vectors trained on the same Toronto BookCorpus.

The paper proposes a discriminative training formulation for learning sentence representations, where a classifier is required to distinguish between real and fake sentences. The sentences are encoded with a Bi-LSTM and the resulting sentence representations are then used in a number of sentence-level tasks (classification, entailment, and retrieval). The experiments show benefits on most tasks compared to Skip-Thought and FastSent baselines, and the information captured by the representations is analyzed with probing tasks, showing that they are better at capturing certain kinds of information like the presence or order of words.