Title: Adversarial Training Methods for Semi-Supervised Text Classification
Authors: Takeru Miyato
Andrew M. Dai
Ian Goodfellow
Venue: ICLR 2017
Keywords: Natural language processing
Deep learning
Semi-Supervised Learning

Abstract: Adversarial training provides a means of regularizing supervised learning algorithms while virtual adversarial training is able to extend supervised learning algorithms to the semi-supervised setting.
However, both methods require making small perturbations to numerous entries of the input vector, which is inappropriate for sparse high-dimensional inputs such as one-hot word representations.
We extend adversarial and virtual adversarial training to the text domain by applying perturbations to the word embeddings in a recurrent neural network rather than to the original input itself.
The proposed method achieves state of the art results on multiple benchmark semi-supervised and purely supervised tasks.
We provide visualizations and analysis showing that the learned word embeddings have improved in quality and that while training, the model is less prone to overfitting.



Review discussion:
Hi,
Is the virtual adversarial training used in the pre-training phase?
Also In the results of Table 2, for the line of "Virtual Adv" and "Adv+Virtual Adv", the cost term of formula (3) are applied to all the samples, or only applied to unlabeled samples?
Thanks for your comments!
>Is the virtual adversarial training used in the pre-training phase?
No, we used virtual adversarial training only in the fine-tune phase.
> Also In the results of Table 2, for the line of "Virtual Adv" and "Adv+Virtual Adv", the cost term of formula (3) are applied to all the samples, or only applied to unlabeled samples?
The former is correct. We calculate the virtual adversarial training loss (3) on both labeled and unlabeled samples.
*** Paper Summary ***
This paper applies adversarial and virtual adversarial training to LSTM for text classification. Since text inputs are discrete adversarial perturbation are applied to the (normalized) word embeddings. Extensive experiments are reported and demonstrate the advantage of these methods.
*** Review Summary ***
The paper reads well and has sufficent references. The application of adversarial training to text data is a simple but not trivial extension. The experimental section presents extensive experiments with comparison to alternative strategies. The proposed method is simple and effective and can be easily be applied after reading the paper.
*** Detailed Review ***
The paper reads well. I have only a few comments regarding experiments and link to prior resarch:
Experiments:
- In Table 2 (and for other datasets as well), could you include an SVM baseline? e.g. S Wang and C Manning 2012?
- As another baseline, did you consider dropping words, i.e. masking noise? It is generally better than dropout/gaussian noise for text application (e.g. denoising autoencoders)?
- I am not sure I understand why virtual adversarial is worse than the baseline in Table 5. If you tune epsilon, in the worse case you would get the same performance as the baseline? Was it that validation was unreliable?
Related Work:
I think it would be interesting to point at SVM, transductive SVM who achieve something similar to adversarial training.
When maximizing the margin in a (transductive) SVM, it is equivalent to move the example toward the decision boundary, i.e. moving them in the direction of increase of the loss gradient.
Also it would be interesting to draw a parallel between adversarial training and contrastive divergence. The adversarial samples are very close in nature to the one step Markov Chain samples from CD. See Bengio 2009. Related to this technique are also approaches that try to explicitely cancel the Jacobian at data points, e.g. Rifai et al 2011.
*** References ***
Marginalized Denoising Autoencoders for Domain Adaptation. Minmin Chen, K Weinberger.
Stacked Denoising Autoencoders. Pascal Vincent. JMLR 2011.
Learning invariant features through local space contraction, Salah Rifai, Xavier Muller, Xavier Glorot, Gregoire Mesnil, Yoshua Bengio and Pascal Vincent, 2011.
Learning Deep Architectures for AI, Yoshua Bengio 2009
Large Scale Transductive SVMs. Ronan Collobert et al 2006
Optimization for Transductive SVM.  O Chapelle, V Sindhwani, SS Keerthi JMLR 2008
Thank you for the review!
“In Table 2 (and for other datasets as well), could you include an SVM baseline? e.g. S Wang and C Manning 2012?”
Yes, we added the IMDB performance with Naive Bayes SVM by Wang and Manning (2012) to the Table 2.
“As another baseline, did you consider dropping words, i.e. masking noise?”
The performance using dropping words is demonstrated by Dai and Le (2015) (SA-LSTM in Table 2 in our paper.), and it is little bit better than our baseline ( 7.24% with SA-LSTM, and 7.33% with our baseline).
We also tested dropping words ( randomly masking each embedding of the word ) on our baseline models, however, we could not find any improvement from the baseline.
“I am not sure I understand why virtual adversarial is worse than the baseline in Table 5.”
It is because we optimized epsilon between [1.0, 10.0] on each dataset. If epsilon is set to 0, the performance should be same as the baseline. The reason why we optimized epsilon between such a narrow range is that training the LSTM model is computationally costly, and on IMDB, Elec and RCV1, the optimal epsilons seem to be within [1.0, 10.0] on each validation set.
“I think it would be interesting to point at SVM, transductive SVM who achieve something similar to adversarial training.”
We added the performance with transductive SVMs on IMDB, Elec and RCV1 by Johnson and Zhang (2015) to Table 2 and 4, and our proposed method outperforms the SVM-based methods.
Note that transductive learning is somewhat different from semi-supervised learning because transductive learning allows the model to look at the test set.
We added an explanation of the similarities between adversarial training methods and SVM-based methods in the related work section. Please see the revised version of our paper.
“Also it would be interesting to draw a parallel between adversarial training and contrastive divergence. “
The similarities and differences between adversarial training and other methods like CAEs, double backprop, tangent propagation, etc. are described in the Deep Learning textbook ( http://www.deeplearningbook.org/contents/regularization.html , http://www.deeplearningbook.org/contents/representation.html ) so we don’t feel it’s necessary to repeat this amount of background explanation in the conference paper. There is also a separate ICLR 2015 workshop paper describing the connections between adversarial training and contrastive divergence: Goodfellow, On distinguishability criteria for estimating generative models. https://arxiv.org/abs/1412.6515
Did the author try setups were classes were unbalanced? (In past semi-supervised literature, balancing constraints were important).
We did not try that constraint on the datasets we used in our paper,
but I think the balancing constraint would improve the performance especially on RCV1 dataset (The classes in RCV1 dataset are unbalanced).
I have tried similar constraint with virtual adversarial training on semi-supervised learning task on MNIST (I constrained the marginal distribution p(y) := \sum_x p(y|x)q(x) to be uniform) and it stabilized the training process and improved performance over the training without the constraint.
Thank you for your comment!
This paper applies the idea of the adversarial training and virtual adversarial training to the LSTM-based model in the text context. The paper is in general well written and easy to follow. Extending the idea of the adversarial training to the text tasks is simple but non-trivial.
Overall the paper is worth to publish.
I only have a minor comment: it is also interesting to see how much adversarial training can help in the performance of RNN, which is a simpler model and may be easier to analyze.
Thank you for the review!
“it is also interesting to see how much adversarial training can help in the performance of RNN, which is a simpler model and may be easier to analyze.“
An LSTM is a kind of RNN. Simple RNNs, such as those based on linear transformation and tanh at each time step, are very difficult to interpret, just as LSTMs are. LSTMs may even be somewhat easier to interpret because information is propagated through time using addition.
The authors propose to apply virtual adversarial training to semi-supervised classification.
It is quite hard to assess the novelty on the algorithmic side at this stage: there is a huge available literature on semi-supervised learning (especially SVM-related literature, but some work were applied to neural networks too); unfortunately the authors do not mention it, nor relate their approach to it, and stick to the adversarial world.
In terms of novelty on the adversarial side, the authors propose to add perturbations at the level of words embeddings, rather than the input itself (having in mind applications to NLP).
Concerning the experimental section, authors focus on text classification methods. Again, comparison with the existing SVM-related literature is important to assess the viability of the proposed approach;
for example (Wang et al, 2012) report 8.8% on IMBD with a very simple linear SVM (without transductive setup).
Overall, the paper reads well and propose a semi-supervised learning algorithm which is shown to work in practice. Theoretical and experimental comparison with past work is missing.
Thank you for the review!
As you suggested, we added results on IMDB performance with Naive Bayes SVM by Wang and Manning (2012) to Table 2 in the paper.
We also added results with transductive SVMs on IMDB, Elec and RCV1 by Johnson and Zhang (2015) to Table 2 and 4.
We confirmed that adversarial and virtual adversarial training outperform the SVM-based approaches.
Additionally, we explain the similarities between SVM-based methods and adversarial training methods in the related work section. Please see the revised version of our paper.

Meta-review:
role: PC
data: comment: This paper is concerned with extending adversarial and virtual adversarial training to text classification tasks. The main technical contribution is to apply perturbations to word embeddings rather than discrete input symbols. Excellent empirical performance is reported across a variety of tasks. 
 
 The reviewers were consensual in acknowledging the clarity and significance of the contribution, highlighting the quality of the numerical experiments. Moreover, the authors were responsive in the rebuttal phase and updated their paper with reviewers suggestions (such as the svm-related comparisons). 
 
 The AC thus recommends accepting this work as a poster.

Decision: Accept (Poster)