Q: In this task, you are given a paper review. Based on the review, your job is to identify language and generate "en" if the review is in English or generate "es" if the review is in Spanish. Note that URLs in the text have been replaced with [Link].
*** Paper Summary *** This paper applies adversarial and virtual adversarial training to LSTM for text classification. Since text inputs are discrete adversarial perturbation are applied to the (normalized) word embeddings. Extensive experiments are reported and demonstrate the advantage of these methods. *** Review Summary *** The paper reads well and has sufficent references. The application of adversarial training to text data is a simple but not trivial extension. The experimental section presents extensive experiments with comparison to alternative strategies. The proposed method is simple and effective and can be easily be applied after reading the paper. *** Detailed Review *** The paper reads well. I have only a few comments regarding experiments and link to prior resarch: Experiments: - In Table 2 (and for other datasets as well), could you include an SVM baseline? e.g. S Wang and C Manning 2012? - As another baseline, did you consider dropping words, i.e. masking noise? It is generally better than dropout/gaussian noise for text application (e.g. denoising autoencoders)? - I am not sure I understand why virtual adversarial is worse than the baseline in Table 5. If you tune epsilon, in the worse case you would get the same performance as the baseline? Was it that validation was unreliable? Related Work: I think it would be interesting to point at SVM, transductive SVM who achieve something similar to adversarial training. When maximizing the margin in a (transductive) SVM, it is equivalent to move the example toward the decision boundary, i.e. moving them in the direction of increase of the loss gradient. Also it would be interesting to draw a parallel between adversarial training and contrastive divergence. The adversarial samples are very close in nature to the one step Markov Chain samples from CD. See Bengio 2009. Related to this technique are also approaches that try to explicitely cancel the Jacobian at data points, e.g. Rifai et al 2011. *** References *** Marginalized Denoising Autoencoders for Domain Adaptation. Minmin Chen, K Weinberger. Stacked Denoising Autoencoders. Pascal Vincent. JMLR 2011. Learning invariant features through local space contraction, Salah Rifai, Xavier Muller, Xavier Glorot, Gregoire Mesnil, Yoshua Bengio and Pascal Vincent, 2011. Learning Deep Architectures for AI, Yoshua Bengio 2009 Large Scale Transductive SVMs. Ronan Collobert et al 2006 Optimization for Transductive SVM. O Chapelle, V Sindhwani, SS Keerthi JMLR 2008
A:
en