TASK DEFINITION: In this task, you are given a paper review. Based on the review, your job is to identify language and generate "en" if the review is in English or generate "es" if the review is in Spanish. Note that URLs in the text have been replaced with [Link].
PROBLEM: In this paper authors propose a novel data augmentation scheme where instead of augmenting the input data, they augment intermediate feature representations.  Sequence auto-encoder based features are considered, and random perturbation, feature interpolation, and extrapolation based augmentation are evaluated. On three sequence classification tasks and on MNIST and CIFAR-10, it is shown that augmentation in feature space, specifically extrapolation based augmentation, results in good accuracy gains w.r.t. authors baseline.  My main questions and suggestions for further strengthening the paper are:  a) The proposed data augmentation approach is applied to a learnt auto-encoder based feature space termed ‘context vector’ in the paper.  The context vectors are then augmented and used as input to train classification models. Have the authors considered applying their feature space augmentation idea directly to the classification model during training, and applying it to potentially many layers of the model?  Also, have the authors considered convolutional neural network (CNN) architectures as well for feature space augmentation?  CNNs are now the state-of-the-art in many image and sequence classification task, it would be very valuable to see the impact of the proposed approach in that model.  b) When interpolation or extrapolation based augmentation was being applied, did the authors also consider utilizing nearby samples from competing classes as well?  Especially in case of extrapolation based augmentation it will be interesting to check if the extrapolated features are closer to competing classes than original ones.  c) With random interpolation or nearest neighbor interpolation based augmentation the accuracy seems to degrade pretty consistently.  This is counter-intuitive.  Do the authors have explanation for why the accuracy degraded with interpolation based augmentation?  d) The results on MNIST and CIFAR-10 are inconclusive.  For instance the error rate on CIFAR-10 is well below 10% these days, so I think it is hard to draw conclusions based on error rates above 30%.  For MNIST it is surprising to see that data augmentation in the input space substantially degrades the accuracy (1.093% -> 1.477%).  As mentioned above, I think this will require extending the feature space augmentation idea to CNN based models.

SOLUTION: en

PROBLEM: Strengths -- An interesting proposal for a smaller CNN architecture designed for embedded CNN applications.  -- Balanced exploration of CNN macroarchitecture and microarchitecture with fire modules. -- x50 less memory usage than AlexNet, keeping similar accuracy  -- strong experimental results  Weaknesses --Would be nice to test Sqeezenet on multiple tasks  --lack of insights and rigorous analysis into what factors are responsible for the success of SqueezeNet. For example, how are ResNet and GoogleNet connected to the current architecture? Another old paper (Analysis of correlation structure for a neural predictive model with application to speech recognition, Neural Networks, 1994) also showed that the “by-pass” architecture by mixing linear and nonlinear prediction terms improves long term dependency in NN based on rigorous perturbation analysis. Can the current work be placed more rigorously on theoretical analysis?

SOLUTION: en

PROBLEM: The paper introduces a simple and effective method for morphological paradigm completion in low-resource settings. The method uses a character-based seq2seq model trained on a mix of examples in two languages: a resource-poor language and a closely-related resource-rich language; each training example is annotated with a paradigm properties and a language ID. Thus, the model enables transfer learning across languages when the two languages share common characters and common paradigms. While the proposed multi-lingual solution is not novel (similar architectures have been explored in syntax, language modeling, and MT), the novelty of this paper is to apply the approach to morphology. Experimental results show substantial improvements over monolingual baselines, and include a very thorough analysis of the impact of language similarities on the quality of results. The paper is interesting, very clearly written, I think it’ll be a nice contribution to the conference program.   Detailed comments:   — My main question is why the proposed general multilingual methodology was limited to pairs of languages, rather than to sets of similar languages? For example, all Romance languages could be included in the training to improve Spanish paradigm completion, and all Slavic languages with Cyrillic script could be mixed to improve Ukrainian. It would be interesting to see the extension of the models from bi-lingual to multilingual settings.   — I think Arabic is not a fair (and fairly meaningless) baseline, given how different is its script and morphology from the target languages. A more interesting baseline would be, e.g., a language with a partially shared alphabet but a different typology. For example, a Slavic language with Latin script could be used as a baseline language for Romance languages. If Arabic is excluded, and if we consider a most distant language in the same the same family as a baseline, experimental results are still strong.   — A half-page discussion of contribution of Arabic as a regularizer also adds little to the paper; I’d just remove Arabic from all the experiments and would add a regularizer (which, according to footnote 5, works even better than adding Arabic as a transfer language).                — Related work is missing a line of work on “language-universal” RNN models that use basically the same approach: they learn shared parameters for inputs in multiple languages, and add a language tag to the input to mediate between languages. Related studies include a multilingual parser (Ammar et al., 2016), language models (Tsvetkov et al., 2016), and machine translation (Johnson et al., 2016 )  Minor:  — I don’t think that the claim is correct in line 144 that POS tags are easy to transfer across languages. Transfer of POS annotations is also a challenging task.    References:   Waleed              Ammar, George Mulcaire, Miguel Ballesteros, Chris Dyer, and Noah A. Smith. "Many languages, one parser.” TACL 2016.   Yulia Tsvetkov, Sunayana Sitaram, Manaal Faruqui, Guillaume Lample, Patrick Littell, David Mortensen, Alan W. Black, Lori Levin, and Chris Dyer. "Polyglot neural language models: A case study in cross-lingual phonetic representation learning.” NAACL 2016.  Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat et al. "Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation." arXiv preprint arXiv:1611.04558 2016.  -- Response to author response:   Thanks for your response & I'm looking forward to reading the final version!

SOLUTION:
en