instruction:
In this task, you are given a paper review. Based on the review, your job is to identify language and generate "en" if the review is in English or generate "es" if the review is in Spanish. Note that URLs in the text have been replaced with [Link].
question:
Un buen trabajo que esta en sintonia con el INFONOR. De hecho este trabajo da continuidad a uno presentado en el INFONOR 2012. Mi unica ponderacion es que la referencia [13] deberia ser el articulo publicado y presentado en el INFONOR en vez de la pagina del INFONOR.
answer:
es

question:
This work combines a LDA-type topic model with a RNN and models this by having an additive effect on the predictive distribution via the topic parameters. A variational auto-encoder is used to infer the topic distribution for a given piece of text and the RNN is trained as a RNNLM. The last hidden state of the RNNLM and the topic parameters are then concatenated to use as a feature representation. The paper is well written and easy to understand. Using the topic as an additive effect on the vocabulary allows for easy inference but intuitively I would expect the topic to affect the dynamics too, e.g. the state of the RNN. The results on using this model as a feature extractor for IMDB are quite strong. Is the RNN fine-tuned on the labelled IMDB data? However, the results for PTB are weaker. From the original paper, an ensemble of 2 LSTMs is able to match the topicRNN score. This method of jointly modelling topics and a language model seems effective and relatively easy to implement. Finally, the IMDB result is no longer state of the art since this result appeared in May (Miyato et al., Adversarial Training Methods for Semi-Supervised Text Classification). Some questions: How important is the stop word modelling? What do the results look like if l_t = 0.5 for all t? It seems surprising that the RNN was more effective than the LSTM. Was gradient clipping tried in the topicLSTM case? Do GRUs also fail to work? It is also unfortunate that the model requires a stop-word list. Is the link in footnote 4 the one that is used in the experiments? Does factoring out the topics in this way allow the RNN to scale better with more neurons? How reasonable does the topic distribution look for individual documents? How peaked do they tend to be? Can you show some examples of the inferred distribution? The topics look odd for IMDB with the top word of two of the topics being the same: 'campbell'. It would be interesting to compare these topics with those inferred by LDA on the same datasets. Minor comments: Below figure 2: GHz -> GB \Gamma is not defined.
answer:
en

question:
EDIT: Updated score. See additional comment. I quite like the main idea of the paper, which is based on the observation in Sec. 3.0 - that the authors find many predictable patterns in the independent evolution of weights during neural network training. It is very encouraging that a simple neural network can be used to speed up training by directly predicting weights. However the technical quality of the current paper leaves much to be desired, and I encourage the authors to do more rigorous analysis of the approach. Here are some concrete suggestions: - The findings in Section 3.0 which motivate the approach, should be clearly presented in the paper. Presently they are stated as anecdotes. - A central issue with the paper is that the training of the Introspection network I is completely glossed over. How well did the training work, in terms of training, validation/test losses? How well does it need to work in order to be useful for speeding up training? These are important questions for anyone interested in this approach. - An additional important issue is that of baselines. Would a simple linear/quadratic model also work instead of a neural network? What about a simple heuristic rule to increase/decrease weights? I think it's important to compare to such baselines to understand the complexity of the weight evolution learned by the neural network. - I do not think that default tensorflow example hyperparameters should be used, as mentioned by authors on OpenReview. There is no scientific basis for using them. Instead, first hyperparameters which produce good results in a reasonable time should be selected as the baseline, and then added the benefit of the introspection network to speed up training (and reaching a similar result) should be shown. - The authors state in the discussion on OpenReview that they also tried RNNs as the introspection network but it didn't work with small state size. What does "didn't work" mean in this context? Did it underfit? I find it hard to imagine that a large state size would be required for this task. Even if it is, that doesn't rule out evaluation due to memory issues because the RNN can be run on the weights in 'mini-batch' mode. In general, I think other baselines are more important than RNN. - A question about jump points: The I is trained on SGD trajectories. While using I to speed up training at several jump points, if the input weights cross previous jump points, then I gets input data from a weight evolution which is not from SGD (it has been altered by I). This seems problematic but doesn't seem to affect your experiments. I feel that this again highlights the importance of the baselines. Perhaps I is doing something extremely simple that is not affected by this issue. Since the main idea is very interesting, I will be happy to update my score if the above concerns are addressed.
answer:
en