In this task, you are given a paper review. Based on the review, your job is to identify language and generate "en" if the review is in English or generate "es" if the review is in Spanish. Note that URLs in the text have been replaced with [Link].
--------
Question: This work combines a LDA-type topic model with a RNN and models this by having an additive effect on the predictive distribution via the topic parameters. A variational auto-encoder is used to infer the topic distribution for a given piece of text and the RNN is trained as a RNNLM. The last hidden state of the RNNLM and the topic parameters are then concatenated to use as a feature representation.  The paper is well written and easy to understand. Using the topic as an additive effect on the vocabulary allows for easy inference but intuitively I would expect the topic to affect the dynamics too, e.g. the state of the RNN. The results on using this model as a feature extractor for IMDB are quite strong. Is the RNN fine-tuned on the labelled IMDB data? However, the results for PTB are weaker. From the original paper, an ensemble of 2 LSTMs is able to match the topicRNN score. This method of jointly modelling topics and a language model seems effective and relatively easy to implement.  Finally, the IMDB result is no longer state of the art since this result appeared in May (Miyato et al., Adversarial Training Methods for Semi-Supervised Text Classification).  Some questions: How important is the stop word modelling? What do the results look like if l_t = 0.5 for all t?  It seems surprising that the RNN was more effective than the LSTM. Was gradient clipping tried in the topicLSTM case? Do GRUs also fail to work?  It is also unfortunate that the model requires a stop-word list. Is the link in footnote 4 the one that is used in the experiments?  Does factoring out the topics in this way allow the RNN to scale better with more neurons? How reasonable does the topic distribution look for individual documents? How peaked do they tend to be? Can you show some examples of the inferred distribution? The topics look odd for IMDB with the top word of two of the topics being the same: 'campbell'. It would be interesting to compare these topics with those inferred by LDA on the same datasets.  Minor comments: Below figure 2: GHz -> GB \Gamma is not defined.

Answer: en


Question: This work showed that word representation learning can benefit from sememes when used in an appropriate attention scheme. Authors hypothesized that sememes can act as an essential regularizer for WRL and WSI tasks and proposed SE-WL model which detects word senses and learn representations simultaneously. Though experimental results indicate that WRL benefits, exact gains for WSI are unclear since a qualitative case study of a couple of examples has only been done. Overall, paper is well-written and well-structured.  In the last paragraph of introduction section, authors tried to tell three contributions of this work. (1) and (2) are more of novelties of the work rather than contributions. I see the main contribution of the work to be the results which show that we can learn better word representations (unsure about WSI) by modeling sememe information than other competitive baselines. (3) is neither a contribution nor a novelty.  The three strategies tried for SE-WRL modeling makes sense and can be intuitively ranked in terms of how well they will work. Authors did a good job explaining that and experimental results supported the intuition but the reviewer also sees MST as a fourth strategy rather than a baseline inspired by Chen et al. 2014 (many WSI systems assume one sense per word given a context). MST many times performed better than SSA and SAC. Unless authors missed to clarify otherwise, MST seems to be exactly like SAT with a difference that target word is represented by the most probable sense rather than taking an attention weighted average over all its senses. MST is still an attention based scheme where sense with maximum attention weight is chosen though it has not been clearly mentioned if target word is represented by chosen sense embedding or some function of it.  Authors did not explain the selection of datasets for training and evaluation tasks. Reference page to Sogou-T text corpus did not help as reviewer does not know Chinese language. It was unclear which exact dataset was used as there are several datasets mentioned on that page. Why two word similarity datasets were used and how they are different  (like does one has more rare words than another) since different models performed differently on these datasets. The choice of these datasets did not allow evaluating against results of other works which makes the reviewer wonder about next question.  Are proposed SAT model results state of the art for Chinese word similarity?  E.g. Schnabel et al. (2015) report a score of 0.640 on WordSim-353 data by using CBOW word embeddings.  Reviewer needs clarification on some model parameters like vocabulary sizes for words (Does Sogou-T contains 2.7 billion unique words) and word senses (how many word types from HowNet). Because of the notation used it is not clear if embeddings for senses and sememes for different words were shared. Reviewer hopes that is the case but then why 200 dimensional embeddings were used for only 1889 sememes. It would be better if complexity of model parameters can also be discussed.  May be due to lack of space but experiment results discussion lack insight into observations other than SAT performing the best. Also, authors claimed that words with lower frequency were learned better with sememes without evaluating on a rare words dataset.  I have read author's response.

Answer: en


Question: This paper aims to mine explicit rules from KB embedding space, and casts it into a sparse reconstruction problem. Experiments demonstrate its ability of extracting reasonable rules on a few link prediction datasets.  The solution part sounds plausible. However, it confuses me that why we need to mine rules from learned KB embeddings.   - It is still unclear what information these KB embeddings encode and it looks strange that we aim to learn rules including negation / disjunction from them.  - If the goal is to extract useful rules (for other applications), it is necessary to compare it to “graph random walk” (

Answer:
en