In this task, you are given a paper review. Based on the review, your job is to identify language and generate "en" if the review is in English or generate "es" if the review is in Spanish. Note that URLs in the text have been replaced with [Link].

- Strengths: This paper presents a 2 x 2 x 3 x 10 array of accuracy results based on systematically changing the parameters of embeddings models: (context type, position sensitive, embedding model, task), accuracy - context type {Linear, Syntactic} - position sensitive {True, False} - embedding model {Skip Gram, BOW, GLOVE} - task {Word Similarity, Analogies, POS, NER, Chunking, 5 text classific. tasks} The aim of these experiments was to investigate the variation in performance as these parameters are changed. The goal of the study itself is interesting for the ACL community and similar papers have appeared before as workshop papers and have been well cited, such as Nayak et al.'s paper mentioned below. - Weaknesses: Since this paper essentially presents the effect of systematically changing the context types and position sensitivity, I will focus on the execution of the investigation and the analysis of the results, which I am afraid is not satisfactory. A) The lack of hyper-parameter tuning is worrisome. E.g. - 395 Unless otherwise notes, the number of word embedding dimension is set to 500. - 232 It still enlarges the context vocabulary about 5 times in practice. - 385 Most hyper-parameters are the same as Levy et al' best configuration. This is worrisome because lack of hyperparameter tuning makes it difficult to make statements like method A is better than method B. E.g. bound methods may perform better with a lower dimensionality than unbound models, since their effective context vocabulary size is larger. B) The paper sometimes presents strange explanations for its results. E.g. - 115 "Experimental results suggest that although it's hard to find any universal insight, the characteristics of different contexts on different models are concluded according to specific tasks." What does this sentence even mean? - 580 Sequence labeling tasks tend to classify words with the same syntax to the same category. The ignorance of syntax for word embeddings which are learned by bound representation becomes beneficial. These two sentences are contradictory, if a sequence labeling task classified words with "same syntax" to same category then syntx becomes a ver valuable feature. Bound representation's ignorance of syntax should cause a drop in performance just like other tasks which does not happen. C) It is not enough to merely mention Lai et. al. 2016 who have also done a systematic study of the word embeddings, and similarly the paper "Evaluating Word Embeddings Using a Representative Suite of Practical Tasks", Nayak, Angeli, Manning. appeared at the repeval workshop at ACL 2016. should have been cited. I understand that the focus of Nayak et al's paper is not exactly the same as this paper, however they provide recommendations about hyperparameter tuning and experiment design and even provide a web interface for automatically running tagging experiments using neural networks instead of the "simple linear classifiers" used in the current paper. D) The paper uses a neural BOW words classifier for the text classification tasks but a simple linear classifier for the sequence labeling tasks. What is the justification for this choice of classifiers? Why not use a simple neural classifier for the tagging tasks as well? I raise this point, since the tagging task seems to be the only task where bound representations are consistently beating the unbound representations, which makes this task the odd one out. - General Discussion: Finally, I will make one speculative suggestion to the authors regarding the analysis of the data. As I said earlier, this paper's main contribution is an analysis of the following table. (context type, position sensitive, embedding model, task, accuracy) So essentially there are 120 accuracy values that we want to explain in terms of the aspects of the model. It may be beneficial to perform factor analysis or some other pattern mining technique on this 120 sample data.
en

This paper develops Submodular Sum Product Networks (SSPNs) and an efficient inference algorithm for approximately computing the most probable labeling of variables in the model. The main application in the paper is on scene parsing. In this context, SSPNs define an energy function with a grammar component for representing a hierarchy of labels and an MRF for encoding smoothness of labels over space. To perform inference, the authors develop a move-making algorithm, somewhat in the spirit of fusion moves (Lempitsky et al., 2010) that repeatedly improves a solution by considering a large neighborhood of alternative segmentations and solving an optimization problem to choose the best neighbor. Empirical results show that the proposed algorithm achieves better energy that belief propagation of alpha expansion and is much faster. This is generally a well-executed paper. The model is interesting and clearly defined, the algorithm is well presented with proper analysis of the relevant runtimes and guarantees on the behavior. Overall, the algorithm seems effective at minimizing the energy of SSPN models. Having said that, I don't think this paper is a great fit for ICLR. The model is even somewhat to the antithesis of the idea of learning representations, in that a highly structured form of energy function is asserted by the human modeller, and then inference is performed. I don't see the connection to learning representations. One additional issue is that while the proposed algorithm is faster than alternatives, the times are still on the order of 1-287 seconds per image, which means that the applicability of this method (as is) to something like training ConvNets is limited. Finally, there is no attempt to argue that the model produces better segmentations than alternative models. The only evaluations in the paper are on energy values achieved and on training data. So overall I think this is a good paper that should be published at a good machine learning conference, but I don't think ICLR is the right fit.
en

This submission proposes to learn the word decomposition, or word to sub-word sequence mapping jointly with the attention based sequence-to-sequence model. A particular feature of this approach is that the decomposition is not static, instead, it also conditions on the acoustic input, and the mapping is probabilistic, i.e., one word may map to multiple sub-word sequences. The authors argue that the dynamic decomposition approach can more naturally reflect the acoustic pattern. Interestingly, the motivation behind this approach is analogous to learning the pronunciation mixture model for HMM based speech recognition, where the probabilistic mapping from a word to its pronunciations also conditions on the acoustic input, e.g., I. McGraw, I. Badr, and J. Glass, "Learning lexicons form speech using a pronunciation mixture model," in IEEE Transactions on Audio, Speech, and Language Processing, 2013 L. Lu, A. Ghoshal, S. Renals, "Acoustic data-driven pronunciation lexicon for large vocabulary speech recognition", in Proc. ASRU R. Singh, B. Raj, and R. Stern, "Automatic generation of subword units for speech recognition systems," in IEEE Transactions on Speech and Audio Processing, 2002 It would be interesting to put this work in the context by linking it to some previous works in the HMM framework. Overall, the paper is well written, and it is theoretically convincing. The experimental study could be more solid, e.g., it is reasonable to have a word-level baseline, as the proposed approach lies in between the character-level and word-level systems. the vocabulary size of the WSJ si284 dataset is 20K at maximum, which is not very large for the softmax layer, and it is a closed vocabulary task. I guess the word-level system may be also competitive to the numbers reported in this paper. Furthermore, can you explain what is the computational bottleneck of the proposed approach? You downsampled the data by the factor of 4 using an RNN, and it still took around 5 days to converge. To me, it is a bit expensive, especially given that you only take one sample when computing the gradient. Table 2 is a little bit misleading, as CTC with language model and seq2seq with a language model model from Bahdanau et al. is much closer to the best number reported in this Table 2, while you may only get a very small improvement using a language model. Finally, "O(5) days to converge" sounds a bit odd to me.
en