In this task, you are given a paper review. Based on the review, your job is to identify language and generate "en" if the review is in English or generate "es" if the review is in Spanish. Note that URLs in the text have been replaced with [Link].

This paper considers the problem of KB completion and proposes ITransF for this purpose. Unlike STransE that assigns each relation an independent matrix, this paper proposes to share the parameters between different relations. A model is proposed where a tensor D is constructed that contains various relational matrices as its slices and a selectional vector \alpha is used to select a subset of relevant relational matrix for composing a particular semantic relation. The paper then discuss a technique to make \alpha sparse. Experimental results on two standard benchmark datasets shows the superiority of ITransF over prior proposals.  The paper is overall well written and the experimental results are good. However, I have several concerns regarding this work that I hope the authors will answer in their response.  1. Just by arranging relational matrices in a tensor and selecting (or more appropriately considering a linearly weighted sum of the relational matrices) does not ensure any information sharing between different relational matrices. This would have been the case if you had performed some of a tensor decomposition and projected the different slices (relational matrices) into some common lower-dimensional core tensor. It is not clear why this approach was not taken despite the motivation to share information between different relational matrices. 2. The two requirements (a) to share information across different relational matrices and (b) make the attention vectors sparse are some what contradictory. If the attention vector is truly sparse and has many zeros then information will not flow into those slices during optimisation.  3. The authors spend a lot of space discussing techniques for computing sparse attention vectors. The authors mention in page 3 that \ell_1 regularisation did not work in their preliminary experiments. However, no experimental results are shown for \ell_1 regularisation nor they explain why \ell_1 is not suitable for this task. To this reviewer, it appears as an obvious baseline to try, especially given the ease of optimisation. You use \ell_0 instead and get into NP hard optimisations because of it. Then you propose a technique and a rather crude approximation in the end to solve it. All that trouble could be spared if \ell_1 was used. 4. The vector \alpha is performing a selection or a weighing over the slices of D. It is slightly misleading to call this as “attention” as it is a term used in NLP for a more different type of models (see attention model used in machine translation). 5. It is not clear why you need to initialise the optimisation by pre-trained embeddings from TransE. Why cannot you simply randomly initialise the embeddings as done in TransE and then update them? It is not fair to compare against TransE if you use TransE as your initial point.  Learning the association between semantic relations is an idea that has been used in related problems in NLP such as relational similarity measurement [Turney JAIR 2012] and relation adaptation [Bollegala et al. IJCAI 2011]. It would be good to put the current work with respect to such prior proposals in NLP for modelling inter-relational correlation/similarity.  Thanks for providing feedback. I have read it.
en

- Strengths: This paper proposes an evaluation metric for automatically evaluating the quality of dialogue responses in non-task-oriented dialogue. The metric operates on continuous vector space representations obtained by using RNNs and it comprises two components: one that compares the context and the given response and the other that compares a reference response and the given response. The comparisons are conducted by means of dot product after projecting the response into corresponding context and reference response spaces. These projection matrices are learned by minimizing the squared error between the model predictions and human annotations.  I think this work gives a remarkable step forward towards the evaluation of non-task-oriented dialogue systems. Different from previous works in this area, where pure semantic similarity was pursued, the authors are going beyond pure semantic similarity in a very elegant manner by learning projection matrices that transform the response vector into both context and reference space representations. I am very curious on how your projection matrices M and N differ from the original identity initialization after training the models. I think the paper will be more valuable if further discussion on this is introduced, rather than focusing so much on resulting correlations.   - Weaknesses:  The paper also leaves lots questions related to the implementation. For instance, it is not clear whether the human scores used to train and evaluate the system were single AMT annotations or the resulting average of few annotations. Also, it is not clear how the dataset was split into train/dev/test and whether n-fold cross validation was conducted or not. Also, it would be nice to better explain why in table 2 correlation for ADEM related scores are presented for the validation and test sets, while for the other scores they are presented for the full dataset and test set. The section on pre-training with VHRED is also very clumsy and confusing, probably it is better to give less technical details but a better high level explanation of the pre-training strategy and its advantages.  - General Discussion:  “There are many obvious cases where these metrics fail, as they are often incapable of considering the semantic similarity between responses (see Figure 1).” Be careful with statements like this one. This is not a problem of semantic similarity! Opposite to it, the problem is that completely different semantic cues might constitute pragmatically valid responses. Then, semantic similarity itself is not enough to evaluate a dialogue system response. Dialogue system response evaluation must go beyond semantics (This is actually what your M and N matrices are helping to do!!!)   “an accurate model that can evaluate dialogue response quality automatically — what could be considered an automatic Turing test —“ The original intention of Turing test was to be a proxy to identify/define intelligent behaviour. It actually proposes a test on intelligence based on an “intelligent” machine capability to imitate human behaviour in such a way that it would be difficult for a common human to distinguish between such a machine responses and actual human responses. It is of course related to dialogue system performance, but I think it is not correct to say that automatically evaluating dialogue response quality is an automatic Turing test. Actually, the title itself “Towards an Automatic Turing Test” is somehow misleading!  “the simplifying assumption that a ‘good’ chatbot is one whose responses are scored highly on appropriateness by human evaluators.” This is certainly the correct angle to introduce the problem of non-task-oriented dialogue systems, rather than “Turing Test”. Regarding this, there has been related work you might like to take a look at, as well as to make reference to, in the WOCHAT workshop series (see the shared task description and corresponding annotation guidelines).  In the discussion session: “and has has been used” -> “and it has been used”
en

This paper develops Submodular Sum Product Networks (SSPNs) and an efficient inference algorithm for approximately computing the most probable labeling of variables in the model. The main application in the paper is on scene parsing. In this context, SSPNs define an energy function with a grammar component for representing a hierarchy of labels and an MRF for encoding smoothness of labels over space. To perform inference, the authors develop a move-making algorithm, somewhat in the spirit of fusion moves (Lempitsky et al., 2010) that repeatedly improves a solution by considering a large neighborhood of alternative segmentations and solving an optimization problem to choose the best neighbor. Empirical results show that the proposed algorithm achieves better energy that belief propagation of alpha expansion and is much faster.  This is generally a well-executed paper. The model is interesting and clearly defined, the algorithm is well presented with proper analysis of the relevant runtimes and guarantees on the behavior. Overall, the algorithm seems effective at minimizing the energy of SSPN models.  Having said that, I don't think this paper is a great fit for ICLR. The model is even somewhat to the antithesis of the idea of learning representations, in that a highly structured form of energy function is asserted by the human modeller, and then inference is performed. I don't see the connection to learning representations. One additional issue is that while the proposed algorithm is faster than alternatives, the times are still on the order of 1-287 seconds per image, which means that the applicability of this method (as is) to something like training ConvNets is limited.  Finally, there is no attempt to argue that the model produces better segmentations than alternative models. The only evaluations in the paper are on energy values achieved and on training data.  So overall I think this is a good paper that should be published at a good machine learning conference, but I don't think ICLR is the right fit.

en