TASK DEFINITION: In this task, you are given a paper review. Based on the review, your job is to identify language and generate "en" if the review is in English or generate "es" if the review is in Spanish. Note that URLs in the text have been replaced with [Link].
PROBLEM: - Strengths: Zero-shot relation extraction is an interesting problem. The authors have created a large dataset for relation extraction as question answering which would likely be useful to the community. - Weaknesses: Comparison and credit to existing work is severely lacking. Contributions of the paper don't seen particularly novel. - General Discussion: The authors perform relation extraction as reading comprehension. In order to train reading comprehension models to perform relation extraction, they create a large dataset of 30m "querified" (converted to natural language) relations by asking mechanical turk annotators to write natural language queries for relations from a schema. They use the reading comprehension model of Seo et al. 2016, adding the ability to return "no relation," as the original model must always return an answer. The main motivation/result of the paper appears to be that the authors can perform zero-shot relation extraction, extracting relations only seen at test time. This paper is well-written and the idea is interesting. However, there are insufficient experiments and comparison to previous work to convince me that the paper's contributions are novel and impactful. First, the authors are missing a great deal of related work: Neelakantan at al. 2015 ([Link] perform zero-shot relation extraction using RNNs over KB paths. Verga et al. 2017 ([Link] perform relation extraction on unseen entities. The authors cite Bordes et al. ([Link] who collect a similar dataset and perform relation extraction using memory networks (which are commonly used for reading comprehension). However, they merely note that their data was annotated at the "relation" level rather than at the triple (relation, entity pair) level but couldn't Bordes et al. have done the same in their annotation? If there is some significant difference here, it is not made clear in the paper. There is also a NAACL 2016 paper ([Link] which performs relation extraction using a new model based on memory networks and I'm sure there are more. Your work is so similar to much of this work that you should really cite and establish novelty wrt at least some of them as early as the introduction -- that's how early I was wondering how your work differed, and it was not made clear. Second, the authors neither 1) evaluate their model on another dataset or 2) evaluate any previously published models on their dataset. This makes their empirical results extremely weak. Given that there is a wealth of existing work that performs the same task and the lack of novelty of this work, the authors need to include experiments that demonstrate that their technique outperforms others on this task, or otherwise show that their dataset is superior to others (e.g. since it is much larger than previous, does it allow for better generalization?)

SOLUTION: en

PROBLEM: El articulo se deja leer con facilidad, en especial en su primera mitad. Luego, hay errores de escritura y redaccion que van afectando la comprension de lo realizado (PostgrSQL en vez de PostgresSQL, con tiene en vez de contiene, ...). Adicionalmente se detallan pasos, que caen en detalles irrelevantes, como indicar que debe instalarse tal o cual herramienta, sin entrar en los aspectos realmente importantes, especialmente en la implementacion del acoplamiento medio (donde, no se detalla que se implemento a nivel del SABD y que a nivel externo). Lo mas criticable es el uso de dos SABD (PostgreSQL para el acoplamiento debil, y Oracle para los acoplamientos medio y fuerte) en la experimentacion, esto hace que sea muy discutible la comparacion que, ya no solo tienen la implementacion del algoritmo como variable independiente, sino tambien el SABD a utilizar. Pse a lo anterior, pienso que el trabajo es digno de ser presentado, de forma de mostrar, en particular, las diferentes alternativas de integracion de algoritmos de Mineria de Datos con Sistemas Administradores de Bases de Datos.

SOLUTION: es

PROBLEM: The authors propose and evaluate using SPN's to generate embeddings of input and output variables, and using MPN to decode output embeddings to output variables. The advantage of predicting label embeddings is to decouple dependencies in the predicted space. The authors show experimentally that using SPN based embeddings is better than those produced by RBM's. This paper is fairly dense and a bit hard to read. After the discussion, the main contributions of the authors are: 1. They propose the scheme of learning SPN's over Y and then using MPN's to decode the output, or just SPNs to embed X. 2. They propose how to decode MPN's with partial data. 3. They perform some analysis of when their scheme will lead to perfect encoding/decodings. 4. They run many, many experiments comparing various ways of using their proposed method to make predictions on multi-label classification datasets. My main concerns with this paper are as follows: - The point of this paper is about using generative models for representation learning. In their experiments, the main task is discriminative; e.g. predict multiple Y from X. The only discriminative baseline is a L2 regularized logistic regression, which does not have any structure on the output; it'd be nice to see how a discriminative structured prediction method would do, such as CRF or belief propagation. - The many experiments suggest that their encoder/decoder scheme is working better than the alternatives; can you please give more details on the relative computation complexity of each method? - One thing I'm still having trouble understanding is *why* this method works better than MADE and the other alternatives. Is it learning a better model of the distribution of Y? Is it better at separating out correlations in the output into individual nodes? Does it have larger representations? - I think the experiments are overkill and if anything, they way they are presented detract from the paper. There's already far too many numbers and graphs presented to be easy to understand. If I have to dig through hundreds of numbers to figure out if your claim is correct, the paper is not clear enough. And, I said this before in my comments, please do not refer to Q1, Q2, etc. -- these shortcuts let you make the paper more dense with fewer words but at the cost of readability. I *think* I convinced myself that your method works...I would love to see a table that shows, for each condition: (A) a baseline X->Y, (B) one *average* result across datasets for your method, and (C) one *average* result from a reasonable best competitor method. Please show for both the exact match and hamming losses, as that will demonstrate the gap between independent linear prediction and structured prediction. That would still be plenty of numbers but would make it much easier for the reader to verify your claims and you can put everything else in the Appendix. E.g. something like: Input | Predicted Output | Decoder | Hamming | Exact Match ---- X | P(Y) | CRF | xx.xx | xx.xx (this is your baseline) SPN E_X | P(Y) | n/a | xx.xx | xx.xx X | SPN E_Y | MPN | xx.xx | xx.xx (given X, predict E_Y, then decode it with an MPN) Does a presentation like that make sense? It's just really hard and time-consuming for me as a reviewer to verify your results, the way you've laid them out currently.

SOLUTION:
en