In this task, you are given a paper review. Based on the review, your job is to identify language and generate "en" if the review is in English or generate "es" if the review is in Spanish. Note that URLs in the text have been replaced with [Link].
--------
Question: Summary: The paper proposes a large-scale dataset for reading comprehension, with the final goal of releasing 1 million questions and answers. The authors have currently released 100,000 queries and their answers. The dataset differs from existing reading comprehension datasets mainly w.r.t queries being sampled from user queries rather than being generated by crowd-workers and answers being generated by crowd-workers rather than being spans of text from the provided passage. The paper presents some analysis of the dataset such as distribution of answer types. The paper also presents the results of some generative and some cloze-style models on the MS MARCO dataset.  Strengths:  1. The paper provides useful insights about the limitations of the existing reading comprehension datasets – questions asked by crowd-workers have different distribution compared to that of questions asked by actual users of intelligent agents, answers being restricted to span from the reading text rather than requiring reasoning across multiple pieces of text/passages.  2. MS MARCO dataset has novel useful characteristics compared to existing reading comprehension datasets – questions are sampled from user queries, answers are generated by humans.  3. The experimental evaluation of the existing baseline models on the MS MARCO dataset is satisfactory.  Weaknesses/Suggestions:  1. The paper does not report human performance on the dataset. Human performance should be reported to estimate the difficulty of the dataset. The degree of inter-human agreement will also reflect how well the metric (being used to compute inter-human agreement and accuracies of the baseline models) can deal with variance in the sentence structure with similar semantics.  2. I would like to see the comparison between the answer type distribution in the MS MARCO dataset and that in existing reading comprehension datasets such as SQuAD. This would ground the claim made in the paper the distributions of questions asked by crowd-workers is different from that of user queries.  3. The paper uses automatic metrics such as ROUGE, BLEU for evaluating natural language answers. However, it is known that such metrics poorly correlate with human judgement for tasks such as image caption evaluation (Chen et al., Microsoft COCO Captions: Data Collection and Evaluation Server, CoRR abs/1504.00325 (2015)). So, I wonder how authors justify using such metrics for evaluating open-ended natural language answers.  4. The paper mentions that a classifier was used to filter answer seeking queries from all Bing queries. It would be good to mention the accuracy of this classifier. This will provide insights into what percentage of the MS MARCO questions are answer seeking queries. Similarly, what is the accuracy of the information retrieval based system used to retrieve passages for filtered queries?  5. Please include the description of the best passage baseline in the paper.    6. Fix opening quotes, i.e. ” -> “ (for instance, on page 5, ”what” -> “what”).  Review Summary: The paper is well motivated, the use of user queries and human generated answers makes the dataset different from existing datasets. However, I would like to see the human performance on the dataset and quantitative comparison between the distribution of questions obtained from user queries and that of crowd-sourced questions. I would also like the authors to comment on the use of automatic metrics (such as ROUGE, BLEU) in the light of the fact that such metrics do not correlate well with human judgements for tasks such as image caption evaluation.

Answer: en


Question: Observación de fondo:  No se presenta un ejemplo de implementación (un ejemplo que sea interesante) de una máquina secuencial mediante la metodolgía propuesta y la tradicional cableada con el fin de establecer comparativos y métricas de desempeño.  Observaciones de forma: realizar las siguientes correcciones:  Omitir el punto final del título en español.  En página 3, segunda columna: "... entrada y las variables de estado de la máquina a secuencial."  En página 4, primera columna: justificación a la izquierda.  En página 5, primera columna: "... cantidad de puertos de entrada y salid ..."  En página 5, primera columna: "... completo a partir de la dirección de cada de cada ...", "... máquina con hasta 15 considerando la suma de ...".  En página 5, segunda columna: "... máquina con hasta 15 considerando la suma de ...".

Answer: es


Question: After rebuttal:  Thanks for reporting the AlexNet results. The fact that they are not great is not so bad by itself, and as the authors mention, it would be interesting to understand why this happens. But the fact that these results  were not in the paper (and in fact still are not there) is disturbing. Moreover, some claims in the paper look wrong in the light of these results, for example: - "This suggests that our gains stem from the CC-GAN method rather than the use of a better architecture." - "Since discrimination of real/fake in-paintings is more closely related to the target task of object classification than extracting a feature representation suitable for in-filling, it is not surprising that we are able to exceed the performance of Pathak et al. (2016) on PASCAL classification."  These statements, and possibly other parts of the paper, have to be updated. I think the paper cannot be published in its current form. Perhaps after a revision.  -------- Initial review:  The paper demonstrates an application of generative adversarial networks (GAN) to unsupervised feature learning. The authors show that the representation learned by the discriminator of a conditional GAN trained for image inpainting performs well on image classification. As a side-effect, fairly convincing inpaintings are produced.  The proposed method combines two existing ideas: using the discriminator of a GAN as a feature learner [Radford et al. 2015] and performing unsupervised feature learning with image inpainting [Pathak et al. 2016]. Therefore conceptual novelty of the paper is limited. On the plus side, the authors implement their idea well and demonstrate state-of-the-art results on STL-10 and good results on Pascal VOC (although Pascal experiments are incomplete, see below). Overall, I am in the borderline mode, and I will gladly raise the score if the authors address my concerns regarding the experiments.  1) Experimental evaluation on Pascal VOC is not quite satisfactory. Comparison with prior work is unfair because the network architecture used by the authors (VGG) is different from the architecture used by all existing methods (AlexNet). It is great that the authors do not try to hide this fact in the paper, but I do not understand why the authors are not willing to simply run their method with AlexNet architecture, although two commenters asked them to do so. Such an experiment would strongly support authors’ claims. Current reasoning that “we thought it reasonable to use more current models while making the difference clear” is not convincing. It is great that better architectures lead to better results, but it is also very important to properly compare to prior work. On a related topic, Doersch et al. also tried using VGG architecture, would it be possible to compare to that? Yet another question: why are you not comparing to [Noroozi&Favaro, ECCV 2016] ? I would also like the authors to address the comment by Richard Zhang.  2) Qualitative inpainting results are incomplete: comparison with previous methods (for instance, [Pathak et al 2016]) is missing, and it is impossible to compare different versions of the proposed method because different images are used for different variants. I realize there may be too little space in the main paper to show all the results, but many more results should be shown in the supplementary material. Quantitative results are missing. Currently the inpainting results are just interesting pictures to look at, but they do not add as much to the paper as they could.

Answer:
en