In this task, you are given a paper review. Based on the review, your job is to identify language and generate "en" if the review is in English or generate "es" if the review is in Spanish. Note that URLs in the text have been replaced with [Link].
--------
Question: This paper proposed to use RL and RNN to design the architecture of networks for specific tasks. The idea of the paper is quite promising and the experimental results on two datasets show that method is solid. The pros of the paper are: 1. The idea of using RNN to produce the description of the network and using RL to train the RNN is interesting and promising. 2. The generated architecture looks similar to what human designed, which shows that the human expertise and the generated network architectures are compatible.  The cons of the paper are: 1. The training time of the network is long, even with a lot of computing resources.  2. The experiments did not provide the generality of the generated architectures. It would be nice to see the performances of the generated architecture on other similar but different datasets, especially the generated sequential models.  Overall, I believe this is a nice paper. But it need more experiments to show its potential advantage over the human designed models.

Answer: en


Question: This paper introduces a new approach to semantic parsing in which the model is equipped with a neural sequence to sequence (seq2seq) model (referred to as the “programmer”) which encodes a natural language question and produces a program. The programmer is also equipped with a ‘key variable’ memory component which stores (a) entities in the questions (b) values of intermediate variables formed during execution of intermediate programs. These variables are referred to further build the program.                    The model is also equipped with certain discrete operations (such as argmax or 'hop to next edges in a KB'). A separate component ("interpreter/computer") executes these operations and stores intermediate values (as explained before). Since the ‘programmer' is inherently a seq2seq model, the "interpreter/computer” also acts as a syntax/type checker only allowing the decoder to generate valid tokens. For example, the second argument to the “hop” operation has to be a KB predicate. Finally the model is trained with weak supervision and directly optimizes the metric which is used to evaluate the performance (F score). Because of the discrete operations and the non differentiable reward functions, the model is trained with policy gradients (REINFORCE). Since gradients obtained through REINFORCE have high variance, it is common to first pretrain the model with a max-likelihood objective or find some good sequences of actions trained through some auxiliary objective. This paper takes a latter approach in which it finds good sequences via an iterative maximum likelihood approach. The results and discussion sections are presented in a very nice way and the model achieves SOTA results on the WebQuestions dataset when compared to other weakly supervised model.  The paper is written clearly and is very easy to follow.  This paper presents a new and exciting direction and there is scope for a lot of future research in this direction. I would definitely love to see this presented in the conference.  Questions for the authors (important ones first)  1. Another alternative way of training the model would be to bootstrap the parameters (\theta) from the iterative ML method instead of adding pseudo gold programs in the beam (Line 510 would be deleted). Did you try that and if so why do you think it didn’t work? 2. What was the baseline model in REINFORCE. Did you have a separate network which predicts the value function. This must be discussed in the paper in detail. 3. Were there programs which required multiple hop operations? Or were they limited to single hops. If there were, can you provide an example? (I will understand if you are bound by word limit of the response) 4. Can you give an example where the filter operation would be used? 5. I did not follow the motivation behind replacing the entities in the question with special ENT symbol  Minor comments: Line 161 describe -> describing Line 318 decoder reads ‘)’ -> decoder generates ‘)'

Answer: en


Question: The paper describes an experience concerning the automated inspection of spectra for the Pipeline Hubble Legacy Archive Grism data.  Comments:  In the conclusions the authors say that "We have identified two classes of flawed spectra which were not picked up by the automatic classification because of their very small number of training samples. Per construction machine learning techniques can not classify such outliers." This sentence should be further explained. As Hastie, Tibshirani, and Friedman say in The Elements of Statistical Learning (see Chapter 7): "it is too difficult to give a general rule on how much training data is enough; among other things, this depends on the signal-to-noise ratio of the underlying function, and the complexity of the models being fit to the data. " So, by just saying that the number of training samples is small you do not provide enough information to decide whether machine learning techniques are adequate or not to solve a problem.  Other minor comments:  + If reference [2] has not yet been written or published it should be deleted from the paper.  The dataset may be of scientific importance.  It is mainly descriptive.

Answer:
en