In this task, you are given a paper review. Based on the review, your job is to identify language and generate "en" if the review is in English or generate "es" if the review is in Spanish. Note that URLs in the text have been replaced with [Link].

Ex Input:
This paper presents an intriguing study of how one can pose architecture search as a meta learning problem. By collecting features from networks trained on various datasets and training a “ranking classifier” (the actual details of the classifier do not seem to be described in detail) one can potentially infer what a good architecture for a new problem could be by simply running the ranker on the extracted features for a new problem setup.  One notable comment from the paper is that the authors fix some important hyper-parameters for all the networks. I am of the opinion that optimizing the learning rate (and its decay schedule) is actually quite important. I hypothesize that a lot of the conclusions of this paper may change quite a bit if the authors did an actual search over the rates instead. I suspect that instead of training 11k nets, one can train 2k nets with 5 learning rates each and get a much better result that is actually compelling.  I am not convinced that the protocol for generating the various architectures is doing a good job at creating a diversity of architecture (simply because of the max depth of 8 layers and 14 components overall). I suspect that most of these generated architectures are actually almost identical performance-wise and that it’s a waste to train so many of them on so many tasks. Unless the authors are already doing this, they should define a pruning mechanism that filters out nets that are too similar to already existing ones.  The batch normalization experiments in Table 2 seem odd and under-explained. It is also well-known that the optimal learning rates when using batch norm vs. not using batch norm can differ by an order of magnitude so given the fixed learning rate throughout all experiments, I take these results with some grain of salt.  I am not sure we got many insights into the kinds of architectures that ended up being at the top. Either visualizations, or trends (or both), would be great.  This work seems to conflate the study of parallel vs. serial architectures with the study of meta learning, which are somewhat distinct issues. I take issue with the table that compares parallel vs. serial performance (table 2) simply because the right way would be to filter the architectures by the same number of parameters / capacity.  Ultimately the conclusion seems to be that when applying deep nets in a new domain, it is difficult to come up with a good architecture in advance. In that sense, it is hard to see the paper as a constructive result, because it’s conclusions are that while the ranker may do a good job often-times, it’s not that reliable. Thus I am not convinced that this particular result will be of practical use to folks who are intending to use deep nets for a new domain.

Ex Output:
en


Ex Input:
## Paper summary  The paper reconsiders the idea of using a binary classifier to do two-sample testing. The idea is to split the sample into two disjoint training and test sets, train a classifier on the training set, and use the accuracy on the test set as the test statistic. If the accuracy is above chance level, one concludes that the two samples are from different distributions i.e., reject H0.  A theoretical result on an asymptotic approximate test power is provided. One implication is that the test is consistent, assuming that the classifier is better than coin tossing. Experiments on toy problems, evaluation of GANs, and causal discovery verify the effectiveness of the test. In addition, when the classifier is a neural net, examining the first linear filter layer allows one to see features which are most activated. The result is an interpretable visual indicator of how the two samples differ.  ## Review summary   The paper is well written and easy to follow. The idea of using a binary classifier for a two-sample testing is not new, as made clear in the paper. The main contributions are the analysis of the asymptotic test power, the use of modern deep nets as the classifier in this context, and the empirical studies on various tasks. The empirical results are satisfactorily convincing.  Although not much discussion is made on why the method works well in practice, overall contributions have a potential to start a new direction of research on model criticisms of generative models, as well as visualization of where a model fails. I vote for an acceptance.  ## Major comments / questions   My main concern is on Theorem 1 (asymptotic test power) and its assumptions.  But, I understand that these can be fixed as discussed below.  * Under H0, the distribution of the test statistic (i.e., sum of 0-1 classification results) follows Binomial(nte, 1/2) as stated.  However, under H1, terms in the sum are independent but *not* identical Bernoulli random variable. This is because each term depends on a data point z_i, which can be from either P or Q. So, in the paragraph in Sec3.1: "... the random variable n_te \hat{t} follows a Binomial(nte, p)..." is not correct. Essentially p depends on z_i. It should follow a Poisson binomial distribution.  * In the same paragraph, for the same reason, the alternative distribution of Binomial(nte, p=p_{risk}) is probably not correct. I guess you mention it to use Moivre-Laplace to get the asymptotic normality.   Anyway, I see no reason why you would need this statement as the Binomial is not required in the proof, but only its asymptotic normality. A variant of the central limit theorem (instead of the Moivre-Laplace theorem) for independent, non-identical variables would still allow you to conclude the asymptotic normality of the Poisson binomial (with some conditions). See for example

Ex Output:
en


Ex Input:
Summary: The paper proposes a neural model for predicting Python syntax trees from text descriptions. Guided by the actual Python grammar, the model generates tree nodes sequentially in a depth-first fashion. Key ideas include injecting the information from the parent node as part of the LSTM input, a pointer network for copying the terminals, and unary closure which collapses chains of unary productions to reduce the tree size. The model is evaluated on three datasets from different domains and outperforms almost all previous work.  Strengths:  The paper is overall very well-written. The explanation of system is clear, and the analysis is thorough.  The system itself is a natural extension of various ideas. The most similar work include tree-based generation with parent feeding (Dong and Lapata, 2016) and various RNN-based semantic parsing with copy mechanism (Jia and Liang, 2016; Ling et al., 2016). [The guidance of parsing based on grammar is also explored in Chen Liang et al., 2016 ([Link] where a code-assist system is used to ensure that the code is valid.] Nevertheless, the model is this paper stands out as it is able to generate much longer and more complex programs than most previous work mentioned.   Weaknesses:  The evaluation is done on code accuracy (exact match) and BLEU score. These metrics (especially BLEU) might not be the best metrics for evaluating the correctness of programs. For instance, the first example in Table 5 shows that while the first two lines in boxes A and B are different, they have the same semantics. Another example is that variable names can be different. Evaluation based on what the code does (e.g., using test cases or static code analysis) would be more convincing.  Another point about evaluation: other systems (e.g., NMT baseline) may generate code with syntactic error. Would it be possible to include the result on the highest-scoring well-formed code (e.g., using beam search) that these baseline systems generate? This would give a fairer comparison since these system can choose to prune malformed code.  General Discussion:  * Lines 120-121: some approaches that use domain-specific languages were also guided by a grammar. One example is Berant and Liang, 2014, which uses a pretty limited grammar for logical forms (Table 1). In addition to comparing to that line of work, emphasizing that the grammar in this paper is much larger than most previous work would make this work stronger.  * Lines 389-397: For the parent feeding mechanism, is the child index being used? In other words, is p_t different when generating a first child versus a second child? In Seq2Tree (Dong and Lapata, 2016) the two non-terminals would have different hidden states.  * Line 373: Are the possible tokens embedded? Is it assumed that the set of possible tokens is known beforehand?  * The examples in the appendix are nice.  ---  I have read the author response.

Ex Output:

en