Instructions: In this task, you are given a paper review. Based on the review, your job is to identify language and generate "en" if the review is in English or generate "es" if the review is in Spanish. Note that URLs in the text have been replaced with [Link].
Input: The paper describes an MT training data selection approach that scores and ranks general-domain sentences using a CNN classifier. Comparison to prior work using continuous or n-gram based language models is well done, even though it is not clear of the paper also compared against bilingual data selection (e.g. sum of difference of cross-entropies). The motivation to use a CNN instead of an RNN/LSTM was first unclear to me, but it is a strength of the paper to argue that certain sections of a text/sentence are more important than others and this is achieved by a CNN. However, the paper does not experimentally show whether a BOW or SEQ (or the combination of both( representation is more important and why. The textual description of the CNN (one-hot or semi-supervised using pre-trained embeddings) is clear, detailed, and points out the important aspects. However, a picture of the layers showing how inputs are combined would be worth a thousand words. The paper is overall well written, but some parentheses for citations are not necessary (\citet vs. \citep) (e.g line 385). Experiments and evaluation support the claims of the paper, but I am a little bit concerned about the method of determining the number of selected in-domain sentences (line 443) based on a separate validation set: - What validation data is used here? It is also not clear on what data hyperparameters of the CNN models are chosen. How sensitive are the models to this? - Table 2 should really compare scores of different approaches with the same number of sentences selected. As Figure 1 shows, the approach of the paper still seems to outperform the baselines in this case. Other comments: - I would be interested in an experiment that compares the technique of the paper against baselines when more in-domain data is available, not just the development set. - The results or discussion section could feature some example sentences selected by the different methods to support the claims made in section 5.4. - In regards to the argument of abstracting away from surface forms in 5.4: Another baseline to compare against could have been the work of Axelrod, 2015, who replace some words with POS tags to reduce LM data sparsity to see whether the word2vec embeddings provide an additional advantage over this. - Using the sum of source and target classification scores is very similar to source & target Lewis-Moore LM data selection: sum of difference of cross-entropies. A reference to this work around line 435 would be reasonable. Finally, I wonder if you could learn weights for the sum of both source & target classification scores by extending the CNN model to the bilingual/parallel setting.
Output:
en