Task: Language Identification
Input (Spanish, English): SYNOPSIS: The paper proposes a new neural network-based model for reading comprehension (reading a passage of text and answering questions based on the passage). It is similar in spirit to several other recent models, with the main exception that it is able to predict answers of different lengths, as opposed to single words/tokens/entities. The authors compare their model on the Stanford Question Answering Dataset (SQuAD), and show improvements over the baselines, while apparently lagging quite far behind the current state of the art reported on the SQuAD leaderboard.  THOUGHTS: The main novelty of the method is to be able to identify phrases of different lengths as possible answers to the question. However, both approaches considered -- using a POS pattern trie tree to filter out word sequences with POS tags matching those of answers in the training set, and brute-force enumeration of all phrases up to length N -- seem somewhat orthogonal to the idea of "learning end-to-end " an answer chunk extraction model. Furthermore, as other reviews have pointed out, it seems that the linguistic features actually contribute a lot to the final accuracy (Table 3). One could argue that these are easy to obtain using standard taggers, but it takes away even more from the idea of an "end-to-end trained" system.  The paper is generally well written, but there are several crucial sections in parts describing the model where it was really hard for me to follow the descriptions. In particular, the attention mechanism seems fairly standard to me in a seq2seq sense (i.e. there is nothing architecturally novel about it, as is for instance the case with the Gated Attentive Reader). I may be missing something, but even after the clarification round I still don't understand how it is novel compared to standard attention used in for instance seq2seq models.  Finally, although the method is shown to outperform the baseline method reported in the original paper introducing the SQuAD dataset, it currently seems to be 12th (out of 15 systems) on the leaderboard (
Output (English): en