Task: Language Identification
Input (Spanish, English): The paper presents a way to "learn" approximate data structures. They train neural networks (ConvNets here) to perform as an approximate abstract data structure by having an L2 loss (for the unrolled NN) on respecting the axioms of the data structure they want the NN to learn. E.g. you NN.push(8), NN.push(6), NN.push(4), the loss is proportional to the distance with what is NN.pop()ed three times and 4, 6, 8 (this example is the one of Figure 1).  There are several flaws:  - In the case of the stack: I do not see a difference between this and a seq-to-seq RNN trained with e.g. 8, 6, 4 as input sequence, to predict 4, 6, 8.  - While some of the previous work is adequately cited, there is an important body of previous work (some from the 90s) on learning Peano's axioms, stacks, queues, etc. that is not cited nor compared to. For instance [Das et al. 1992], [Wiles & Elman 1995], and more recently [Graves et al. 2014], [Joulin & Mikolov 2015], [Kaiser & Sutskever 2016]...  - Using MNIST digits, and not e.g. a categorical distribution on numbers, is adding complexity for no reason.  - (Probably the biggest flaw) The experimental section is too weak to support the claims. The figures are adequate, but there is no comparison to anything. There is also no description nor attempt to quantify a form of "success rate" of learning such data structures, for instance w.r.t the number of examples, or w.r.t to the size of the input sequences. The current version of the paper (December 9th 2016) provides, at best, anecdotal experimental evidence to support the claims of the rest of the paper.  While an interesting direction of research, I think that this paper is not experimentally sound enough for ICLR.
Output (English): en