Venue: ICLR 2017 conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper is the first (I believe) to establish a simple yet important result that Convnets for NMT encoders can be competitive to RNNs. The authors present a convincing set of results over many translation tasks and compare with very competitive baselines. I also appreciate the detailed report on training and generation speed. I find it's very interesting when position embeddings turn out to be hugely important (beside residual connections); unfortunately, there is little analysis to shed more lights on this aspect and perhaps compare other ways of capturing positions (a wild guess might be to use embeddings that represent some form of relative positions). The only concern I have (similar to the other reviewer) is that this paper perhaps fits better in an NLP conference.

One minor comment: it's slight strange that this well-executed paper doesn't have a single figure on the proposed architecture :) It will also be even better to draw a figure for the biLSTM architecture as well (it does take some effort to understand the last paragraph in Section 2, especially the part on having a linear layer to compute z).
review_title: A well-executed NLP paper

final_score: rating: null
review_content: Hi, I have a few questions about your submission.

First, can you please explain in more detail why your convnet model is faster for decoding? Is it because less FLOPs are required? Or is it because you were able to use less hidden units? Are both implementations efficient? It's not obvious to me why 15+5 layers of convolution are two times faster than 2 layers of LSTM when a single CPU thread is used.

Second, a very interesting finding of the paper is that two convolutional network are needed for good performance, CNN-a and CNN-c. Can you provide a qualitative explanation for this phenomenon?

Finally, I think that the wording "conceptually simpler encoder" that you used in the conclusion is a very subjective judgement. Especially given that you have to use 2 different convnets.
review_title: A few questions

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper reports a very clear and easy to understand result that a convolutional network can be used instead of the recurrent encoder for neural machine translation. 

Apart from the known architectural elements, such as convolution, pooling, residual connections, position embeddings, the paper features one unexpected architectural twist: two stacks of convolutions, one for computing alignment and another for computing the representations.
The empirical evidence that this was necessary is provided, however the question of *why* it is necessary remains open. 

The experimental evaluation is very extensive and leaves no doubt that the proposed approach works well. The convnet-based model was faster at evaluation, but it is not very clear what is the main speed-up factor. It’s however hard to argue against the fact that the speed advantage of convnets is likely to increase if a more parallel implementation is considered. 

My main concern is whether or not the paper is appropriate for ICLR, because the contribution is quite incremental and rather application-specific. ACL, EMNLP and other NLP conferences would be a better venue, I think. 
review_title: Good paper, but very incremental

final_score: rating: 7: Good paper, accept
review_content: The system described works comparably to bi-directional LSTM baseline for NMT, and CNN's are naturally parallelizable.

Key ideas include the use of two stacked CNN's (one for each of encoding and decoding) for translation, with res connections and position embeddings. The use of CNN's for translation has been attempted previously (as described by the authors), but presumably it is the authors' combination of various architectural choices (attention, position embeddings, etc) that make the present system competitive with RNN's, whereas earlier attempts were not. They describe system's sensitivity to some of these choices (e.g. experiments to choose appropriate number of layers in each of the CNN's).

The experimental results are well reported in detail.

One or two figures would definitely be required to help clarify the architecture.

This paper is less about new ways of learning representations than about the combination of choices made (over the set of existing techniques) in order to get the good results that they do on the reported NMT tasks. In this respect, while I am fairly confident that the paper represents good work in machine learning, I am not quite as confident about its fit for this particular conference.
review_title: Final Review: good paper applying CNN to translation to match bi-LSTM baseline

Meta-review: This work demonstrates architectural choices to make conv nets work for NMT. In general the reviewers liked the work and were convinced by the results but found the main contributions to be "incremental". 
 
 Pros:
 - Clarity: The work was clearly presented, and besides for minor comments (diagrams) the reviewers understood the work
 - Quality: The experimental results were thorough, "very extensive and leaves no doubt that the proposed approach works well".
 
 Mixed:
 - Novelty: There is appreciation that the work is novel. However as the work is somewhat "application-specific" the reviewers felt the technical contribution was not an overwhelming contribution.
 - Impact: While some speed ups were shown, not all reviewers were convinced that the benefit was sufficient, or "main speed-up factor(s)" were. 
 
 This work is clearly worthwhile, but the reviews place it slightly below the top papers in this area.
Decision: Reject