Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: Sorry, this is a very quick review.

The paper is about an improved method of training latency-limited (wait-k) decoders for transformer-based machine translation, in which the right context is limited to various numbers.  So it's a kind of augmentation method that's well matched to the test scenario.  At least, that is my understanding.

I am not really an MT expert so cannot comment with much authority.  On the plus side the paper says it sets a new state of the art for latency-limited decoding for a German-English MT task, and it involves transformers, which are quite hot right now so the  attendees might find it interesting because of that connection.
On the minus side, it is all really quite task-specific.
I am putting weak accept.. regular-strength accept might be my other choice.
It's all with low confidence.
review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: This paper extends the idea of prefix-to-prefix in STACL and proposes two different variations. The authors did some interesting experiments between caching and updating decoder.

My questions are as follows:

1) in section 3.1.2, the authors mentioned two adaptations. Is the proposed AR encoder uni-directional? If the AR encoder is uni-directional, then I would be surprised that the uni-directional encoder outperforms the bi-directional encoder in the original wait-k model. For the second bullet, I think the original wait-k also did the same thing(they mentioned this in the paper clearly). So there is nothing new about bullet 2. 

2) the idea mentioned in fig 1 is very similar to [1]. I suggest the authors compare with the aforementioned methods. 

3) updating the hidden state of the decoder introduces more complexity during the inference time. I recommend the authors to perform some analysis about decoding time with CPU and GPU.

4) it is also interesting to show more comparison between different models' training time with the original STACL. 

[1] Zheng et al. "Simultaneous Translation with Flexible Policy via Restricted Imitation Learning" ACL 2019
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: This work apply the wait-k decoding policy on the 2D CNN-based architecture and transformer.  In the transformer-based model the author proposed to recalculate the decoder hidden states when a new source token arrives. The author also suggested to train with multiple k at the decoder level with shared encoder output. The experiments showed that the transformer model provide the best quality on IWSLT14 En-De, De-En, and WMT15 De-EN.

The masking and using causal attention for the transformer has been proposed in previous works. The hidden state updates provide some gains for the model but also makes the decoder more expensive. The training with multiple k provides similar gain as training with one k larger than the value used at the inference time. Overall the contributions are limited.

There is quite some room for this paper to improve its clarify, especially in terms of annotations and explaining the proposed ideas.
review_title: Official Blind Review #2

Meta-review: The paper proposes a method of training latency-limited (wait-k) decoders for online machine translation. The authors investigate the impact of the value of k, and of recalculating the transformer's decoder hidden states when a new source token arrives. They significantly improve over state-of-the-art results for German-English translation on the WMT15 dataset, however there is limited novelty wrt previous approaches. The authors responded in depth to reviews and updated the paper with improvements, for which there was no reviewer response. The paper presents interesting results but IMO the approach is not novel enough to justify acceptance at ICLR. 

Decision: Reject