Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: The authors observed that some lookahead information during training time is helpful to improve the translation accuracy for simultaneous translation. Base on this observation, this paper proposes to use RL-based methods to learn a certain number of lookahead words during the training of the wait-k-based simultaneous translation model.

This paper proposes a new approach for improving the translation quality and the results indeed show some improvements over the baseline methods. However, I still have the following concerns:
1) I think the proposed RL-based methods are very completed (in terms of hyperparameter searching, extra training time compared with other baselines) but the improvements are quite marginal compared with wait-k* and random.

2) For the experiments, the authors did not compare with other agent-based or adaptive methods. 

3) This paper only designs a controller for training. I think we could also have a controller for the inference. In this way, I believe we could use regular wait-k training, and use a controller to decide a smaller k during inference. Then it will be very similar to Gu. et al's RL-based methods. I suggest the authors include this method's experiments as well.

4) If I did not misunderstand, the m of wait-m is defined on each training pair, but I think this will dramatically increase the training time and hard to do batch training. How slow is your training compared with baseline wait-k? I believe the baseline wait-k is already very slow.
review_title: complicated method with marginal improvements

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper proposes a training strategy for simultaneous translation to choose appropriate amount of look-ahead information for each decoding. Based on the observation that the wait-k method can be improved by training with longer information, the method introduces a function to determine its length given the current example (source x, target y, and the translation model f). It is used as only a guidance during training, and it remains the same decoding criterion at inference.

The method is interesting and has promising improvements compared with bare wait-k methods according to the experiments, but the paper seems to have some major questions which should affect the conclusion. I recommend to revise the paper appropriately, especially to resolve following concerns:

- The proposed method is intuitively strange because of mismatching between training/inference strategies. It is also unclear to choose this method rather than adaptive wait-k methods, e.g., one referred as Zheng et al. (2020a), which may solve a similar problem directly. The paper needs at least some comparison of these kind of methods to figure out the advantages of the proposed method.
- Algorithm 1 involves a suspicious use of the development set: it is used directly to optimize a parameter. Specifically, since \omega is optimized using the D_va, it brings information of the D_va into f, resulting that the training process does not include any strategy to avoid overfitting (in other words, your training set is actually D_tr + D_va, and there is no so-called development set).

Minor comments:

- The title sounds misleading: the proposed method still does not learn how to use future information because it uses only k look-ahead information at inference (same as usual wait-k methods), i.e., there is nothing special to represent "future".
- Figure 1 involves some common mistakes of using bar charts: it must use 0 as the origin, and must not shorten the bar. If you want to focus on differences between each value, you should use other chart.
review_title: Promising methods, but suspicious settings of model training

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes a new training method for wait-k simultaneous translation. Rather than training on prefix pairs where the target prefix lags the source by k tokens, it uses an RL controller to determine an optimal lag for each sentence pair. The controller uses a small set of features intended to capture training progress, and is trained with REINFORCE to minimize wait-k loss on a validation set, in alternation with main training steps. This method shows consistent gains over various wait-k training heuristics, and some gains over other approaches that adapt the lag at inference time.

The paper is very well written and organized. The method makes sense, and the experiments are quite thorough, comparing to a competitive set of heuristic baselines, and showing credible - though fairly modest - gains in this setting.

The comparisons to adaptive baselines are less convincing. They are shown only for one small-data language pair, no implementation details are given (for instance, architectures and model capacities), and the relative results are very different from those in the literature (MILK << wait-k, WIW, WID). I think the paper would be stronger if these were simply omitted.

Given its ease of implementation and efficiency, I think there is room for a paper focusing on improving wait-k training, even if wait-k isn’t quite state of the art. But having to set up an RL controller detracts from this picture, especially since the gains over various heuristics - in particular the random heuristic that works for any inference-time k - aren’t spectacular. My main reaction to this work is that it should be possible to get most of the gains using some predetermined curriculum inspired by figure 5 (random sampling at the beginning, annealing to some m > k) that is effective for a broad range of inference-time k’s. Note that this is quite different from the CL baseline included in the results. As it stands, I fear this paper risks being a dead end: too complex to be worth implementing, as the field moves on past wait-k.

Questions and suggestions:

1. Controller feature (6): will this value ever repeat during an entire training run?
2. For Mk, since this runs only over the validation set and you are using RL, why not use the actual wait-k BLEU score?
3. You need to include all heuristic baselines in figure 4.
review_title: Using a sledgehammer to kill a fly?

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper improves the wait-k based simultaneous NMT by training on an adaptive wait-m policy. The proposed method and experiments are clearly described. Experiments demonstrate that the proposed method is significant better than the wait-k baseline.

However, compared to heuristic-based baselines, seemingly the proposed method is not significantly better in most cases, especially on the WMT En-De dataset. Given that the WMT dataset is much larger than IWSLT datasets, does this suggest that the proposed method may not work well on larger dataset?

I’m also curious about the reason of using adaptive wait-m only during training since it is a more straightforward idea to apply adaptive policy during both training and inference. Would it be better if the adaptive policy (by re-design features) was used on inference as well? 

review_title: The effectiveness of the proposed method is not convincing

Meta-review: This paper improves the wait-k based simultaneous NMT by training on an adaptive wait-m policy with a controller determining the lag for sentence pair.  The controller is trained with RL to minimize the loss on a validation set. The overall model is reasonable, which is well presented. I however have the following two concerns
1. There is a clear mismatch between training/inference strategies, which raises two problems
    1. The motivation:  the authors tried to explain that in discussion,  but it is not convincing enough
    2. The title is misleading since there is no future information to use during inference 
2. The experiments is not convincing enough in that a) the improvement over baseline is modest, and b) comparison to adaptive wait-k and other strong baseline is insufficient 

In conclusion I would suggest to reject this paper.


Decision: Reject