Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: This paper proposes a novel training scheme for seq2seq models where attention or reference alignment is used in combination with free-running mode for improving training.

The positives of this paper are that it is well written and very clear. It also is very relevant as seq2seq models can be hard to train and techniques like scheduled sampling and x-forcing algorithms are good heuristics but heuristics none-the-less.

The downside of this paper is in the experimental results and also complexity. It would’ve been good to see a broader set of experiments to really benchmark attention-forcing from other self-attention models.

Attention forcing also requires a reference or ground-truth alignment, which is often not available. Hence the authors propose to simultaneously train another teacher-forcing model to estimate the reference alignment. However, this would incur twice the computation complexity. 

Attention forcing could also be used in conjunction with scheduled sampling. How does that compare with the reported results for attention forcing?
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: This paper proposes a method for fixing exposure bias (ie. training vs generated distribution mismatch) in seq2seq modeling with attention, particularly for the application of speech synthesis where reference alignments are available.


Related Work is missing:

- Another paper that studies fixing exposure bias in seq2seq learning:

Wiseman & Rush. Sequence-to-Sequence Learning as Beam-Search Optimization
https://arxiv.org/pdf/1606.02960.pdf

- Other papers that try to enforce attention to attend to specific locations:

Bao et al. Deriving Machine Attention from Human Rationales. https://arxiv.org/abs/1808.09367

Liu et al. Neural Machine Translation with Supervised Attention. https://www.aclweb.org/anthology/C16-1291/

Yu et al. Supervising Neural Attention Models for Video Captioning by Human Gaze Data. https://arxiv.org/abs/1707.06029

Without the comparison against other related papers that also aim to supervise attention mechanisms (there are other beyond the ones I cited above)s, it is unclear how much is novel about this paper.

- Furthermore, it is conceptually clear to me that attention-forcing fully matches the training vs generated distributions. The authors should describe in greater detail why this happens this, or whether these distributions are not required to fully match in attention-forcing (and in this case, why this would be desirable).

- The experiments are not very convincing (only 30 human evaluators for Speech synthesis with no other quantitative evaluation, NMT results that are not particularly promising).

- Use of non-anonymous github link is questionable for blinded submissions.
review_title: Official Blind Review #1

final_score: rating: 1: Reject
review_content: This paper proposes an alternative mechanism of training the attention values of a sequence to sequence learning model as applied to tasks like speech synthesis and translation.  During training they compute two forms of attention: (1) the standard soft-attention from a decoder fed with teacher forced output, and (2) the inference-time attention from a decoder fed with predicted outputs.  Their training objective consists of two terms: The first is the token-wise cross entropy loss but by conditioning on the predicted output  but with teacher-forced attention.  The second is a KL distance between the above two types of attention distributions.   Experiments with mechanical  turks indicate that their attention forcing mechanism is strongly preferred over the existing teacher forced output and attention model.  On translation their method provides little or no improvement.

I am inclined towards rejecting the paper because the experiment and related work section still requires a lot of work before 1. The claimed utility of the idea is established, and 2. The novelty over the many existing attention architectures is established.   I elaborate on each of these next.

Related work: Recently, many papers have directly or indirectly handled the problem of exposure bias that this paper attempts to address.  The paper does not discuss most of these.  Here are some that are missed from the paper:

1.   Sequence level training with recurrent neural networks
MA Ranzato, S Chopra, M Auli, W Zaremba, 2015.
This paper shows that the scheduled sampling method (discussed in the paper) is much worse than a reinforce-based training mechanism of handling exposure bias.  

2. An actor-critic algorithm for sequence prediction
D Bahdanau, P Brakel, K Xu, A Goyal, R Lowe

3.  Posterior Attention Models for Sequence to Sequence Learning
S Shankar, S Sarawagi - 2019

4. Latent Alignment and Variational Attention
Yuntian Deng, Yoon Kim, Justin Chiu, Demi Guo, Alexander M. Rush 2018

5. Attention Focusing for Neural Machine Translation by Bridging Source and Target Embeddings
Shaohui Kuang, Junhui Li, António Branco, Weihua Luo, Deyi Xiong

Experiments:  Their experiments are rather sketchy and limited.
The TTS experiments are only on one dataset.  Their method is compared only with the standard seq2seq learning approach.  Even the scheduled sampling or professor forcing methods are not compared with.  In addition, state of the art TTS methods have gained significantly from hierarchical attention.  As such as far as the TTS task is concerned the significance of the improved quality over a baseline seq2seq method is limited.

For translation they consider only the English-Vietnamese task whereas there are tens of other translation tasks that are used in recent literature.

Overall, the idea proposed seems quite incremental, experiments are limited, and related work discussion incomplete.

*********
I read the author response but I do not think the paper is ready for publication yet without the thorough comparison with related work.

review_title: Official Blind Review #3

Meta-review: The paper proposed an attention-forcing algorithm that guides the sequence-to-sequence model training to make it more stable. But as pointed out by the reviewers, the proposed method requires alignment which is normally unavailable. The solution to address that is using another teacher-forcing model, which can be expensive. 

The major concern about this paper is the experimental justification is not sufficient:
* lack of evaluations of the proposed method on different tasks;
* lack of experiments on understanding how it interact with existing techniques such as scheduled sampling etc;
* lack of comparisons to related existing supervised attention mechanisms. 

Decision: Reject