Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper presents a self-training approach for improving sequence-to-sequence tasks. As a preliminary experiment, this study randomly sampled 100k sentences from WMT 2014 English-German dataset (WMT100K, hereafter), trained a baseline (Transformer) model on WMT100K, and applied self-training methods on the remaining English sentences as the unlabeled monolingual data. After exploring different procedures for self-training, this study uses the fine-tuning strategy: train a model on the supervision data; build pseudo parallel data by predicting translations for all unlabeled data using the trained model; train a new model on the pseudo parallel data; and fine-tune the new model on the supervision data. This strategy alone gave a 3 points improvement of BLEU.

This paper hypothesized the reasons behind the performance improvements: beam-search decoding (+0.5 BLEU) and dropout (+1.2 BLEU). The authors argue that the beam-search decoding contributes partially to the performance gains, while the dropout accounts for the most. The authors infer that the dropout causes an implicit perturbation. Exploring different perturbation strategies, synthetic noise (e.g., input tokens are randomly dropped, masked, and shuffled) and paraphrase (round-trip translation, e.g., English-German-English), the authors reported no significant difference between these two strategies. Finally, this paper reports empirical results (on WMT 2014 English-German, FloRes English-Nepali, and English Gigaword (summarization task)) of self-training strategies presented in this paper. This paper concluded that self-training can be an effective method to improve generalization and that the noise injected during self-training plays a critical role for its success.

This paper is well structured and well written. It is interesting to see the improvements of the sequence-to-sequence tasks by using the self-training approach. This paper is interesting because it has a close connection with the back-translation approach that has been popular in recent years.

Although the hypotheses presented in this paper are interesting, they were not fully validated after all. The analyses on the loss functions, ablation tests, and experiments on the toy task can only bring indirect explanations about why we could observe the performance improvements. This impression is also demonstrated by the fact that this paper uses 'might' five times when explaining interpretations of the experimental results. Having said, I tend to agree that identifying the exact reason is difficult.

However, I have two other questions before recommending this paper: (1) whether the baseline model was trained sufficiently, and (2) whether this paper is about self-training strategies or regularization methods. 

(1) In order to accept the experimental results that the self-training approach can improve the performance, we need to make sure that the baseline model was trained sufficiently. However, the appendix explains, "we basically use the same learning rate schedule and label smoothing as in fairseq examples." I'm not sure whether this training procedure was fair among different models because the baseline model and self-supervised model received totally different number of training instances (100k vs 3.9M). This claim would be stronger if this paper could show an evidence that the baseline model was trained properly by, for example, explaining the stopping criteria for iterations, tuning hyper-parameters (e.g., learning rate) individually for the baseline and self-trained models, showing the mean and variance of BLEU scores with different initializations, and showing the training curve of the baseline model.

(2) We can view the self-training strategies presented in this paper as regularization methods. For this reason, I am wondering whether the self-training strategies presented in this paper are only for self-training or general to pure supervised setting. We can easily guess that the performance would drop if we removed the dropout from the baseline method. In contrast, I would like to see whether the synthetic noise could improve the performance of the baseline method alone, behaving as a regularization method. It would be useful to see the performance of the baseline method without the dropout and with the synthetic noise to highlight the effect of the presented strategies under the self-training scenario.

Minor comments:

It would be useful to see the number of unlabeled instances in Section 3.1.

Sectoin 3.2: "This is different from back-translation where new knowledge may originate from an additional backward translation model."
I'm not sure whether a backward translation model can introduce new knowledge because the supervision data are usually the same between forward and backward directions.

Section 3.3: "at test/decoding time the model does not use dropout"
To be precise, the weights are scaled by the dropout rate (p) in the decoding time.

Reference: Lample et al. 2018 should be replaced with another paper:
Guillaume Lample, Alexis Conneau, Ludovic Denoyer, Marc'Aurelio Ranzato. Unsupervised Machine Translation Using Monolingual Corpora Only. ICLR 2018.
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper investigates why self-training helps in machine translation and text summarization tasks, identifying that auxiliary noise can amplify the benefits of this process. The paper is well-structured and clearly written, and conducts a fairly thorough analysis of the issue at hand.

Comments:

- The authors argue self-training enhances smoothness, but I would like to see this explained mathematically/conceptually in greater detail. It is not immediately clear to me why this would be, particularly in the case of discrete text data.

- Why not evaluate smoothness on the actual MT task instead of just the toy task?
The authors could measure the L2 norm between encodings of source sentences for neighboring sentences (eg. based on edit distance or word-movers) vs very different sentences. And then compare the base model vs the one obtained from self-supervised training.

- If the primary beneficial effect of self-supervised training is smoothness as the authors claim, then they should try enforcing smoothness in alternative ways to see if performance improves.  Some options here could be (using dropout in all of them): 1) add your same noise process to the original labeled dataset to create augmented examples, 2) rather than the self-supervised objective, use an auxiliary training objective which says the predictions on each unlabeled datapoint should be similar to the predictions on noised versions of this datapoint, 3) some form of virtual adversarial training [VAT]. In fact, the authors should discuss [VAT] a bit more, as this paper also presents a smoothness-regularizer that is highly useful for semi-supervised learning.

[VAT] Miyato et al. Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning. https://arxiv.org/abs/1704.03976


- As the authors write, self-training can be viewed as a form of entropy-regularization. Likewise, the input perturbation+dropout process also seems like it should affect the conditional entropies. Can the authors expound upon this connection a bit further?  Some mathematical analysis would be nice to have here as well.


- "Another way is to treat them separately – first we train the model on pseudo parallel data S, and then fine-tune it on real data L."

The authors should clarify the overall training process with a more precise explanation. I assume is actually: 
1) train initial model M on (limited) real data L
2) use M to generate pseudo-targets for unlabeled data in S
3) train M on this pseudo-dataset S
4)fine-tune M on real data L

Is this correct? And it should be clarified whether M in step (3) is fine-tuned from the M in step (1) or re-initialized from scratch before training begins (Based on later text, it seems like the latter, but this should be clarified early on).

- Baseline in Figure 1 should be described a bit more clearly.

- Since the BLEU score dropped from 3 to 1.9 when the authors continued training from the baseline model (Sec 3.2), isn't the optimum hypothesis not ruled out?  I don't think the authors should make this claim, and rather state that the initialization does seem to play some role, but does not fully explain the benefits of self-training.  

- "We use a small LSTM model for 10K, Base Transformer for 100K/640K, and Big Transformer for 3.9M" 

Is this because these are the best performing models on these respective datasets? The authors should explain these decisions.


- "We include quantitative comparison regarding joint training, separate training, and pseudo-parallel data filtering in Appendix B"

Should clarify here (in main text) that separate training matches the performance of joint training.

- typos: "joint traing"
review_title: Official Blind Review #1

final_score: rating: 8: Accept
review_content: The paper introduces an interesting study that tries to explain why conditional text generation models with autoregressive decoders benefit from self-training on pseudo labels created from the same model. The paper introduces and verifies two hypotheses: 1) Decoding strategy: Since beam search is a biased estimator sampling using it doesn't reflect the learned distribution from the model and hence variations happen that benefit learning. (this partially help).
Authors verify that by replacing the beam search decoder by an unbiased sampling method for decoding. 2) Additional noise during training: The use of dropout adds discrepancy between training and inference which creates favourable regularization. Through smoothing of the latent spaces so similar semantic inputs get mapped to similar outputs.  Authors verify that using a toy task of summing two numbers. 

Following this intuition, authors introduce a new method of noisy self-training that even enforces hypothesis #2 (rather than increasing dropout which isn't practical to train). Authors experiment with word perturbation from (Lample et al. 18) and paraphrasing through translation back and forth.

Authors provide a thorough experimental section evaluating the noisy ST on Machine translation, low resource MT and Summarization the latter two latter are especially interesting to use ST rather than Back translation as target side documents might be hard to find (low resource MT) or quite challenging to recover the source side from (in case of summarization ).

Additional analysis experiments were performed to show the effect noisy self-training with regard to, increasing noise, number of available parallel data, size of ST samples generated from monolingual data.

I am in favour of this work acceptance, overall the paper introduces very interesting insights to explain the usefulness of self-training for auto-regressive generation tasks and could inspire future work along this line in designing better ST algorithms and/or adoption of self-training in low-resourced generation tasks. 


Questions to authors: 
As explained in section 2, you preferred to model the ST task as an iterative process between pseudo training and fine-tuning was mainly chose for simplicity while providing equal results as shown in Table 6 (appendix). I wonder why joint training does provide lower results than separate training? I doubt this might be due to that it is given less time to converge compared to the 3 iterations of self-training. Can you provide more details about how this is done?

Figure 4 (appendix), I was wondering if there's an intuition about the artefacts in the error heat maps of the toy task ( the patterns with -45 degrees slope)


Suggestions to enhance readability:
A lot of abbreviations finish by "T" this makes it quite hard to follow, I would suggest authors to remove "PT" and "FT" since they haven't been used much in the paper. 

Labels in Figure 1: especially early in the paper becomes hard to grasp, would be nice if you can add description to what each abbreviation means in the figure caption. 

typo table 6 appendix: joint "traing"








review_title: Official Blind Review #3

Meta-review: This paper analyzes self-training for sequence-to-sequence models and proposes a noisy version of self training. An empirical study shows the proposed noisy version improves results for machine translation and summarization tasks.

All reviewers appreciate the interesting contributions of the research, as well as clear writing. They also offer several comments for the revision of the paper. 

We look forward to seeing this paper presented at the conference!
Decision: Accept (Poster)