Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes relaxing CTC-based models, which train the model with the objectives that allow non-monotonic alignment between the output and the reference.  Furthermore, the authors propose to finetune a CTC with n-gram matching (n=1, 2). Experiments show that such approaches consistently improve performance, showing the effectiveness of relaxing objectives and finetuning strategy.
strengths_and_weaknesses: Overall, I think the paper is publishable, which has:
- The motivation for allowing non-monotonic alignment of the CTC model is clear.
- There is a systematic study on invoking non-monotonic alignment, and the authors show consistent improvements over receptive baselines.
- Their best variant achieves strong performance.
- The proposed SCTC provides a bridge to n-gram matching for the general CTC model, which is novel.
questions: n/a
limitations: n/a
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The multi-modality problem in Non-Autoregressive Machine Translation (NAT) makes non-monotonic alignment an important research direction. This paper explores the non-monotonic matching on latent alignment models. The authors first introduce a Simpliﬁed Connectionist Temporal Classiﬁcation (SCTC) where only blank tokens are removed in the collapsing function. Then they derive bipartite matching and n-gram matching as two non-monotonic alignments under SCTC. Finally, they generalize the n-gram matching method under SCTC to regular CTC. The experimental results show the superiority of non-monotonic training objectives, and the best method achieves comparable performance to AT models.
strengths_and_weaknesses: **Strengths**

* The motivation is clearly given and the paper is well-written.
* The proposed non-monotonic alignment methods may facilitate the development of this field.

**Weaknesses**

The relationship of this paper to related work is not very clear. What specific problems exist in the previous works are not sufficiently described.

Typo (line 108): the s translation ->  the translation ?
questions: What best method does NMLA stand for specifically? CTC+2-gram?
limitations: No majot limitations.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposed methods to train non-autoregressive NMT models by relaxing the monotonic alignment restriction in the CTC loss. The paper first proposed a simplified CTC (SCTC) which only takes the blank symbol into consideration and does not allow token repetitions. In order to taking non-monotonic alignments into consideration during training, two solutions, i.e., bipartite matching and n-gram matching, are introduced. The n-gram matching for the regular CTC is further explored.

Experiments are conducted on WMT En—De and En—Ro tasks. The results show the proposed bipartite matching and n-gram matching can significantly improve over the SCTC and CTC baseline, where the combination of n-gram and CTC achieves the best performance. Compared with related works, the proposed systems achieves comparable results.

strengths_and_weaknesses: Strengths:

The motivation of this paper is reasonable and clear as the regular CTC loss would penalize predictions which are correct but have different word orders.

Experiments show the proposed solutions for calculating non-monotonic alignment losses are effective. 


Weaknesses: 

Experimental results are not convincing enough. Compared with the existing Fully-NAT, the proposed NMLA systems achieves comparable results but slower speed. Although the proposed method show improvements over naïve baselines, whether the proposed method could improve Fully-NAT further as an addition is unknown which uses the CTC loss as well during training.

The paper writing should be improved with more clear descriptions of the proposed methods. For example, Line 109 introduces a forward variable which is not mentioned anywhere else. More detailed introduction on CTC may be needed as a comparison so that readers would know the differences between CTC and SCTC formally if any. In addition, I did not get the role of the bipartite matching in this paper which seemingly neither shows better results nor help the derivation of n-gram matching for CTC.

questions: See the above weaknesses.
limitations: NAN
review_title: null

Meta-review: metareview: This paper proposes relaxing CTC-based models, which train the model with the objectives that allow non-monotonic alignment between the output and the reference. Although there is some concern about the empirical results. Most reviewers think the motivation is clearly given and the paper is well-written.
confidence: Certain
Decision: Accept