Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors propose a new Contrastive Neural Text generation framework CONT, which improves contrastive learning in neural text generation from three aspects: the construction of contrastive examples, the choice of the contrastive loss, and the strategy in decoding. Experimental results on five generation tasks with ten benchmarks show the effectiveness of the proposed framework.
strengths_and_weaknesses: Strengths
- The authors propose a simple and effective framework to improve performance of text generation. The framework addresses bottlenecks that prevent contrastive learning from three aspects.
- The authors conduct extensive experiments. Experimental results show that CONT outperforms previous contrastive learning-based methods on ten benchmarks.

Weaknesses
- The use of self-generated texts as negative samples has been used in many works that use reinforcement learning to mitigate exposure bias problem, so the innovation of this strategy is limited. For example,
Self-Critical Sequence Training for Image Captioning,
Discourse-Aware Neural Rewards for Coherent Text Generation.
- The use of additional similarity constraints in decoding is also used by many works.
Considering the above two points, I think the innovation of the whole method is limited.

questions: N/A
limitations: N/A
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes to improve text generation with a sequence-level contrastive loss. The loss take advantage of self-generated negatives and consider pairwise ranking relations. The paper also proposes to use the ranking score to rerank the results during inference. Experiments on five generation tasks with ten benchmarks show its effectivenss.
strengths_and_weaknesses: Strengths:
1. The method is well-motivated. (using self-negatives alleviates the exposure bias problem; pairwise ranking relations address the problem in previous contrastive learning approaches: ignoring the difference between negative samples)
2. The experiments are elaborate. (five generation tasks and detailed ablation and analysis)
3. The results seem good.

Weaknesses:
1. The improvement over Naive CL seems incremental <= as many ideas (self-generated negatives & pairwise loss) are well-known, though in different areas other than text generation.
questions: I am most curious about the computation of the sequence-level representations (i.e., z_x and z_y). This paper spends little efforts in explaining it. Essentially, the contrastive loss is at the sequence-level and the ordinary generative loss is at the token-level. and the key is how to effectively combine them. I am wondering if you test any variants of the computation of sequence-level representations (e.g., different pooling methods or more sophisticated designs).
limitations: n/a
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper presents a new contrastive neural text generation framework, CoNT. There are three major technical improvement and contribution for this framework. First, they use model’s beam search result as additional contrastive examples. Second, they construct an N-pairs contrastive loss, compared to traditional pairwise loss. Third, they manipulate the hypothesis score by adding the learned similarity function. The experiment on five downstream tasks shows that the proposed methods are effective in general. 

strengths_and_weaknesses: The approaches proposed in this paper are overall intuitive and effective. The use of model’s prediction is a smart way to collect more high quality negative examples. The n-pairs cross constrictive loss captures the relative difference between ground truth and the constructive examples. The approach is pretty general and could be applied to many tasks. 

The experiment is comprehensive. It includes experimental results and analysis on 5 downstream tasks and 10 different tasks. Although some of the potential comparison is not provided, the results generally look good to me. The extensive comparison will be beneficial to the community. 

The paper is well written and presented. I don’t have any major difficulty understanding the paper. 


Weakness

The results aren’t 100% solid to me. According the Liu and Liu [24] cited by this paper, on XSUM, one of the summarization dataset, BART/PEGASUS achieve 45.14 and 47.21 on ROUGE-1. The contrastive learning method proposed by Liu and Liu [24], SimCLS, achieves 47.61 on R-1. Given the paper cites [24], are there any reason 1) the “earlier SOTA” was much lower than baseline methods in previous work (SimCLS is 47.61)? 2) the SimCLS is not directly listed and compared?
I have experience working on text summarization, so It’s easier for me to verify the results in summarization. I am a little worried since the author shows many crappy baselines “T5-xxx”, which seems to be misleading. 


There are some mathematical issue about some of the equations. In Eq 4, the likelihood is defined as the product of the decoding sequence. A product of many probabilities will be a very small number, right? In that sense, the second term in the equation will very small compared to the first term ranging from -1 to 1. The equation is just wrong to me. Also, I don’t see the reason using alpha and 1-alpha to balance these two terms because they aren’t in the same range or scale. In Figure 2, the prob of two examples are -.48 and 0.53, which don’t seem to be the product of a sequence of probabilities. 
In this sense, the result of Figure 5 seems spurious to me. 

questions: The diverse beam search algorithm might not be actually diverse given my experience. Have you considered using some other methods, like sampling methods? Top-p sampling, top-k sampling, something like these. 

The arrows between z_y, z_b, … in Figure 2 are a little confusing. The use of zy’ and zy’’ is also a little confusing. Please add some notation and explanation for these arrows and symbols. Also, some lines are used to connect z_y and z_x while some are for z_y and z_y’’. The function of lines and arrows needs to clearly stated. 

limitations: n/a
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper presents a new algorithm for training neural text generation models. The standard approach to training text generators uses the MLE objective, which is known to suffer from the "exposure bias" problem --- a mismatch between training and testing. This paper proposes to tackle this issue using a contrastive learning objective.

The authors perform contrastive learning on the encoder / decoder representations of the model, pushing the input vectors close to the ground truth and away from negatives. Three key ideas are used to implement this --- (1) model generations are used to create harder negatives than simply in-batch negatives; (2) negatives are broken down into (positive, negative) pairs according to an oracle scoring function (like BLEU); (3) during decoding an additional re-ranking step is used which minimizes the cosine similarity of the learned contrastive representations.

The authors evaluate their approach on 5 conditional text generation tasks (over 10 datasets), and report 0.5-2 point improvements across tasks over competing baselines using automatic token-overlap metrics.
strengths_and_weaknesses: **Strengths**

1. The authors present a novel method of incorporating contrastive learning into training text generation models. The method seems like a neat way of tackling the exposure bias problem in text generation since it's a sequence level objective (rather than MLE).

2. The authors have a comprehensive experimental setup covering 10 datasets across five practically relevant text generation tasks (like summarization, translation). The authors consistently report small improvements in automatic metrics over competing baselines across all tasks, despite the different tasks having very different input/output spaces.

3. Experimental results are complemented with good ablation studies justifying each design choice, and model generations in the appendix.

4. The authors have a good limitations section acknowleding the slower training time of the approach along with strategies to reduce its effect.

**Weaknesses**

I have two major concerns with the paper:

1. *No human evaluation*: While experiments were conducted on 10 benchmarks, no human evaluation was done --- all improvements are solely on automatic evaluation metrics. Most of the metrics used are token overlap metrics well known to be brittle, see [1, 2, 3, 4, 13, 14] for example. I think using human evaluation (for a subset of tasks at least) is particularly important in this paper due to the oracle scoring function used during training. What oracle function was used in different tasks? Is it the same metric that is used for evaluation of the models? My worry is that the oracle function will encourage over-fitting to the properties of the flawed metric (like BLEU / ROUGE) rather than actually improve generated text. Also, why not use BLEURT for translation evaluation [4], or some of the newer summarization metrics [5] with higher correlation that ROUGE? In any case, I think human evaluation is a gold standard which needs to be done in text generation work to validate the methods.

2. *Scaling the idea*: The authors primarily try their idea on small transformer models (60M - 220M parameters), with PEGASUS-large (560M) for XSUM being an exception. While it's great that contrastive learning on smaller models matches the performance on bigger models, I'm curious to know whether the proposed approach scales to larger transformer models like T5-large, T5-3B and T5-11B (I think T5-large / 3B can be done on NVIDIA A100 GPUs with Adafactor?). This is important for a few reasons --- (1) the proposed approach has a 2-4x slowdown in training (as the authors discuss in their limitations section), so it may be more efficient to simply scale the model size with MLE (and get similar performance); (2) scaling could go either way in terms of performance improvements, and this is important factor determining the generality of the approach (informal tweet on this in [7]) (3) some of the results (like IWSLT De-En) are below SoTA [6], and the CONT approach may be able to cover up the gap using larger Transformers.

**Minor**

1. It will be good to cite and discuss the relations to [8, 9] in the related work since they use contrastive learning for text generation as well. These two papers have come out in the last 2-3 months (contemporary to this paper), so no direct comparison is necessary. A few older papers with related ideas you may want to cite as well --- [10, 11, 12].

**Overall**: This paper has good clarity and moderate originality, quality and significance. Overall, I liked the ideas in the work and the comprehensive evaluation setup across 10 tasks, but the lack of human evaluation + small scale of the most models tested (60M - 220M parameters) makes me slightly lean reject. The missing human evaluation is a bigger concern for me. I will be happy to move to the accept range with human evaluation on at least 2 of the 10 datasets, or a strong rebuttal discussing why the human evaluation / model scaling is not needed.

---

**After Rebuttal** - The authors presented a preliminary human evaluation with promising results. I am raising my score to 5, but I recommend the authors to conduct a more thorough human evaluation in the next version of the paper.

[1] - https://arxiv.org/pdf/2202.06935.pdf  
[2] - https://arxiv.org/abs/1604.00400  
[3] - https://arxiv.org/abs/2103.06332  
[4] - https://aclanthology.org/2020.acl-main.704/  
[5] - https://arxiv.org/abs/2111.09525  
[6] - https://paperswithcode.com/sota/machine-translation-on-iwslt2014-german  
[7] - https://mobile.twitter.com/_jasonwei/status/1526589104758042624  
[8] - https://arxiv.org/abs/2205.09726  
[9] - https://arxiv.org/abs/2202.06417  
[10] - https://arxiv.org/abs/2004.11714  
[11] - https://aclanthology.org/D16-1137/  
[12] - https://arxiv.org/abs/1804.06451  
[13] - https://arxiv.org/pdf/2107.10821.pdf  
[14] - https://arxiv.org/abs/2006.06264  
questions: What are the GPU memory requirements for the approach?

Does performance improve with more negative samples?

how important is beam size during inference? I'm guessing larger beam-sizes are important to have more hypotheses to re-rank with the vector cosine similarity.
limitations: Yes, good limitations section!
review_title: null

Meta-review: metareview: This paper received mostly positive ratings, but the reviewers also had some overall reservations about the novelty of this work and that makes this paper relatively borderline. 

Pros:
- The paper makes three targeted contributions to contrastive learning in text generation to help mitigate exposure bias problem, and the approach of the paper outperforms SoTA contrastive learning methods. 
- The work is supported by extensive experimentation, and on a wide variety of tasks (MT, summarization, code comment generation, data-to-text generation and commonsense generation). 
- Experimentation appears to be solid for the most part, but reviewers expressed some minor concerns (e.g., about the summarization evaluation)

Cons:
- The reviewers’ main concern is with the relatively limited novelty of the work, as many of its ideas (e.g., self-generated negatives and pairwise loss) are well-known. That said, the application of these methods to text generation appears to be novel.
- The work contains no human evaluation, but automated evaluation is defensible for some of the tasks (MT and summarization at least).

In sum, the work is quite solid and suffers from no major flaws, but the ideas underlying the methods of the paper are not particularly surprising.
confidence: Less certain
Decision: Accept