CoNT: Contrastive Neural Text Generation
Authors: Chenxin An
Jiangtao Feng
Kai Lv
Lingpeng Kong
Xipeng Qiu
Xuanjing Huang
Venue: NeurIPS 2022 Accept (2022)

Abstract: Recently, contrastive learning attracts increasing interests in neural text generation as a new solution to alleviate the exposure bias problem.  It introduces a sequence-level training signal which is crucial to generation tasks that always rely on auto-regressive decoding. However, previous methods using contrastive learning in neural text generation usually lead to inferior performance. In this paper, we analyse the underlying reasons and propose a new Contrastive Neural Text generation framework, CoNT.  CoNT addresses bottlenecks that prevent contrastive learning from being widely adopted in generation tasks from three aspects -- the construction of contrastive examples, the choice of the contrastive loss, and the strategy in decoding. We validate CoNT on five generation tasks with ten benchmarks, including machine translation, summarization, code comment generation, data-to-text generation and commonsense generation.  Experimental results show that CoNT clearly outperforms its baseline on all the ten benchmarks with a convincing margin.  Especially, CoNT surpasses previous the most competitive contrastive learning method for text generation, by 1.50 BLEU on machine translation and 1.77 ROUGE-1 on summarization, respectively. It achieves new state-of-the-art on summarization, code comment generation (without external data) and data-to-text generation.

CoNT: Contrastive Neural Text Generation
Chenxin
1: !n! Jiangtao Feng", Kai Lv', Lingpeng Kong?., Xipeng Qiu', Xuanjing Huang
Fudan University, 2Shark-NLP Shanghai AI Laboratory
Thhe University Hong Kong
Shanghai Collaborative Innovation Center of Intelligent Visual Computing
{cxan20, k1v21, xpqiu, x #jhuanng)@fude eendan.edu.cco edu. cn
fengjiangtao@pj lab. org. cn, lpkêcs hku. hk
Abstract
Recently, contrastive learning attracts increasing interests in neural text generation solution to alleviate the exposure bias problem. It introduces sequence- level training signal which is crucial to generation tasks that always rely on auto- regressive decoding. However, previous methods using contrastive learning in neural text generation usually lead to inferior performance. In this paper, we analyse the underlying reasons and propose new Contrastive Neural Text generation framework, CONT. CoNT addresses bottlenecks that prevent contrastive learning from being widely adopted in generation tasks from three aspects the construction of contrastive examples, the choice of the contrastive loss, and the strategy in decoding. We validate CoNT on five generation tasks with ten benchmarks, including machine translation, summarization, code comment generation, data- to-text generation and commonsense generation. Experimental results show that CoNT clearly outperforms the conventional training framework on all the ten benchmarks with a convincing margin. Especially, CONT surpasses previous the most competitive contrastive learning method for text generation, by 1.50 BLEU on machine translation and 1.77 ROUGE-1 on summarization, respectively. achieves new kate-of-theear summarization, code comment generation (without external data) and data-to-text generation.
Introduction
also attracts enormous interests in neural text generation recently. By creating positive and negative examples in response to unseen (or erroneous) inputs
Contrastive learning has achieved great success in representation learning 6 44
[ contrastive learning offers new solution alleviate autoregressive model trained only using the ground truths exhibits inferior generalization performance. Apart from that, contrastive learning also introduces sequence-level loss in addition to the conventional token-level language model loss with maximum likelihood estimation (MLE). This is crucial to most conditional text generation tasks (c.g., machine translation and summarization) which are evaluated on sequence-level metrics (e.g.. BLEU
the exposure bias problem
6 35i
32
However, is non-trivial get contrastive learning working on neural text generation. we simply use from- batch Dositive-negative samples following simCLR [6]. and adopt the InfoNCE loss 1
which ignores the difference between negative samples 222 Naive CL), the improvement over non-contrastive baselines on generation tasks is rather marginal. Previous work attempts to build
better contrastive samples by disturbing the ground truth I0. 23 BG in the discrete space or the
continuous embedding space, but when it comes to text generation tasks, their performance gains are still far from satisfactory.
This work done during Chenxin An's internship at Shanghai Al Laboratory The code is available at betts://ggithe com Som/Shark-aa Sahark-NNa. NLP/CoNTT
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Ground truth
Ground truth
<bos> leave that to
<bos> up
<bos> leave that quiet to
honn sover senpyles
Encoder-Decoder
<bos> um my brand
<bos> leave that quiet down
<bos> this is frightening
<bos> let's take that quietly
<bos> they sell certificates
überlasson sie das ruhig uns
from-batch samples
Decoder input of naive method
Encoder input
Decoder input of method
Figure 1: case study from IWSLT' De-En translation task. The naive setting uses from-batch samples following SimCLR [d) Compared with the naive method, CONT both incorporates self- generated samples and from-bateh batch samples. The border color indicates the acutual distance between the ground truth and the contrastive example.
In this work, we propose a new contrastive neural text generation framework, CONT. CoNT does three different things from previous frameworks that make suboptimal use of contrastive learning. First, CoNT samples contrastive examples from its own predictions (e.g., through beam search algorithm). This training procedure exposes the model to its mistakes in the inference stage and effectively alleviate the exposure bias problem. We show a comparison between negative samples in CONT and in Naive CL. in Figure [ Second, we use N-pairs contrastive loss which gives fine- gra-grai treatment to the contrastive examples based on their sequence-lever scores (e.g.. BLEU).. It allows the model to fully leverage the supervision from the ground truth example (and its own generated examples) to learn a better sequence-level distance function between the source and the target representation. Third, we directly incorporate the learned sequence similarity score from the distance function into the inference stage. This helps the model to find better global configuration, than merely follows the language model likelihood objective decoding.
We validate CONT on various important conditional language generation tasks machine translation, summarization, code comment generation, data-to-text generation, and common- sense generation. Extensive experiments demonstrate that CoNT greatly improve the conventional MLE baselines and significantly outperforms all previous contrastive generation models. CoNT establishes new state-of-the-art results on summarization, code comment generation (without external data), and data-to-text generation. Particularly, data-to-text generation and commonsense gen- eration, CoNT achieves on-par performance with the powerful large pre-trained models: T5-large, T5-3B 36
including
with only the base version of T5 while maintaining the efficiency of lightweight models.
2 Background
2.1 Neural Conditional Text Generation
A neural sequence-to-sequenee model 4
M (f.g) generates the target sequence conditioning
on a source sequence, where f and g denote the encoder and decoder, respectively. It is typically trained using the language model objective with the maximum likelihood estimation (MLE). Given a source sequence x { and its target sequence y = [ we minimize the following negative log likelihood (NLL) loss:
N
CNLL =
log (y x. yee)
(I)
1=1
At training stage, it predict the next word based on previous ground truth input yer y. but inference stage, tokens of y are predicted by itself, this introduces the exposure bias.
Naive Contrastive Learning for Text Generation
Contrastive text generation introduces contrastive term in addition to the original NLL loss. In Naive CL., we simply follows SimCLR i and use from-batch negative samples the InfoNCE
loss B 45

---

from-batch samples
Decoder
Zy
Za
Training obj
Decoding abj
Sin-0.37
Zy'
<bos> that us
Z
pair-wise
contrastive loss
sim=0.23
Encoder
<bos> leave that quiet to US, =0.48
Überlassen sie das ruhig uns
<bos> let's take that quietly, P =0.53 score= 0.53 + 0.23
search
An overview of CONT. Zoz By is the representation of source sequence and target sequence y- y' and y" with their representations zyy, Zy" are returned by beam search algorithm. The feature representations come from pooling the output of the encoder (source sequence) or decoder (target sequence). Our training objective is obtained by comparing by all contrastive samples in pair. The decoding objective not only considers the likelihood cach sequence, but also the sequence similarity score modeled in training.
Figure 2:
CNCE - log exp(cos(Zz: zy)/T) En'er ex exp(cos(zza Box Zny )/T)
-log
(2)
where Ez, Zy, Zny' E Rd denote the vector representation of input æ. ground truth y and negative sample y E B, respectively. T is the temperature and cos(:,) defines the cosine similarity, Intuitively, the contrastive loss LNCE seeks to learn a similarity function that drives the distance between the source sequence representation Zy and its ground-truth target sequence representation Zy closer.
3 Method
In this section, we present new contrastive neural text generation framework, CONT. CONT ad- vances the Naive CL ($2.2) in three aspects. First, CONT uses negative examples from its own predictions ($3.T) to construct the set B. Second, CoNT replaces the InfoNCE loss (Eq with N-pairs contrastive loss (Eq which leverages finer-grained supervision given by the the sequence- level scores of all pairs ($3.)) Third, CoNT incorporates the learned similarity function into its
inference score directly 3.3 An overview of our approach can be found in Figure 2
3.1
Contrastive Examples from Predictions
Instead of only using contrastive examples from the same batch 6 propose add contrastive examples from the model's 's own predictions. Kalkstein et al. I8 samples helps the generalization ability of the model. Therefore, we use the diverse beam search algorithm 4Р] to create contrastive examples from the top-K list of the model's 's lastest predictions and then append them the batch samples form the contrastive examples. warm-up stage where the model is only supervised by LNLL is recommended as it guarantees the quality of the examples from the model's 's prediction. These self- celf-generated contrastive examples alleviate the model's exposure bias. Besides, with the model's performance improving gradually, this approach creates high-quality hard negative examples that is known to be important in contrastive learning 16
point that using diverse contrastive
3.2 N-Pairs Contrastive Loss
One major drawback of the InfoNCE loss is that it has the same treatment for all negative samples. In text generation, this means that the relative difference between the ground truth and the contrastive examples is ignored, while this can be easily quantified using sequence level score (e.g. BLEU) and the quality these contrastive examples varies. mitigate this problem, we propose to employ a pair-wise margin loss. We first rank all the contrastive examples based on an oracle function o(., y), which computes a sequence-level score with the ground truth y. Given input sequence æz the ground truth y, and set of K contrastive samples B { y/
.y }. we can create series
3

---

Algorithm 1 Inference algorithm: Given an input sequence æ, a contrastive generation model
M (f,)); return the output sequence.
1: procedure BEAMSFARCH(,. Hæ. b)
beam search algorithm
2: return Text, likelihood, logits the b hypotheses
procedure INFERENCE(G, x))
2:
H < f (z), b < beam size, a balance factor E (0, 1)
3:
y 1:hh ,P H BEAMSEARCH( Hz:
> Get b candidates with beam search
4:
Ze. 2 + Avg(H,, Avg(
21:
>Avg(-) an average pooling function
5:
D + Cosine distance between Za and representation of hypotheses 1:b
6:
P Likelihood of hypotheses returned by beam search
7: == arg m.. (a * D + (1 - a) }
8: return y
of example pairs (y+,y ) P, where + and are determined by their ranks The contrastive learning objective formulated margin loss according to their cosine similarity to the source representation Zez:
LN-Pairs
Σ (1/t.0 )
Cly+, , ()=
(ar Σ max(0.. cos 2y-) ccss(zo) 4y+) + E2-
(3)
We further set { 7 (rank(y)) - rank(y+)) following Zhong et to reflect the quality difference in these pairs, where 7 is hyperparameter controlling the the strength. Full details of the training algorithm can found in Algorithm| Appendix
3.3 Inference with Learned Similarity Function
Previous inference algorithm for contrastive text generation method 23 usually remains the same with con-contrastive approaches. In CoNT, incorporate the similarity function learned in the N-pairs contrastive loss into the decoding stage. Despite such a inference objective can be generalized to other contrastive learning methods sequence pair exist, the design of CoNT can better make use of the learned similarity function ($4.) The decoding objective in CoNT is find the sequence y" that maximizes both the learned similarity score and the conventional language model likelihood:
long the vector representations for source and target
y = arg max cos(Za: Zz) + (1 a) II p(il 9c))),
(4)
where Zz, Zg E Rd is the vector representation of x, 9. and a is the hyperparameter that balances the contribution of each term. In most cases, be directly set 0.5, tuning the validation set will usually get better results. Algorithm Iillustrates the inference stage in CoNT details.
The relationship between different modules of CoNT is summarized in Figure 3
modeling relative difference
contrbute to
self-generated samples
noticing mistakes
global scoring
performance
src-trg similarity
pairwise loss
improving quality
inference
Figure 3: Relationship between different modules in CONT. Both design of pairwise loss function and self- -effeaenedt samples could contribute the source-target similarity function that com- putes the sequence-leve score at inference stage. With the performance improved, self-generated contrastive samples tend to be more indistinguishable.
3p contains Ck pairs constructed from B, ground truth y. and from-bate wott-btchh examples.

---

4 Experiments
We experiment CONT on 5 downstream tasks with 10 different benchmarks. Our contrastive learning framework supports most -eatennce----seennce models at multiple scales. Concretely, we experiment CONT on 4 kinds of base models: (a) Transformer-same (60M) 47 and transformer-base (220M), (b) TS-small (60M) and TS-base (220M) 16. (c) Code (220M) [5I] (d) PEGASUS-large (560M) 155. Details of our experimental setup for each benchmarks can be found in Appendix C
On WMT 16 Ro-En (machine translation) and XSum (summarization) which are also used in previous contrastive text generation frameworks [23]. results show that CONT is able to substantially improve the MLE baseline and clearly outperform all previous contrastive baselines by a large margin. We also build CoNT on state-of-the-art (SOTA) baselines: PEGASUS-large (summarization), Code'T5- base (code comment generation) and achieve new SOTA. Moreover, on data-to-text generation and commonsense generation, CoNT also shows its superior performance strong MLE baselines.
4.1 Baselines
1. Naïve CL 3
Naive CL denotes the naive contrastive learning approach that treats the ground truth as the positive sample and the target sequences from the same mini-batch as the negative examples. The training object of naive CL takes the form of Eq. - We also implement Naive CL with N-Pairs contrastive loss, it can be viewed as an ablation study when setting beam size of CoNT 0 during training.
SSMBA CL [30]: Compared with naive CL. SSMBA builds more positive samples via disturbing the ground truth in the discrete space. Concretely, SSMBA first randomly masks 25% tokens in the target sequence and then reconstructs the ground truth with a masked language model BERT.
3. Dropout CL [10] Dropout CL enhances the positive samples by using dropout mechanism on the target sequence. We use the default dropout rate of standard transformer decoder 47 and input the ground truth to the decoder twice.
4. CLAPS 23 CLAPS is previous the best contrastive learning framework for conditional text generation task. order to provide more challenging contrastive examples, CLAPS propose to simultaneously create extra positive and negative pairs by adding perturbations to the ground truth sequence in the continuous embedding space.
5. CoNT (this work): CONT is the contrastive neural text generation framework proposed in this work. We implement its InfoNCE version by treating ground truth as positive sample and self-generater samples are also treated negative samples.
4.2 Quantitative Results
Machine Translation For machine translation, we evaluate CONT WMT 2016 Romanian-to- English translation task (WMT' 16 Ro-En), WMT 2014 English-to-German translation task (WMT' 14 En-De) and IWSLT 2014 German-to-Ennlissh translation task (IWSLT' 14 De-En). We use BLEU as the evaluation metric. Results in Table (rows with gray background) indicates our model CoNT significantly improves the traditional maximum likelihood estimation training and inference framework. On WMT' Ro-En, CONT outperforms previous the best contrastive learning approach CLAPS by 1.50 BLEU and exceeds the MLE baseline by 2.70 BLEU with the same base model T5-small. We also compare the infoNCE loss used in previous methods with the N-Pairs margin loss described in Eq. 3 Results show that the N-pairs contrasting samples generally works better than dividing all samples into predefined positive-negative categories. Similar to CLAPS and Naive CL, only incorporating contrastive learning into training improves the performance TS-small baseline on WMT' 6 Ro-En to 30.55 (+2.34) BLEU. If we further add learned target-source similarity as decoding target as Eq. 4 the result is further boosted to 30.91 BLEU, We observe that the benefits of introducing sequence similarity into inference is more obvious IWSLT' 14 De-En the additional decoding target improves the vanilla beam search algorithm up to 0.86 BLEU.
Summarization For summarization, we use the XSum 2 dataset collected from BBC News whose reference summaries are provided by human writers. We also evaluate CONT on a multi- document summarzation dataset multi-news [9] consisting of news articles from the site newser.com. Compared with the common summarization task, multi-document more challenging where the model need to automatically summarize several articles and usually has to handle long input sequence
5

---

Table 1: BLEU on WMT" 16 Ro-En, IWSLT De-En and WMT En-De translation tasks. For IWSLT 14 De-En and WMT 14 En-De, we use Transformer-smal (Tr-small) and Transformer-bes (Tr-base) as baselines. For WMT" 16 Ro-En, we add a pre-trained baseline T3-small. w/o seq sim means we use the origin beam search without target-source representation similarity. The best results in each block are underlined and the best results are bold. Rows gray denotes the contrastive learning based model strongly outperforms its MLE version. Results with are token from
23
16 Ro-En
IWSLT'14 De-En
WMT'14 En-De
Model
Tr-small T5-small
Tr-small
Tr-base
MLE
25.78
28.21
34.18
27.30
Contrastive loss: InfoNCE loss
Naive CL
25.49
27.79
34,45
27.28
SSMBA CL
25.98
28.48
34.32
27.16
Dropout CL
26.01
29.10
34.41
27.34
CLAPS CONT
23.59
29.41
25.74
29.64
34.46
27.35
Contrastive loss: N-Pairs loss
Naive CL
26.15
29.86
34.47
27.41
w/o seq sim
26.27
29.74
34.26
27.45
CONT
27.70
30.91
35.55
28.04
w/o seq sim
27.42
30.54
34.69
27.77
Table ROUGE score on Summarization datasets. Results with are token from and results with are from [55]. Current state-of-the-ar models and the best results are in bold. Previous SOTA means the best results before CoNT.
XSum
Multi-News
Model
R-1
R-2
R-L
R-1
R-2
R-L
T5-small
36.10
14.72
29.16
42.36
15.34
21.91
T5-SSMBA CL
36.58
14.81
29.68
42.06
14.98
21.73
T5-Dropou CL
36.82
14.93
29.26
42.43
15.32
21.95
T5-CLAPS
37.89
15.78
30.59
NaiNaive CL
36.34
14.81
29.41
42.20
15.18
21.78
T5-Naive CL (N-Pairs)
37.76
15.48
30.15
43.04
15.83
22.03
T5-CONT
39.66
16.96
31.86
44.08
16.39
22.58
Previous SOTA"
47.61
24.57
39.44
47.52
18.72
24.91
PEGASUS (base)"
39.79
16.58
31.70
42.24
13.27
21.44
PEGASUS (large)"
47.21
24.56
39.25
47.52
18.72
24.91
PEGA-CONT
47.76
24.69
39.46
48.68
19.29
24.58
and target sequence. Experimental results are in Table 2 The first block includes the performance of different contrastive frameworks with T5-small. On XSum, it shows that our proposed model strongly outperform previous contrastive frameworks by about 2.0 ROUGE-I score. We also illustrate our method is not restricted to the small model. By employing CoNT state-of-the-arr base model PEGASUS, it is able to establish new state-of-the-e on the two summarization benchmarks.
Code Comment Generation Code comment generation aims to generate an English description for function-lever code snippet. We test our method on two widely used datasets Java and Python from the CodeXGLUE benchmark 227 Results are shown Table 3 Our model built upon state- of-the-art pre-trained model program language model CodeTT-basee
CodeT5-DDall-eer means they further involve a comment-to-code task which the best model on Python and Java without using externl data. We also include the results of earlier strong pre-trained basclines: PLBART

---

Table 3: BLEU on code comment generation datasets Java and Python. Results with and from 61
Table 4: BLEU data-to-text generation dataset WikiBio. We run our model three times and report the mean and variance of the BLEU metric. Results with i are token from [2 and results with from
Model
Python
Java
CodeBERT
19.06
17.65
Model
BLEU
PLBART
19.30
18.45
CodeT5
20.01
20.31
Table NLM
34.70 #0.36
Code15-DuallGeen
20.11
20.41
vanilla Seq2Seq
42.06 +0.32
StructureAware
44.89
With N-Pairs CL
R2D2
46.23
TS-aivee CL.
20.26
20.31
T5-small
40.36
CodeTS-CoNT
20.43
20.56
With N-Pairs CL
With External Training Data
TS-small-Naive CL
46.50+0.24
CodeTS-Mulli-Taase 20.36
20.46
T5-small-CoNT
47.17=0.19
REDCODER*
21.01
22.94
Table 5: Results on the dev set and test set Totto. PAR short for PARENT score. Dev Set (Non)
means the non-overlap subset of the dev set. results with are reported in [7
Dev Set (AII)
Dev Set (Non)
Test Set (All)
Model
BLEU
PAR
BLEU
PAR
BLEU
PAR BLEURT
BERT-to -ERTT
44.0
52.6
34.8
46.7
44.0
52.6
0.121
Targer
48.1
57.3
39.8
52.8
T5-3B²
48.4
57.8
40.4
53.3
49.5
58.4
0.230
T5-base
47.7
57.1
39.6
52.6
35-base-CoNTY
49.2
59.4
41.5
55.0
49.1
58.9
0.238
and CodeBERT, We also report some data augmentation methods in the third block of Table CodeTS-Multi-Tass makes use of training datasets of other program languages. REDCODER [34 uses retrieval to enhance the task with open-source code base and achieve the bset results on this task. Our model is orthogonal to these methods and clearly outperforms all baselines without external data.
Data-to-text Generation Data-to-text generation aims to produce text from non-linguistic input. The first benchmark where the infobox is treated as input sequence, and the target sequence is the first sentence of the biography. Totto s is also collected from Wikipedia whose input is table with its highlighted cells and target sequences are professionally annotated by human. Results on WikiBio is shown in Table the performance of some popular baselines (first block) are token from 225 R2D2 (2] using XLNet [53) llarge as base model, is previous state-of-the-an model on WikiBio. We experiment CoNT on T5-small, and results show that we exceed R2D2 by about 0.94 BLEU. The test set and dev set of Totto are both split into two parts overlap and non-overlap. The non-overlap part contains out-of-domain samples from the training set. The test set Totto is invisible and we report the results dev set and test set by the feedback of the Totto authors. We also add PARENT [ and BLEURT 40) as evaluation metrics. PARENT is a word-overlap based metric that designed to evaluate the factual accuracy of generation results. BLEURT is trained under human supervision and correlates well with human judgement. As we can see from Table 5 by comparing our model with different T5 variants, built on a TS-base model. Compared with previous state-of-the- model T5-3B, our model still show ad advantage in PARENT and BLEURT.
is WikiBio 2 consisting of biography pairs from English Wikipedia
show that CoNT is able to greatly outperform the large version of T5
Commonsense Generation The task of commonsense generation aims to explicitly test the ability of machines on commonsense reasoning. The source sequence consists of set of concepts and the target sequence is fluent sentence mentioning all the input concepts. We evaluate CONT
7

---

Table 6: Results on CommonGen. Results with are reported in (24). The metrics used in the official leaderboard are in bold. Human performance is also reported as an upper bound.
Model
ROUGE-2/1.
BLEU--/4
METFOR CIDEr SPICE Coverage
GPT-2
16.85
39.01
33,92
23.73
26.83
12.19
23.57
79.09
BART
22.02
41.78
39.52
29.01
31.83
13.98
28.00
97.35
T5-largel
21.74
42.75
43.01
31.96
31.12
15.13
28.86
92.29
T5-base
14.63
34.56
28.76
18.54
23.94
9.40
19.87
76.67
T5-base- D5-base-CoNT
20.96
43.15
42.60
31.42
32.05
15.96
28.95
96.55
Human
36.72
54.45
52.55
46.49
38.79
37.64
52.43
99.33
Table 7: BLEURT and BERT'Score on the test set of 4 translation and summarization datasets. The first column of each dataset represents BLURT and the second column is BERTScore.
Model
IWSLT'14 De-En
1M Ro-En
XSum
Multi-News
MLE 0.137
62.28
0.272
69.09
-0.552
44.10
-0.568
17.21
CoNT
0.167
63.38
0.281
69.33
-0.462
46.78
-0.505
17.47
on the CommonGen ! benchmark with a hidden test set and the final results is obtained with the help of the authors of CommonGen. addition to the mostly used metrics, CIDEr [48] and SPICE (1. concerning evaluating semantic faithfulness, are highlighted by the leaderboard of commonGin. In Table we demonstrate that the lightweight T5-base model is able to greatly benefit from our contrastive learning framework. Moreover, CONT not only surprisingly outperforms its MLE baseline but also surpass the large version of T5 in terms of CIDEr and SPICE metrics.
Advanced Evaluation Metries Considering the training efficiency of CONT, we mainly select the lexical matching metrics as oracle measurement function. To verify that the improvement brought by CoNT not due to the over-fitting of lexical matching metrics, we further evaluate generated text with advanced metrics based on neural models: BERTScore (56] and BLEURT [40] For BERTScore,
L17 we use their roberta- laaeellrrre____ L17_no-ide _no-idf_ version and for BLEURT we use the default setting provided on their githut Results are shown in Table 7 The base model used on IWSLT' 14 De-En is transformer small and on the other datasets we select T5 as the base model. For all datasets CONT also make non-trivial improvements in terms of the two neural metrics. Particularly, CONT improve the results of MLE model on IWSLT' 14 De-En by 0.03 BLEURT and improve the results of MLE model on XSum by 2.68 BERTScore.
To get more accurate and convincing results, we also conduct a ranking based human evaluation on two mainstream tasks: machine translation (IWSLT' 14 De- De-En) and text summarization (XSum) with 60 samples for each tasks. Following Cheng and Lapata 77) we hired 2 annotators asking them to rank the given candidate output based on fluency, coherence, and their personal preference ( rank these systems stt, 2nd, and so on) and we calculate the average ranking. For each sample, there are four candidates consist of human-writte
reference, sequence from MLE model, sequence from Naive CL., and a sequence from CONT. Table & shows the results of our human evaluation. Generally CoNT outperform all baseline systems according the average ranking.
4.3 Discussion
Discrimination of Hard Negative Samples To deeply look into the learnt representations, we visualize target sequences, that is trained by MLE, naive CL and CoNT IWSLT" 14 De-En, with the t-SNE algorithm [46]. The visualized sequences consist of three groups of target sequence: a) batch targets that is mostly unrelated to ground -truth target; b) beam search hypothesis that could be of high/low quality; c) ground truth target. As can be seen in Figure 4a the representations trained by MLE are distributed almost uniformly in the vector space, and there one group to another. With naive CI. we find there is clear boundary between batch tokens and
clear boundary between
8

---

Table Results of human evaluation on the test sets of translation and summarization.
Machine translation
Summarization
Model
1st
2nd
3rd
4th
avg rank
1st
2nd
3rd
4th
avg rank
Ground truth
0.88
0.08
0.0
0.04
1.2
0.52
0.25
0.12
0.12
1.86
CONT
0.07
0.5
0.31
0.12
2.48
0.27
0.4
0.2
0.13
2.2
Naive CL
0.02
0.25
0.4
0.33
3.04
0.12
0.22
0.38
0.28
2.82
MLE
0.03
0.17
0.28
0.52
3.29
0.1
0.13
0.3
0.47
3.13
Dishh Togen
Tagm
Bown Sourh
Gocch
-00
MLE
(b) Naive CL
CONT
Figure 4: T-SNE experiments on IWSLT'14 De-En. Each point represents a target sequence. Batch target is blue; beam search hypothesis is orange; ground-truth sequence is green. Darker points indicate sequences with higher BLEU.
others. Naive CL does contribute to discriminating related sequences with unrelated ones, but it still cannot distinguish hypothesis of high quality from the others. Even without contrastive learning, the generation model trained has already pulled From-batch samples away from the ground truth and the naive contrastive learning procedure is only make the margin more obvious. As for CoNT set of hypotheses low quality are excluded from the neighborhoods of the ground-truth targets. The experimental results verifies that CoNT enables better representations in sequence generation.
Sequence Likelihood and Sequence Similarity We also perform ablation study on our ranking objective and self-generated negative samples on IWSLT' 14 De-En with Transformer-small base model. For the ranking objective, we compare our N-pairs contrastive objective with InfoNCE with increasing a. Figure Sa that N-pairs contrastive loss consistently outperform InfoNCE. With [0.3, 0.7], N-pairs contrastive loss can benefit target-source representation similarity. sequence- level score in inference process, while InfoNCE cannot. For the sampling strategy, we compare CONT with naive CL and vanilla MLE. CONT use self-generater hypotheses as negatives while
that
37
Pais-wrise InfoNCE
CONT
36
Naive CL
BLEU score
36
35
35
BLEU
34
31
33
33
32
32
0.0
0.3
0.5
0.7
0.9
1.0
0.0
0.3
0.5
0.7
0.9
1.0
Ox
a
(a) Relationship between a and BLEU with dif- ferent contrastive loss.
(b) Relationship between a and BLEU with dif- ferent training methods.
Figure 5: Ablation study on the balance factor on the test set of IWSLT' 14 De-En where a = 0.0 means selecting output only relying on likelihood and a 1.0 means choosing output with only sequence similarity.

---

naive CL only use samples within the same mini-batch. Figure shows that contrastive learning with self-generated hypotheses more effective than using batch samples. With E [0.3, 0.7] CONT gains about 1.0 BLEU improvements, while naive CL. is only improved with less than 0.5 BLEU.
5 Related Work
aims to learn better representation via contrasting positive and negative samples. also has been widely used in the field of natural language processing II0 14
Contrastive Learning Contrastive learning 66 13 39
21 59]. Jiang et al. [|5]] show that contrastive learning helps learn a robust pre-trained model and Lee et al. 23 first introduce contrastive learning into text generation to mitigate the exposure bias problem. They propose an adversarial method to build more challenging positive-negative samples in additional to the from-batch samples. SimCTG 42 is also contrastive framework on text generation. Our work differs from SimCTG from motivation and method. They introduce contrastive leanrning mainly to encourage diversity which is important in dialogue systems. And they perform token-level contrastive learning while our method focus on sequence-level contrastive examples.
Adopting binary supervision in contrastive loss is originally proposed in FaceNet [39 which helps learn the face recognition of the same person in various positions and angles. Given an anchor face image x, a positive sample x (usually the same person) and negative sample x from other people, the triplet loss makes x become close to x and maximize the distance between x and x The pair-wise contrastive loss has also been widely used in metric learning Chen and Deng [3. Kim et al. [1 Wang et al. [50]. Sohn [I] extend the triplet loss to multi-class and multi-pair. Recent work thinks the margin value between samples should not be fixed. Zhou al. (58] divide the sample set into multiple subsets and assign different margin value to different subsets. Ha and Blanz 12, Zhong
al. 57 suggests dynamically adjust the margin value via determination metric.
Post-generation Re-ranking Models Post-generation re-ranking re-score the multiple output sequences via training another re-ranking module. Noisy Channel Modeling (NCM) 22 [e widely-used re-ranking scheme for neural machine translation. NCM parameterizes the noisy channel probability with
is
seaeencc-to-sequnee model. There also various structures to instantiate the re-ranking module: Gulcehre et al. | select the candidate with a language model, Bhattacharyya et al. [4] leverage energy- based model in NMT and Liu and Liu 261. Salazar et al. 3X re-score candidates with masked language models such as BERT. Despite this paradigm achieves impressive results while having large size of candidate sequences, most of post-generation re-ranking systems trade efficiency and simplicity for accuracy.
6 Conclusion
We introduce new contrastive neural text generation framework called CONT, It models an addi- tional contrastive learning objective to provide a sequence-level supervision for auto-regressive neural text generation models. We explore three shortcomings that limit the development of contrastive learning on text generation tasks. Results on five generation tasks with ten different benchmarks show that CONT not only clearly beats all previous contrastive generation models, but also boosts the performance of ofte--of-the-ar large models to a new level. CoNT practically does not have a negative impact on decoding speed. Nevertheless, CoNT suffers from the training inefficiency prob- lem. In general, the total training time of CONT is about 2-4 times more than that of a MLE based model. detailed discussion and some speed-accurace trade-off tricks can found Appendix B Speeding up the training stage without losing accuracy is the next important step to improve CONT.
Acknowledgement
We would like to thank the anonymous reviewers for their valuable advice. This research was supported in part by the joint research scheme of the National Natural Science Foundation of China (NSFC) and the Research Grants Council (RGC) under grant number N HKU714/21.
10

---

References
[1] Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. Spice: Semantic propositional image caption evaluation. In European conference on computer vision, pages 382-398. Springer. 2016,
[2] Aryan Arbabi, Minggiu Wang, Laurent El Shafey, Nan Du, and Izhak Shafran. R2d2: Relational text decoding with transformers, arXiv preprint arXiv:2105.04645 2021.
[3] Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. Scheduled sampling for sequence prediction with recurrent neural networks. Advances in neural information processing systems, 28, 2015.
[4] Sumanta Bhattacharyya, Amirmohammad Rooshenas, Subhajit Naskar, Simeng Sun, Mohit lyyer, and Andrew McCallum. Energy-based reranking: Improving neural machine translation using energy-hased models. arXiv preprint arXiv:2009. 13267, 2020.
[5] Binghui Chen and Weihong Deng. Energy confused adversarial metric learning for zero-shot image retrieval and clustering, Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 1-34-811,, 2019.
[6] Ting Chen, Simon Komblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597-16077 PMIR,, 2020.
[7] Jianpeng Cheng and Mirella Lapata. Neural summarization by extracting sentences and words. arXiv preprini wrXiv:1603.07255 2016.
[8] Bhuwan Dhingra, Manaal Faruqui, Ankur Parikh, Ming- Wei Chang, Dipanjan Das, and William W Cohen. Handling divergent reference texts when evaluating table-to-text generation. arXiv preprint 100..0090000011
[9] Alexander Richard Fabbri, Irene Li, Tianwei She, Suyi Li, and Dragomir Radev. Multi-nwws: A large-scale multi-document summarization datasct and abstractive hicrarchical model. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1074-1084, 2019.
[10] Tianyu Gao, Xingeheng Yao, and Danqi Chen. Simese: Simple contrastive learning of sentence embeddings. arXiv preprint arXiv:2104.0m 08821, 2021.
[11] Caglar Gulcehre, Orhan Firat, Kelvin Xu, Kyunghyun Cho, and Yoshua Bengio. On integrating language model into neural machine translation. Computer Speech & Language, 45:137-148, 2017.
[12] Mai Lan Ha and Volker Blanz. Deep ranking with adaptive margin triplet loss. arXiv preprint arXiv:2107.06187, 2021.
[13] Kaiming Hc, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729-9738, 2020.
[14] R Devon Hjelm, Alex Fedorov, Samuel Lavoic-Marchildon, Karan Grewal, Philip Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximization. In International Conference on Learning Representations. 2018
[15] Ziyu Jiang, Tianlong Chen, Ting Chen, and Zhangyang Wang. Robust pre-training by adversarial contrastive learning. Advances in Neural Information Processing Systems, 33:16199-166 16210, 2020.
[16] Yannis Kalantidis, Mert Bulent Sariyildiz, Noe Pion, Philippe Weinzaepfel, and Diane Larlus. Hard negative mixing for contrastive learning. Advances in Neural Information Processing Systems, 33:21798-21 21888, 2020.
[17] Mihir Kale and Abhinav Rastogi. Text-to-text pre-training for data-to-text tasks. arXiv preprint arXiv:2005.104 10433, 2020.
[18] David Kalkstein, David A Bosch, and Tali Kleiman. The contrast diversity effect: Increasing the diversity of contrast examples increases generalization from a single item. Journal of Experimental Psychology: Learning, Memory, and Cognition, 46(2):296, 2020.
[19] Sungyeon Kim, Minkyo Seo, Ivan Laptev, Minsu Cho, and Suha Kwak. Deep metric learning beyond binary supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 2288 2297, 2019.
11

---

[20] Lingpeng Kong. Cyprien de Masson d' d'Autume, Lei Yu, Wang Ling, Zihang Dai, and Dani Yogatama.
A mutual information maximization perspective of language representation learning. In 8th Interna- tional Conference on Learning Representations, ICLR 2020, Addis Abuba, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL BtLLittt////poereiiiie new/teeunnieeeeeeeeeeeen htt.:///ww......oooooooooooooaaa
[21] Kalpesh Krishna, Yapei Chang, John Wieting, and Mohit lyyer. Rankgen: Improving text generation with large ranking models. arXiv preprint arXiv:2205.097 097266, 2022.
[22] Rémi Lehret, David Grangier, and Michael Auli. Neural text generation from structured data with application to the biography domain. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1203-1213, 2016.
[23] Seanie Lee, Dong Bok Lee, and Sung Ju Hwang. Contrastive learning with adversarial perturbations for conditional text generation. In International Conference on Learning Representations, 2020.
[24] Bill Yuchen Lin, Wangchunshu Zhou, Ming Shen, Pei Zhou, Chandra Bhagavatula. Yejin Choi, and Xiang Ren. Commongen: A constrained text generation challenge for generative commonsense reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1823-1840, 2020.
[25] Tianyu Liu, Kexiang Wang. Lei Sha, Baobao Chang, and Zhifang Sui. Table-to-ter generation by structure-aware seq2seq learning. In Thirty-Second AAA/ Conference on Artificial Intelligence, 2018.
[26] Yixin Liu and Pengfei Liu. Simcls: A simple framework for contrastive learning of abstractive summariza- tion. arXiv preprint arXiv:2106.0113 2021.
[27] Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, Duyu Tang, et al. Codexglue: A machine learning benchmark dataset for code understanding and generation. arXiv preprini arXiv: 202.::::1020004 04664, 2021.
[28] Shashi Narayan. Shay B Cohen, and Mirella Lapata. Don't give me the details, just the summary! topic- aware convolutional neural networks for extreme summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1 1797-1807, 2018.
[29] Nathan Ng, Kyra Yee, Alexci Baevski, Myle Ott, Michael Auli, and Sergey Edunov. Facebook fair's wmt19 news translation task submission. arXiv preprint arXiv: 1rXiv:1907.06616 2019.
[30] Nathan Ng, Kyunghyun Cho, and Marzych Ghassemi. Ssmba: Self-supervised manifold based data augmentation for improving out-of-domain robustness. Empirical Methods in Naturul Language Prucessing. EMNLP, 2020.
[31] Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng. David Grangier, and Michael Auli. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstra- tions), pages 48-53, Minneapolis, Minncsota, June 2019. Association for Computational Linguistics. doi: 10. 10.18653/v1 IN 19-4009. URL
orw.loooyoooooggooeeee www//o/g////////// 4009
[32] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine tramslation. In Proceddings of the 40th annual meeting of the Association for Computational Linguistics, pages 31 1-318, 2002.
[33] Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui. Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. Totto: A controlled table-to-text text generation datasct. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1173 1 1186, 2020.
[34] Md Rizwan Parvez, Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Wei Chang. Retrieval augmented code generation and summarization. arXiv preprint arXiv:2108.1 1601, 2021.
[35] Romain Paulus, Caiming Xiong, and Richard Socher. A deep reinforced model for abstractive summariza- tion. arXiv preprint arXiv: htt::/70. 003304 2017.
[36] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with unified text-to-text transformer. arXiv preprint arXiv:1910. 10683, 2019.
[37] Joshua Robinson. Ching-Yar Yao Chuang. Suvrit Sra, and Stefanie Jegelka. Contrastive learning with hard negative samples. arXiv preprint urXiv:2010.04592, 2020.
[38] Julian Salazar, Davis Liang, Toan Q Nguyen, and Katrin Kirchhoff. Masked language model scoring. arXiv preprint 191.v.1910.1469, 2019..
12

---

[39] Florian Schroff. Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference computer vision and puttern recognition, pages 815-823, 2015.
[40] Thibault Sellam, Dipanjan Das, and Ankur P Parikh. Bleurt: Learning robust metrics for text generation. arXiv preprint arXiv:2004.04690 2020.
[41] Kihyuk Sohn. Improved deep metric learning with multi-class n-pair loss objective. Advances in neural information processing systems, 29, 2016.
[42] Yixuan Su, Tian Lam, Yan Wang, Dani Yogatama, Lingpeng Kong, and Nigel Collier. contrastive framework for neural text generation. arXiv preprint arXiv:2202.064/1 2022.
[43] Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. Sequence to sequence learning with neural networks, 2014. URL https://arxiv. org/ ats//49s/1 1109/1199...0. 3215
[44] Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information proessing systems, 30, 2017.
[45] Aaron Van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, arXiv e-prints, pages
2018.
[46] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008
[47] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances neural information processing systems, 30, 2017.
[48] Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus- image description evaluation. In Proceedingy of the IEEE conference on computer vision and pattern recognition, pages 4566-4575,
[49] Ashwin K Vijayakumar, Michael Cogswell, Ramprasath R Selvaraju. Qing Sun, Stefan Lee, David Crandall, and Dhruv Batra. Diverse beam search: Decoding diverse solutions from neural sequence models. arXiv preprint 1rXiv:1610.0244 2016.
[50] Jiang Wang. Yang Song. Thomas Leung, Chuck Rosenberg, Jingbin Wang, James Philbin, Bo Chen, and Ying Wu. Learning fine-grained image similarity with deep ranking. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3386-1393, 2014.
[51] Yue Wang, Weishi Wang, Shafiq Joty, and Steven CH Hoi. Codet5: awwae unified pre-trained encoder-dccoder models for code understanding and generation. In Proceedings of the 2021 Conference Empirical Methods in Natural Language Processing, pages 8696-8708, 2021.
[52] Thomas Wolf. Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, al. Huggingface's transformers: State-of-the-as art natural language processing, arXiv preprint arXiv:1910.0377. 2019.
[53] Zhilin Yang, Zihang Dai, Yiming Yang. Jaime Carbonell, Russ R Salakhutdinov, and Quoe V Le. Xlnet: Generalized autoregressive pretraining for language understanding. Advances neural information processing systems, 32, 2019.
[54] Kyra Yee, Yann Dauphin, and Michael Auli. Simple and effective noisy channel modeling for neural machine translation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing JEMNLP-IJCN pages 5696-5701, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v1/119-15777 URL htLLitttss//ccaaanolloog wwwp:/9-g1///9110
[55] Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter Liu. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. In International Conference on Machine Learning, pages 11328 11339. PMLR, 2020.
[56] Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi, Bertscore: Evaluating text generation with bert. arXiv preprint 1904 2019.
[57] Ming Zhong, Pengfei Liu, Yiran Chen, Danqing Wang, Xipeng Qiu, and Xuanjing Huang. Extractive summarization as text matching, arXiv preprint arXiv:2004.0895, 2020.
13

---

[58] Mo Zhou, Zhenxing Niu, Le Wang. Zhanning Gao. Qilin Zhang, and Gang Hua. Ladder loss for coherent visual-semantic embedding. In Proceedings the AAAI Conference Artificial Intelligence, volume 34, pages 13050-13057, 2020.
[59] Yunhua Zhou, Peiju Liu, and Xipeng Qiu. KNN-contrastive learning for out-of-domain intent classification. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volame Long Pupers), pages 5129-5141, Dublin, Ireland, May 2022. Association for Computational Linguistics. URL httpt:ttp////. aclacclnnnn http://acccaantooooo.orroo
doi: 10.18063//11
acl long..n...... 352
14


Reviews:
[{"review_id": "Oyh3ol10BGx", "reviewer": "Reviewer_Mnn7", "summary": "This paper presents a new algorithm for training neural text generation models. The standard approach to training text generators uses the MLE objective, which is known to suffer from the \"exposure bias\" problem --- a mismatch between training and testing. This paper proposes to tackle this issue using a contrastive learning objective.\n\nThe authors perform contrastive learning on the encoder / decoder representations of the model, pushing the input vectors close to the ground truth and away from negatives. Three key ideas are used to implement this --- (1) model generations are used to create harder negatives than simply in-batch negatives; (2) negatives are broken down into (positive, negative) pairs according to an oracle scoring function (like BLEU); (3) during decoding an additional re-ranking step is used which minimizes the cosine similarity of the learned contrastive representations.\n\nThe authors evaluate their approach on 5 conditional text generation tasks (over 10 datasets), and report 0.5-2 point improvements across tasks over competing baselines using automatic token-overlap metrics.", "questions": "What are the GPU memory requirements for the approach?\n\nDoes performance improve with more negative samples?\n\nhow important is beam size during inference? I'm guessing larger beam-sizes are important to have more hypotheses to re-rank with the vector cosine similarity.", "limitations": "Yes, good limitations section!", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n1. The authors present a novel method of incorporating contrastive learning into training text generation models. The method seems like a neat way of tackling the exposure bias problem in text generation since it's a sequence level objective (rather than MLE).\n\n2. The authors have a comprehensive experimental setup covering 10 datasets across five practically relevant text generation tasks (like summarization, translation). The authors consistently report small improvements in automatic metrics over competing baselines across all tasks, despite the different tasks having very different input/output spaces.\n\n3. Experimental results are complemented with good ablation studies justifying each design choice, and model generations in the appendix.\n\n4. The authors have a good limitations section acknowleding the slower training time of the approach along with strategies to reduce its effect.\n\n**Weaknesses**\n\nI have two major concerns with the paper:\n\n1. *No human evaluation*: While experiments were conducted on 10 benchmarks, no human evaluation was done --- all improvements are solely on automatic evaluation metrics. Most of the metrics used are token overlap metrics well known to be brittle, see [1, 2, 3, 4, 13, 14] for example. I think using human evaluation (for a subset of tasks at least) is particularly important in this paper due to the oracle scoring function used during training. What oracle function was used in different tasks? Is it the same metric that is used for evaluation of the models? My worry is that the oracle function will encourage over-fitting to the properties of the flawed metric (like BLEU / ROUGE) rather than actually improve generated text. Also, why not use BLEURT for translation evaluation [4], or some of the newer summarization metrics [5] with higher correlation that ROUGE? In any case, I think human evaluation is a gold standard which needs to be done in text generation work to validate the methods.\n\n2. *Scaling the idea*: The authors primarily try their idea on small transformer models (60M - 220M parameters), with PEGASUS-large (560M) for XSUM being an exception. While it's great that contrastive learning on smaller models matches the performance on bigger models, I'm curious to know whether the proposed approach scales to larger transformer models like T5-large, T5-3B and T5-11B (I think T5-large / 3B can be done on NVIDIA A100 GPUs with Adafactor?). This is important for a few reasons --- (1) the proposed approach has a 2-4x slowdown in training (as the authors discuss in their limitations section), so it may be more efficient to simply scale the model size with MLE (and get similar performance); (2) scaling could go either way in terms of performance improvements, and this is important factor determining the generality of the approach (informal tweet on this in [7]) (3) some of the results (like IWSLT De-En) are below SoTA [6], and the CONT approach may be able to cover up the gap using larger Transformers.\n\n**Minor**\n\n1. It will be good to cite and discuss the relations to [8, 9] in the related work since they use contrastive learning for text generation as well. These two papers have come out in the last 2-3 months (contemporary to this paper), so no direct comparison is necessary. A few older papers with related ideas you may want to cite as well --- [10, 11, 12].\n\n**Overall**: This paper has good clarity and moderate originality, quality and significance. Overall, I liked the ideas in the work and the comprehensive evaluation setup across 10 tasks, but the lack of human evaluation + small scale of the most models tested (60M - 220M parameters) makes me slightly lean reject. The missing human evaluation is a bigger concern for me. I will be happy to move to the accept range with human evaluation on at least 2 of the 10 datasets, or a strong rebuttal discussing why the human evaluation / model scaling is not needed.\n\n---\n\n**After Rebuttal** - The authors presented a preliminary human evaluation with promising results. I am raising my score to 5, but I recommend the authors to conduct a more thorough human evaluation in the next version of the paper.\n\n[1] - https://arxiv.org/pdf/2202.06935.pdf  \n[2] - https://arxiv.org/abs/1604.00400  \n[3] - https://arxiv.org/abs/2103.06332  \n[4] - https://aclanthology.org/2020.acl-main.704/  \n[5] - https://arxiv.org/abs/2111.09525  \n[6] - https://paperswithcode.com/sota/machine-translation-on-iwslt2014-german  \n[7] - https://mobile.twitter.com/_jasonwei/status/1526589104758042624  \n[8] - https://arxiv.org/abs/2205.09726  \n[9] - https://arxiv.org/abs/2202.06417  \n[10] - https://arxiv.org/abs/2004.11714  \n[11] - https://aclanthology.org/D16-1137/  \n[12] - https://arxiv.org/abs/1804.06451  \n[13] - https://arxiv.org/pdf/2107.10821.pdf  \n[14] - https://arxiv.org/abs/2006.06264  ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "MFim5UZGNls", "reviewer": "Reviewer_gYyK", "summary": "This paper proposes to improve text generation with a sequence-level contrastive loss. The loss take advantage of self-generated negatives and consider pairwise ranking relations. The paper also proposes to use the ranking score to rerank the results during inference. Experiments on five generation tasks with ten benchmarks show its effectivenss.", "questions": "I am most curious about the computation of the sequence-level representations (i.e., z_x and z_y). This paper spends little efforts in explaining it. Essentially, the contrastive loss is at the sequence-level and the ordinary generative loss is at the token-level. and the key is how to effectively combine them. I am wondering if you test any variants of the computation of sequence-level representations (e.g., different pooling methods or more sophisticated designs).", "limitations": "n/a", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The method is well-motivated. (using self-negatives alleviates the exposure bias problem; pairwise ranking relations address the problem in previous contrastive learning approaches: ignoring the difference between negative samples)\n2. The experiments are elaborate. (five generation tasks and detailed ablation and analysis)\n3. The results seem good.\n\nWeaknesses:\n1. The improvement over Naive CL seems incremental <= as many ideas (self-generated negatives & pairwise loss) are well-known, though in different areas other than text generation.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "BvtQXz-TdOF", "reviewer": "Reviewer_2ngF", "summary": "The authors propose a new Contrastive Neural Text generation framework CONT, which improves contrastive learning in neural text generation from three aspects: the construction of contrastive examples, the choice of the contrastive loss, and the strategy in decoding. Experimental results on five generation tasks with ten benchmarks show the effectiveness of the proposed framework.", "questions": "N/A", "limitations": "N/A", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- The authors propose a simple and effective framework to improve performance of text generation. The framework addresses bottlenecks that prevent contrastive learning from three aspects.\n- The authors conduct extensive experiments. Experimental results show that CONT outperforms previous contrastive learning-based methods on ten benchmarks.\n\nWeaknesses\n- The use of self-generated texts as negative samples has been used in many works that use reinforcement learning to mitigate exposure bias problem, so the innovation of this strategy is limited. For example,\nSelf-Critical Sequence Training for Image Captioning,\nDiscourse-Aware Neural Rewards for Coherent Text Generation.\n- The use of additional similarity constraints in decoding is also used by many works.\nConsidering the above two points, I think the innovation of the whole method is limited.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "7GDgQUWHBZ5", "reviewer": "Reviewer_EAev", "summary": "The paper presents a new contrastive neural text generation framework, CoNT. There are three major technical improvement and contribution for this framework. First, they use model’s beam search result as additional contrastive examples. Second, they construct an N-pairs contrastive loss, compared to traditional pairwise loss. Third, they manipulate the hypothesis score by adding the learned similarity function. The experiment on five downstream tasks shows that the proposed methods are effective in general. \n", "questions": "The diverse beam search algorithm might not be actually diverse given my experience. Have you considered using some other methods, like sampling methods? Top-p sampling, top-k sampling, something like these. \n\nThe arrows between z_y, z_b, … in Figure 2 are a little confusing. The use of zy’ and zy’’ is also a little confusing. Please add some notation and explanation for these arrows and symbols. Also, some lines are used to connect z_y and z_x while some are for z_y and z_y’’. The function of lines and arrows needs to clearly stated. \n", "limitations": "n/a", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The approaches proposed in this paper are overall intuitive and effective. The use of model’s prediction is a smart way to collect more high quality negative examples. The n-pairs cross constrictive loss captures the relative difference between ground truth and the constructive examples. The approach is pretty general and could be applied to many tasks. \n\nThe experiment is comprehensive. It includes experimental results and analysis on 5 downstream tasks and 10 different tasks. Although some of the potential comparison is not provided, the results generally look good to me. The extensive comparison will be beneficial to the community. \n\nThe paper is well written and presented. I don’t have any major difficulty understanding the paper. \n\n\nWeakness\n\nThe results aren’t 100% solid to me. According the Liu and Liu [24] cited by this paper, on XSUM, one of the summarization dataset, BART/PEGASUS achieve 45.14 and 47.21 on ROUGE-1. The contrastive learning method proposed by Liu and Liu [24], SimCLS, achieves 47.61 on R-1. Given the paper cites [24], are there any reason 1) the “earlier SOTA” was much lower than baseline methods in previous work (SimCLS is 47.61)? 2) the SimCLS is not directly listed and compared?\nI have experience working on text summarization, so It’s easier for me to verify the results in summarization. I am a little worried since the author shows many crappy baselines “T5-xxx”, which seems to be misleading. \n\n\nThere are some mathematical issue about some of the equations. In Eq 4, the likelihood is defined as the product of the decoding sequence. A product of many probabilities will be a very small number, right? In that sense, the second term in the equation will very small compared to the first term ranging from -1 to 1. The equation is just wrong to me. Also, I don’t see the reason using alpha and 1-alpha to balance these two terms because they aren’t in the same range or scale. In Figure 2, the prob of two examples are -.48 and 0.53, which don’t seem to be the product of a sequence of probabilities. \nIn this sense, the result of Figure 5 seems spurious to me. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept