C ALIBRATING S EQUENCE L IKELIHOOD I MPROVES C ONDITIONAL L ANGUAGE G ENERATION
Google Research, Brain Team
A BSTRACT
Conditional language models are predominantly trained with maximum likelihood estimation (MLE), giving probability mass to sparsely observed target sequences.
While MLE trained models assign high probability to plausible sequences given the context, the model probabilities often do not accurately rank-order generated sequences by quality.
This has been empirically observed in beam search decoding as output quality degrading with large beam sizes, and decoding strategies beneﬁting from heuristics such as length normalization and repetition-blocking.
In this work, we introduce sequence likelihood calibration (SLiC) where the likelihood of model generated sequences are calibrated to better align with reference sequences in the model’s latent space.
With SLiC, decoding heuristics become unnecessary and decoding candidates’ quality signiﬁcantly improves regardless of the decoding method.
Furthermore, SLiC shows no sign of diminishing returns with model scale, and presents alternative ways to improve quality with limited training and inference budgets.
With SLiC, we exceed or match SOTA results on a wide range of generation tasks spanning abstractive summarization, question generation, abstractive question answering and data-to-text generation, even with modest-sized models.
I NTRODUCTION
Conditional language generation aims to generate text based on input context, and includes many useful and hard tasks such as abstractive summarization (Mani, 2001; Nenkova and McKeown, 2011), generative question answering (Bajaj et al., 2016), question generation (Zhou et al., 2017) and data-to-text (Wiseman et al., 2017; Gardent et al., 2017).
Pretraining large Transformer encoderdecoder models and ﬁne-tuning them on downstream tasks is the common paradigm to address these tasks (Raffel et al., 2020; Lewis et al., 2019; Tay et al., 2022; Zhang et al., 2019a).
Conditional language generation tasks are modeled by learning the probability of a target sequence y given a context sequence x .
Since directly modeling sequence probability P ( y | x ) over all possible generated text sequences is intractable, the canonical solution is to auto-regressively factor the probability and share the parameters at all token prediction steps as P θ ( y | x ) = (cid:81) lt =0 P θ ( y t | y 0 ...
y t − 1 , x ) , where l is the sequence length.
These models are often trained with maximum likelihood estimation (MLE) over observed target sequences.
The learning objective thus becomes L = (cid:80) Ni − log P
θ ( y i | x i ) =
(cid:80) Ni (cid:80) lt =0 − log P θ
( y ti | y
i ...
y t − 1 i , x i ) , where N is the number of training instances.
It is also referred to as next token prediction loss.
In the ideal setting of MLE training, a large number of target sequences are observed for each context, and the relative frequencies of output sequences can calibrate the assigned model probabilities.
However, in practice most language generation training datasets have only a single target sequence given the context.
While the subsequent MLE trained models learn to assign relatively high probability to plausible sequences, they lack the direct supervision to compare such sequences, and solely [Figure] e [Figure]
e S c
[Figure]
r [Figure] rely on models’ generalization capability. We refer to this phenomenon as models’ sequence likelihood not being calibrated .
Prior works (Liu and Liu, 2021; Liu et al., 2022) has shown that the correlation between sequence probability and its quality for MLE trained models can be low.
Liu et al.
(2022) attributed this similarly as the deterministic (one-point) target distribution problem.
Exposure bias (Ranzato et al., 2016) further aggravates the problem, as sequence likelihood estimation is noisier when models’ decoded sequences shift from exposed training data distribution.
Many effective heuristics have been proposed during training and decoding to combat the problem of uncalibrated sequence likelihood.
Label smoothing (Szegedy et al., 2016) prevents the network from becoming over-conﬁdent towards the observed target.
This is particularly necessary in language generation, since the gold target represents just one of many possibilities.
It has been observed that increasing number of decoding candidates past a certain point leads to worse quality for beam search decoding (Yang et al., 2018; Koehn and Knowles, 2017) and sampling (Adiwardana et al., 2020).
An optimal number of decoding candidates is often determined empirically by decoding models on the validation set and measuring their performance.
Using length normalization is also essential for beam search decoding (Wu et al., 2016) and sampling (Adiwardana et al., 2020) as models tend to underestimate sequence likelihood of longer sentences.
Repetition is another common failure mode when models overestimate the probability of repeated sequences (Holtzman et al., 2019).
Trigram blocking (Paulus et al., 2018) and nucleus sampling (Holtzman et al., 2020) have been used to interrupt repeating sequences.
These techniques are pervasive and often the default in modern
Transformer libraries (Wolf et al., 2020; Lewis et al., 2019; Raffel et al., 2020; Zhang et al., 2019a).
Since the lack of observed target sequences in MLE training is the root problem, solutions involving learning with multiple sequence candidates have been proposed to directly address it.
They can be loosely put in three categories: (1) reinforcement learning with sequence-level rewards (Paulus et al., 2018; Ziegler et al., 2019; Stiennon et al., 2020); (2) two-stage systems that generate and rerank candidates (Liu and Liu, 2021;
Ravaut et al., 2022b; Liu et al., 2022); and (3) multi-task learning with sequence-level losses (Edunov et al., 2018; Liu et al., 2022).
Refer to Related Works (section 4) for a more comprehensive discussion.
In this paper, we propose to ﬁrst decode candidates from a ﬁne-tuned model on its own training dataset, and then continue training the model with a new objective.
The new objective aims to align candidates’ sequence likelihoods according to their similarities to the target sequence in the model’s latent space.
We refer to this process as sequence likelihood calibration (SLiC).
Our approach is related to multi-task learning with sequence-level losses in Liu et al.
However, we propose a simple yet effective recipe that eliminates decoding heuristics and doesn’t risk directly optimizing the same metrics that are used to report text generation quality.
Unlike reinforcement learning, it is a one-time ofﬂine process that avoids costly online decoding processes.
Also, when compared to two-stage reranking systems, it doesn’t require a separate reranking model that incurs additional complexity and compute.
As depicted in Figure 1, our calibration stage naturally extends the current paradigm of pretraining and ﬁne-tuning, and we show that calibrated models have strong improvements over ﬁne-tuned-only models across model sizes.
Our main contributions include:
• Proposed a sequence likelihood calibration (SLiC) stage that consistently improves model quality, exceeding or matching state-of-the-art results on abstractive summarization, generative question answering, question generation and data-to-text generation tasks.
• Proposed a novel calibration similarity metric between model decodes and targets measured in the model’s latent space rather than resorting to external metrics or human feedback.
• Demonstrated that SLiC eliminates the need for popular decoding heuristics, such as beam size optimization, length normalization and repetition prevention for the calibrated models.
• Demonstrated that SLiC has persistent signiﬁcant beneﬁts on model performance even as the number of model parameters scales up.
Under the same inference budget, smaller calibrated models might outperform larger counterparts by decoding more candidates.
C ALIBRATING S EQUENCE L IKELIHOOD
We extend the common paradigm of pretraining and ﬁne-tuning by introducing a third calibration stage, SLiC. As shown in Algorithm 1, we ﬁrst decode m candidates { ˆ y } m from a ﬁne-tuned model
P θ ft ( y | x ) on ﬁne-tuning dataset { x , ¯ y } n and then calibrate the ﬁne-tuned model by continuing training on our proposed loss: L ( θ ) = (cid:80) b
L cal ( θ, s ; x , ¯ y , { ˆ y } m ) + λL reg ( θ, θ ft ; x , ¯ y ) , where θ and θ ft are the current and ﬁne-tuned-only model weights, L cal and L reg are the calibration and regularization losses.
s =
s (ˆ y , ¯ y ; x ) measures the similarity between the candidate ˆ y and the target ¯ y conditioned on the context x .
We discuss choices of s , L cal , L reg and decode strategies ˆ y ∼
P θ ( y | x ) in the following sections.
Algorithm 1 Calibrating Sequence Likelihood for x , ¯ y ∈ { x , ¯ y } n do (cid:46) sample m candidates from the ﬁne-tuned model { ˆ y ∼ P θ ft ( y | x ) } m θ ← θ ft (cid:46) initialized from the ﬁne-tuned model for { x , ¯ y , { ˆ y } m } b ∼ { x , ¯ y , { ˆ y } m } n do (cid:46) train with calibration and regularization loss θ ← θ − lr ∇ θ L ( θ )
2. S IMILARITY F UNCTION : s
For a given output sequence y , we take the decoder output hidden states e L × D = emb
( y , x ) as its representations, where L is the number of tokens and D is the hidden states dimension.
Between a candidate ˆ y ’s representations ˆ e and the target ¯ y ’s
representations ¯ e , we calculate their cosine similarities on spans of n tokens and aggregate them across the sequences with a F-measured based function F n .
Notation of F n , P n , R n are same as in BERTScore (Zhang et al., 2019b).
[Equation]
Compared to BERTScore, we use our models’ decoder output representations instead of BERT encoder representations and also consider matching on spans of n = 1 , 2 , 4 , 8 tokens rather than 1 .
Compared to using external metrics, such as ROUGE, BERTScore, this scoring function has a few advantages: (1) it adds very little compute cost, does not require extra model or out-of-graph computation; (2) it differs from the metrics that we evaluate the generation systems with and mitigates the risk of directly optimizing towards those imperfect metrics (Paulus et al., 2018; Stiennon et al., 2020); (3) it is conditioned on the context s
(ˆ y , ¯ y ; x ) , as opposed to metrics in the form of s (ˆ y , ¯ y ) .
2. C ALIBRATION L OSS : L cal
The calibration loss L cal ( θ, s ; x , ¯ y , { ˆ y } m ) aims to align models’ decoded candidates’ sequence likelihood P θ (ˆ y | x ) according to their similarity with the target sequence s (ˆ y ,
¯ y ; x ) .
Given the context x , target ¯ y and a set of candidates { ˆ y } m , we consider the following 4 loss types to answer two questions: (1) does absolute difference in similarities matter?
(2) is there beneﬁt of list-wise over pair-wise comparisons?
Rank loss optimizes the ranking order of positive and negative candidates pairs ˆ y + , ˆ y − uniformly sampled from { ˆ y } m where s (ˆ y + , ¯ y ; x ) > s (ˆ y − , ¯ y ; x ) .
Margin loss maximizes the sequence probability gap of positive and negative candidates pairs.
List-wise rank loss optimizes the ranking orders of a list of candidates, where i, j are positions of ˆ y i , ˆ y j in the set { ˆ y }
m sorted by s (ˆ y , ¯ y ; x ) .
List-wise rank loss is the contrastive loss used in BRIO (Liu et al., 2022).
Expected reward loss (or expected minimum risk) maximizes the expected similarity of a list of candidates (Edunov et al., 2018). Pair-wise losses (Rank, Margin) has smaller training memory footprint than list-wise rank and expected reward.
[Equation]
β values for all losses are chosen empirically for each loss type in subsection 3.3.
2. R EGULARIZATION L OSS : L reg
We consider two alternate types of regularization loss L reg to prevent models from deviating signiﬁcantly from their ﬁne-tuned MLE objective: Cross entropy is the standard ﬁne-tuning MLE objective used in (Liu et al., 2022).
KL divergence directly minimizes the probability distribution distance between the calibrated model and the ﬁne-tuned model at each token on observed target sequence.
The main difference is cross entropy loss regularizes the model toward the gold reference while KL divergence regularize the model toward ﬁne-tuned-only model.
The regularization losses are both on token level.
[Equation]
2. C ANDIDATES D ECODING M ETHODS
We consider the following decoding methods for SLiC:
Beam Search is the standard best-ﬁrst algorithm to solve the intractable maximum likelihood optimization for sequence-to-sequence models (Tillmann and Ney, 2003;
Li et al., 2016; Wiseman et al., 2017; Chen et al., 2018).
Diverse Beam Search (DBS; Vijayakumar et al., 2016) generates a list of diverse outputs by dividing the beam search budget into groups and enforcing dissimilarity between groups of beams.
It strikes balance between quality and diversity and is often the best strategy for two-stage reranking systems (Liu and Liu, 2021; Ravaut et al., 2022b;
Liu et al., 2022).
Nucleus Sampling (Holtzman et al., 2020) only samples high-probable tokens within cumulative probability p at each step of the decoding.
It produces diverse candidates while preventing sampling very low quality ones.
E XPERIMENTS 3. T ASKS AND D ATASETS
For abstractive summarization tasks, we choose CNN/DailyMail (Hermann et al., 2015; See et al., 2017), XSUM (Narayan et al., 2018), RedditTIFU-long (Kim et al., 2019) and SAMSum (Gliwa et al., 2019) due to their diversity in domain, style, abstractiveness, and summary lengths.
For question answering related tasks, we choose generative question answering given context MSMARCO
NLG (Bajaj et al., 2016) and its reverse problem of question generation SQuAD QG (Zhou et al., 2017; Du et al., 2017) .
For data-to-text tasks, we choose text generation given structured data
3. M ODEL TRAINING AND EVALUATION DETAILS
We follow the PEGASUS pretraining (Zhang et al., 2019a) and extend transformer model sizes to
PEGASUS SMALL (50M), PEGASUS BASE (200M), PEGASUS LARGE (500M) and PEGASUS 2B (2B).
Details are reported in Appendix B. Different from the original paper, we use a sentencepiece 96k vocabulary with byte-fallback (Kudo, 2018) and pretraining batch size of 4096 across all models.
See Appendix B for model dimensions.
In all experiments, we use learning rate lr = 10 − 4 , and batch sizes of 512 to ﬁnetune and 64 to calibrate models.
We use beam search to generate calibration candidates and evaluate the calibrated models, unless speciﬁed otherwise.
All ﬁne-tuned-only models utilizes heuristics such as beam size optimization and sweeping beam α for length normalization, unless speciﬁed otherwise.
In our ablation studies (subsection 3.3), beneﬁts analysis (subsection 3.4), and scaling experiments (subsection 3.5), we use models pretrained to 500 , 000 steps and conduct experiments on 4 datasets (CNN/DailyMail, XSUM, RedditTIFU-long and SAMSum).
For ablation studies and beneﬁts analysis, we use PEGASUS LARGE .
We report ROUGE 1/2/L (Lin, 2004) for each dataset on validation splits and their overall score R m deﬁned as geometric mean of ROUGE 1/2/L averaged across datasets, R m = 14 (cid:80) d 3 √ R 1 R 2 R L .
For the ﬁnal results (subsection 3.6), we pretrain PEGASUS 2B model to 2 .
5 M steps, ﬁne-tune it on all 8 datasets, calibrate them using the same recipe and report numbers on the test split (unless speciﬁed otherwise).
We use corresponding standard evaluation scripts for each dataset (details in
Appendix D).
3. A BLATION S TUDIES OF C ALIBRATION
Ablation experiment results discussed below can be found in Table 1.
[Table]
Similarity Function We compare our proposed similarity function, using models’ latent states at decoder output representation s θ (ˆ y , ¯ y ; x ) (subsection 2.1), to directly optimizing the evaluation metric
ROUGE.
They perform similarly on all datasets even when evaluation metrics are ROUGE scores.
We also test a variant of our similarity function by replacing decoder representation emb ( y , x ) with token embeddings.
This variant has lower performance, which suggests beneﬁts of contextualized and input-dependent representations.
Calibration Loss Calibrated models with all loss types signiﬁcantly improve over ﬁne-tuned-only models.
Rank loss performs the best followed by margin, list rank and then reward.
Reward maximization has the advantage of no hyper-parameters β (Equation 1) to sweep while rank and margin loss have smaller training memory footprints.
Rank loss showing the best gain indicates that relative ordering of candidates is more important than the absolute value of their similarity to the target.
Regularization Loss Cross entropy and KL divergence regularization perform similarly.
About 85% of the calibration gain remains if regularization is removed.
Calibration Candidates Decoding Method We choose hyper-parameters for calibration candidates decoding methods based on validation set.
The optimal decoding method is dataset dependent, however the differences between methods are small and the worst method achieves 90% of the gains of the best one.
Beam search yields the highest average quality.
This is opposite to the ﬁndings in the two-stage reranking systems (Liu and Liu, 2021; Ravaut et al., 2022b;
Liu et al., 2022), where more diverse decoding strategies are preferred.
Checkpoint Selection for Fine-tuned Model We compare ROUGE-selected and perplexityselected checkpoints.
The experiments show that starting calibration from the perplexity-selected checkpoint yields same or better performance with the biggest gap on CNN/DailyMail dataset.
TL;DR: We recommend a simple recipe: select the ﬁne-tuned model’s checkpoint by its validation set perplexity; decode candidates using beam search; calibrate the model with rank loss (based on decoder states similarities) and KL divergence regularization.
3. B ENEFITS OF C ALIBRATED S EQUENCE L IKELIHOOD
[Figure]
Calibrated models’ quality monotonically improves as the number of decoding candidates increase, 1 regardless of the calibration-decoding and evaluation-decoding methods, as shown in Figure 2.
On the other hand, ﬁne-tuned-only models suffer from decreased quality when the number of decodes exceeds an optimal value.
Once a model is calibrated with either decoding method, it performs well with both at evaluation time.
Decoding with beam search yields higher scores, veriﬁed up to 20 decodes.
When the calibration-decoding and the evaluation-decoding method align, the ﬁnal quality is slightly better than the mismatched settings.
CNN/DailyMail, XSUM, and SAMSum datasets work best with beam search, however RedditTIFU-long works better with nucleus sampling and decoding it with a larger number of candidates may achieve better results.
Calibrated models do not require length normalization.
As shown in Table 2, length normalization (commonly implemented as α for beam search) is essential for ﬁne-tuned-only models which bias towards longer sequences at decoding time.
In contrast, length normalization has minimal effect on calibrated models.
Calibrated models suffer from far fewer repetitions.
The repetition rate (rep%) measures a common mode of model failures.
It is deﬁned as the percentage of examples that contain any kind of consecutive repeated word n-grams, While length normalization helps general quality on the ﬁnetuned-only models, it leads to a side-effect of higher repetitions.
Calibrated models, with or without [Table] length normalization, have a much lower repetition rate.
When we compare with the repetition rate in the gold reference (repetition may occur naturally), calibrated models without length normalization have similar or lower repetition rate.
TL;DR: Calibrated models do not require decoding heuristics such as beam size optimization, length normalization and repetition blocking.
3. S CALING P ROPERTIES OF C ALIBRATED M ODELS
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure] [Figure]
[Figure] [Figure]
[Figure]
RedditTIFU-long
[Figure]
[Figure] [Figure]
[Figure] [Figure]
[Figure] [Table]
CNN/DailyMail 1010compute (FLOPs)      s
[Figure]
e ( R 1 / R/ R L )
[Figure]
1010compute (FLOPs)      s
[Figure]
e ( R 1 /
[Figure] [Figure]
) 50M
[Figure]
2B 1010compute (FLOPs)      s c o r e ( R 1 / R/ R L ) 50M
[Figure]
2B 0.
[Table]
XSUM
[Figure]
10compute (FLOPs)
[Figure]
Scaling properties are important for projecting a technique’s future relevance as models scale up (Kaplan et al., 2020a).
In Figure [Figure] we compare generation quality versus inference compute at different model sizes and number of decoding candidates using beam search.
Appendix H describes the method to estimate inference compute FLOPs
[Figure] As mentioned earlier in subsection 3. [Figure] ﬁne-tuned-only models have optimal decoding beam sizes while calibrated models’ performance monotonically increase with larger decoding beam sizes.
Even in the case of greedy decoding (beam size of 1), the calibrated models’ performance exceeds the ﬁne-tuned-only models, by a large margin for some datasets (CNN/DailyMail and RedditTIFUlong).
Their gaps grow larger with increasing number of beam sizes.
The magnitude of quality improvement from calibration persists over models sizes spanning from 50M to 2B .
There is no obvious sign of diminishing return as model size scales up.
Inference compute may be used for decoding rather than on larger models.
A calibrated model, once trained, can improve its performance by decoding more candidates, usually more effectively in the beginning, although returns diminish over 10 candidates.
In some cases (SAMSum and especially CNN/DailyMail), a smaller model decoding more candidates can beat a larger one at both quality and efﬁciency.
TL;DR: Calibration beneﬁts persist as model sizes scale up.
Smaller calibrated models can outperform larger ones under the same inference compute budget.
3. F INAL R ESULTS
[Table]
We calibrate the ﬁne-tuned PEGASUS 2B models on 8 language generation tasks using the simple recipe identiﬁed in subsection 3.3 and evaluate them with beam search without decoding heuristics (subsection 3.4).
The only hyper-parameter we optimize for SLiC is learning rate lr (Appendix J).
We use beam size 5 for ﬁne-tuned-only models and 10 for calibrated models.
As shown in Table 3, calibrated models show consistent improvement over ﬁne-tuned-only models across datasets and tasks.
Overall, our calibrated models exceed or match the SOTA models on all datasets.
On XSUM, SAMSum, WebNLG-en and CommonGen, our calibrated 2B models are ten to a hundred times smaller than the SOTA models.
TL;DR: PEGASUS 2B achieves SOTA results on a wide range of language generation tasks using a simple SLiC recipe while eliminating decoding heuristics.
R ELATED W ORKS
In classiﬁcation, model calibration often refers to matching output probabilities with expected accuracy.
In our case of sequence generation, how to generalize this notion is unclear.
Kuleshov and
Liang (2015) explores generalizing probabilistic calibration to structured prediction, but we only focus on aligning the sequence likelihood with target sequence similarity.
Other approaches in this vein are described below.
4. RL APPROACHES
Paulus et al.
(2018) directly optimizes evaluation metric ROUGE in RL ﬁne-tuning stage.
One issue is that ROUGE metric does not enforce ﬂuency.
The authors found summaries to be not always readable and proposed that using a mixed training objective works better.
Ziegler et al.
; Stiennon et al.
(2020) collects human judgements on ﬁne-tuned models’ decodes to train a reward model that ranks candidates according to human preferences.
The supervised policy is then ﬁne-tuned against the reward model using PPO.
The authors found that optimizing their reward model results in better quality summaries than directly optimizing ROUGE.
4. T WO - STAGE RERANKING APPROACHES
SimCLS (Liu and Liu, 2021) proposes formulating text generation as a reference-free quality estimation problem assisted by contrastive learning.
The ﬁrst stage decodes candidates with diverse beam search and a RoBERTa based model is used to rank them in the second stage.
SummaReRanker (Ravaut et al., 2022a) observes improved performance when training the generation and the reranking models on two non-overlapping halves of the ﬁne-tuning data compared to training two models on the same data.
Bhattacharyya et al.
(2021) trains an energy-based model to mimic the behavior of the task measure such as BLEU scores.
Lee et al.
; Fernandes et al.
(2022) train rerankers for neural machine translation (NMT) that predicts the observed distribution of desired automatic metrics (BLEU, COMET and BLEURT) over the n-best list.
BRIO (Liu et al., 2022) includes a two-stage reranking system that uses sequence-to-sequence generation models.
It is shown that the sequence-to-sequence reranker has better performance than encoder-only models in providing ranking scores.
4. M ULTI T ASK L EARNING WITH S EQUENCE - LEVEL L OSS
Edunov et al.
(2018) surveys a range of classical objective functions for structured prediction and apply them to sequence-to-sequence models.
Their experiments showed that combining sequencelevel objectives with token-level objectives yields improved performance on translation and summarization datasets.
Sun and Li (2021) combines contrastive learning objective with negative log-likelihood to decrease the likelihood of the model generated “silver” summaries meanwhile increasing the likelihood of the “gold” references.
Wieting et al.
(2019) introduces an alternative reward function for optimizing neural machine translation systems that is based on semantic similarity.
BRIO (Liu et al., 2022) demonstrates that multi task learning of sequence candidates with contrastive reranking and token-level generation has better performance compared to a two-stage reranking system.
The ranking order is determined by similarity to target using external metrics (ROUGE,
BERTScore).
Models trained to rank by ROUGE also perform well measured on BERTScore and vice versa.
Lukasik et al.
(2020) extends label smoothing from classiﬁcation tasks to semantic label smoothing for sequence-to-sequence learning.
Their technique adds sequence-level losses that smooth over well-formed relevant sequences that are similar to the target sequence semantically and on n-gram level.
C ONCLUSION
We propose adding a third stage of sequence likelihood calibration (SLiC) after the pretraining and ﬁne-tuning stages for conditional language generation.
A simple yet effective recipe for SLiC is using decoder-state similarity, selecting the ﬁne-tuned model’s checkpoint by perplexity, decoding candidates with beam search, calibrating with rank loss and KL divergence regularization.
We are able to eliminate all decoding heuristics for calibrated models.
The beneﬁts of calibration persist as models scale up in size.
Smaller calibrated models might outperform larger ones under the same inference compute budget.
By calibrating a PEGASUS 2B model, we exceed or match state-of-the-art results on 8 datasets spanning abstractive summarization, generative question answering, question generation and data-to-text tasks.
In this work we focus on the setting where ground-truth output sequences are provided.
However, this presupposes high-quality labels that are not always available.
In the future, we plan to extend
SLiC to general language modeling and explore more types of latent similarities.
A CKNOWLEDGEMENT
We thank David Grangier for early and engaging discussions, and Noah Fiedel for feedback on the paper.
A D ATASETS P ROPERTIES A. D ATASETS AND TASKS
CNN/DailyMail (Hermann et al., 2015; See et al., 2017) summarization dataset contains 313k articles from the CNN and Daily Mail newspapers with bullet point summaries.
The summaries are on average 3-4 sentences and relatively extractive.
XSUM (Narayan et al., 2018) summarization dataset consists of 227k BBC articles from 2010 to 2017 with a single sentence highly abstractive summary.
Sometimes the summary contains information not present in the article.
RedditTIFU-long (Kim et al., 2019) summarization dataset contains 42k posts of informal stories from sub-reddit TIFU from 2013-Jan to 2018-Mar with author written summaries.
The style and length of the summaries are very diverse.
SAMSum (Gliwa et al., 2019) summarization dataset contains 16k high-quality chat-dialogues and their summaries written by linguists.
SQuAD QG (Zhou et al., 2017; Du et al., 2017) is the task of generating a question from a passageanswer pair extracted from the SQuAD dataset (Rajpurkar et al., 2016).
In particular, we use the split of Du et al.
(2017), consisting of 75,722, 10,570, and 11,877 examples for training, validation, and testing, respectively.
MSMARCO NLG (Bajaj et al., 2016) is a large scale dataset focused on machine reading comprehension and question answering.
The original QA dataset consists of 1,010,916 queries.
However, we work on the NLGEN data that is a subset of the QA data consisting of 182,669 queries, each with a well formed answer.
The task is to generate a well formed answer to an input query and a set of answering passages.
WebNLG-en (Gardent et al., 2017) consists of 16,095 data inputs in the form of sets of RDF triples extracted from DBpedia.
Each data point was verbalized by humans in more-than-one natural texts, leading to a total of 38,872 data-text pairs.
CommonGen (Lin et al., 2020) introduces a task of generating a coherent sentence describing an input set of common concepts.
The dataset consists of a total of 35,141 common concept sets, split into 32,651/993/1,497 training/validation/test sets.
There are 67,389, 4,018 and 6,042 sentences in training, validation and test, respectively.
[Table]
[Table]
[Table]
[Table]
org datasets
[Table]
https: tensorflow.org/datasets/catalog/squad_
B M ODEL A RCHITECTURE
Model sizes and their conﬁgurations are reported in Table 5.
[Table]
C H YPER - PARAMETER N OTATIONS
[Table]
D E VALUATION S CRIPTS
For summarization tasks, we use pypi package rouge-score to report ROUGE numbers.
We report rougeLsum for ROUGE-L.
For SQuAD QG and MSMARCO NLG, we use the original evaluation scripts provided by Du et al.
(2017) and Bajaj et al.
(2016), respectively.
For WebNLG-en and CommonGen, we use the versions from the GEM benchmark (Gehrmann et al., 2021) and report using the GEM evaluation framework.
Those scripts mainly differ in text tokenization methods.
E A BLATION S TUDY
SLiC methods for ablation study are reported in Table [Table]
[Table]
F D ECODING M ETHODS
SLiC methods for decoding calibrated models are reported in Table 8.
At evaluation time, models are decoded with 1, 2, 5, 10 and 20 candidates.
ROUGE numbers in Figure 2 are reported in Table 9.
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
G L ENGTH N ORMALIZATION
Experimental settings for length normalization analysis is reported in Table 10.
Brevity penalty α is chosen as the best value for ﬁne-tuned models’ ROUGE performance on validation dataset or disabled.
[Table]
H M ODEL F LOPS E STIMATION
We extends formulations in Table 1 of Kaplan et al.
(2020b) to estimate FLOPs of our transformer encoder decoder models following the formula:
[Equation]
(3) where m is the number of decoder candidates, other notations can be referenced in Table 1 of
Kaplan et al.
(2020b).
Because of upper triangle attention masking, the effective decoder attention context length is half of sequence lengths instead of full sequence lengths as in the encoder.
Extra computation incurred by different decoding methods are omitted as they are much smaller.
I S CALING
SLiC method for scaling curves are reported in Table 11.
At evaluation time, models are decoded with 1, 2, 5,
[Table]
and maybe 15, 20 candidates.
ROUGE numbers in Figure 3 are reported in Table 12.
[Table]
[Table]
- 52.38/27.68/43.68 500M45.34/21.47/42.60 46.27/23.02/38.
- 53.68/29.16/44.61 2B 45.52/21.70/42.73 47.89/24.54/39.
calibrated 50M 44.31/20.82/41.65 41.41/17.95/33.
- 51.90/26.71/42.16 200M45.26/21.16/42.57 44.54/21.18/36.
- 53.99/28.90/44.39 500M45.55/20.85/42.76 46.42/22.93/38.
- 54.87/30.31/45.39 2B 46.29/21.92/43.47 48.11/24.59/39.
J F INAL R ESULTS
SLiC method for ﬁnal results is reported in Table 13.
We choose the SLiC best based on subsection 3.3.
There are in total 3 hyper-parameters: learning rate lr (Algorithm 1), ranking constant β (Equation 1), and regularization strength λ (Equation 2).
We ﬁx two of the them: β is set to
[Table]
and lr ∗ λ is set to 1 e − 5 .
Best learning rate lr is determined with hyper-parameter tuning on validation set and reported in Table
[Table]
[Table] [Table]