Out-of-Distribution Detection and Selective Generation for Conditional Language Models
Authors: Jie Ren
Jiaming Luo
Yao Zhao
Kundan Krishna
Mohammad Saleh
Balaji Lakshminarayanan
Peter J Liu
Venue: ICLR 2023 notable top 25% (2023)
Keywords: Out-of-distribution Detection
Natural Language Generation
Selective Generation
Uncertainty

Abstract: Machine learning algorithms typically assume independent and identically distributed samples in training and at test time (IID). Much work has shown that high-performing ML classifiers can degrade significantly and provide overly-confident, wrong classification predictions,  particularly for out-of-distribution (OOD) inputs. Conditional language models (CLMs) are predominantly trained to classify the next token in an output sequence, and may suffer even worse degradation on OOD inputs as the prediction is done auto-regressively over many steps. Furthermore, the space of potential low-quality outputs is larger as arbitrary text can be generated and it is important to know when to trust the generated output. We present a highly accurate and lightweight OOD detection method for CLMs, and demonstrate its effectiveness on abstractive summarization and translation. We also show how our method can be used under the common and realistic setting of distribution shift for selective generation (analogous to selective prediction for classification) of high-quality outputs, while  automatically abstaining from low-quality ones, enabling safer deployment of generative language models.

Published as a conference paper at ICLR 2023
OUT-OF-DISTRIBUTION DETECTION AND SELECTIVE GENERATION FOR CONDITIONAL LANGUAGE MODELS
Jie Ren' Jiaming Luo' Yao Zhao' Kundan Krishna
Mohammad Saleh' Balaji Lakshminarayanan' Peter J Liu"
Goooge Research CCanegie Mellon University, work done while at Google Research
*Correspondence to: {ijren, peterijiuJ@@gogge.com
ABSTRACT
Machine learning algorithms typically assume independent and identically dis- tributed samples in training and at test time. Much work has shown that high- performing ML classifiers degrade significantly and provide everly-confident, wrong classification predictions, particularly for out-of-distribution (OOD) inputs. Conditional language models (CLMs) are predominantly trained classify the next token in an output sequence, and may suffer even worse degradation on OOD inputs as the prediction done rugr-ssegsyy over many steps. Furthermore. the space of potential low-quality outputs is larger as arbitrary text can be gener- ated and it is important to know when to trust the generated output. We present a highly accurate and lightweight OOD detection method for CLMs, and demon- strate its effectiveness on abstractive summarization and translation. We also show how our method can be used under the common and realistic setting of distribution shift for selective generation (analogous to selective prediction for classification) of high-quality outputs, while automatically abstaining from low-quality ones, abling safer deployment of generative language models.
INTRODUCTION
Recent progress in generative language models (Wu et al., 201 Radford et al., 2019; Lewis et al., 2020; Raffel et al., 2020; Zhang et al., 2020) has led to quality approaching human- cemman-prrfommncee on research datasets and has opened up the possibility of their wide deployment beyond the aca- demic setting. In realistic user-facing scenarios such text summarization and translation, it should be expected that user provided inputs can significantly deviate from the training data distribution. This violates the independent, ddentically-distribuee (IID) assumption commonly used in evaluating machine learning models.
Many have shown that performance of machine learning models can degrade significantly and in surprising ways on OOD inputs (Nguyen et al., 2014; Goodfellow et al., 2014; Ovadia et al., 2019). For example an image classifier may detect cows in images with very high accuracy on its IID test set but confidently fails to detect cow when paired with an unseen background (Murphy, 2023; Nagarajan et al., 2020). This has led to active research on OOD detection for a variety of domains, including vision and text but focused primarily on classification. Salehi al. (2021); Bulusu et al. (2020); Ruff et al. (2021) provide comprehensive reviews on this topic.
Conditional language models are typically trained given input sequence regressively generate the next token in a sequence = -/r as a classification over the token- vocabulocy V, ()())=) I Po (y y<e, æ)), Vs E V. Consequently, the perils of out-of- distribution are arguably more severe as (a) eтrors propagate and magnify through auto-regrsssion, and (b) the space of low-quality outputs is greatly increased as arbitrary text sequences can be gen- erated. Common errors from text generation models include disfluencies (Holtzman al., 2020) and factual inaccuracies (Goodrich et al., 2019; Maynez et al., 2020). A common failure case observed in abstractive summarization is for the model to output "All images are copyrighted" as the summary for news articles from a publisher (CNN) different than what was trained on (BBC) (see Figure A.7).
31 -1... FL auto-
In this work, we propose OOD detection methods for CLMs using abstractive summarization and translation as case studies. Similar to classification, we show in Section 2.1 that CLMs have un- trustworthy likelihood estimation OOD examples, making perplexity a poor choice for OOD

---

Published as a conference paper at ICLR 2023
detection. In Section 2.2, we propose highly-accurate, simple, and lightweight OOD score based on the model's imput and output representations (or embeddings) to detect OOD examples, requiring negligible additional compute beyond the model itself.
While accurate OOD detection enables the conservative option of abstaining from generation on OOD examples, it may be desirable generate known near-domain data, e-g. generate sum- maries for articles from news publishers that differ from our fine-tuning set. Thus the ability to selectively generate where the model is more likely produce higher-quality outputs, enables safer deployment of conditional language models. We call this procedure selective generation, analogous the commonly used term selective prediction in classification (Chow, 1957: Bartlett & Wegkamp, 2008: Geifman & El- Yaniv, 2017). In Section 4, we show that while model perplexity is reason- able choice for performing selective generation with in-domain examples, combining with our OOD score works much better when the input distribution is shifted.
In summary, our contributions are:
Propose lightweight and accurate scores derived from CLM's embeddings for OOD de- tection, significantly outperforming baselines on abstractive summarization and translation tasks, without the need for separate detection model.
Show that model perplexity can be an unreliable signal for quality estimation on OOD ex- amples, but combined with our OOD scores can be used effectively to selectively generate higher-quality outputs while abstaining on lower ones.
Propose an evaluation framework for OOD detection and selective generation for CLMs, including human quality ratings for summarization.
OOD DETECTION IN CONDITIONAL LANGUAGE MODELS
The maximum softmax probability (MSP), p(y]=), y = arg max k- I...... (k) is a simple, com- monly used OOD score for K-class classification problem (Hendrycks & Gimpel, 2016; Lakshmi- narayanan et al., 2017). For CLMs, the perplexity, which is monotonically related to the negative log-likelihood of the output sequence averaged over tokens Σ log P(U) Wyc,, 2)) is natural OOD score to consider, and analogous to the negative MSP in classification because both are based on softmax probabilities, We first study how well the perplexity performs for OOD detection tasks.
2.1 PERPLEXITY IS IUL-SUITED FOR OOD DETECTION
(a) Summarization
(b) Translation
holdout
crn_dailymail newsroom_ae
rdt2015
I E x
Koran
Denstty
Denitty
forumsum
subtitles
sam
ment
perplexily
perplexity
Figure 1: Perplexity scores density of a CLM trained on (a) xsum for summarization, and (b) WMT for translation, evaluated on other datasets/domains Perplexity is not well suited for OOD detection due to significant overlap between in-domain and OOD scores.
In Figure 1, we compare the distribution of perplexity of (a) summarization model and (b) translation model trained on in-domain dataset and evaluated on multiple OOD datasets, respec- tively. For summarization, model is trained xsum and evaluated on other news datasets in- cluding cnn_de man_aillyai and newsroom as near-OOD datasets, and forum (forumsum) and dialogue (samsum and reddit_tifu) datasets as far-OOD (see Section 3 for details). The perplexity distribu- tions overlap significantly with each other even though the input documents are significantly differ- ent. Furthermore, perplexity assigns enn dailymail even lower scores than the in-domain xsum.
For translation, the model trained on WMT15 dataset and evaluated on other WMT test splits (Bojar et al., 2015), OPUS100 (Aulamo & Tiedemann, 2019), and MTNT (Michel & Neubig, 2018). The in-domain and OOD datasets perplexity densities overlap even more. Overall, these results suggest that perplexity is not well suited for OOD detection.
2

---

Published as a conference paper at ICLR 2023
2.2 DETECTING OOD USING CLM's S EMBEDDINGS
Given a trained conditional language model, we propose using the input and output representa- tions/embeddlings computed part of the inference/gencraaiov process to detect OOD examples. In this work, we use Transformer encoder-deooer models and obtain the input embedding = by averaging the encoder's final-layer hidden state vectors h, E R (d is the hidden dimension) cor- responding to the input sequence token Z₁. To obtain the output embedding w we average the decoder's final-layer hidden state vectors gi I2d corresponding to the output token yı. Thus
1 T
w
9:.
E IR"
where and are the input and output sequence lengths respectively. Figure illustrates the idea.
"Sory not rspport Euectinn for Whl ippar.
High 005
Indormain
Inpur CCD
Qutpue 0CD acore
Irpui OCD Delactor
Output OOD Datactor
Ingul Emboddinng
Oulput Eribedding
Input Text
Supporad Doman
Encoder
Duooder
Cutpui Text
Unsupported Domain
Conditiona Languago
Porionky
Figure 2:
The proposed OOD detector based on input and output embeddings.
Intuitively, the embedding of a test input or output is far from the embedding distribution of the training data, it is more likely to be OOD. One way of measuring this distance is to fit Gaussian, N(p, E). μ E Rd, E E Rdxd to the training embeddings and use the Mahalanobis distance (MD):
MD(x; p. Σ) = (x 11) (x (1-p))) ps);
This has been used for OOD detection using the representations from classification models (Lee al., 2018) and computing the distances to caasscoonditional Gaussians.
Unlike classification, which has class labels, in conditional language modeling we have paired input and output text sequences. We fit one Gaussian on the training input embeddings, N(1u², "*),,, and second Gaussian on the embeddings of the training ground-truth outputs, N(11" ".)
>²),
For test input and output embedding pair (Ziest Whea), the input MD is computed
MD Input (reat = MD( Zieat: μe" )²)
(Input MD OOD score)
The output MD is computed similarly:
MDouguut (wess) MD (D( f "")
(Output MD OOD score)
Mahalanobis distance is equivalent to computing a negative log- liglliielihodd of the Gaussian distribu- tion (up to a constant and scalar), i.e. log p(=) = log(2r) + 1 log + 11)''
μ4) = const. + 1 MD()).. Ren et al. (2019) showed that normalizing the likelihood with the likeli- hood of background model works better for OOD detection. In similar vein, Ren al. (2021) proposed an analogous Relative Mahalanobis Distance (RMD) for classification: using the rela- tive distance between the class-conditional Gaussians and single background Gaussian using data from all classes. That method cannot be directly applied for CLMs because outputs are not just labels. Thus in this work, we extend the RMD idea to conditional language models,
just class
RMDinpur Ziest) = MDingur (Zeest) MD (Zeezi)),
(Tnput RMD OOD score)
where MD, (2ies) (D(zie 14 Σg) is the MD to background Gaussian N( Eg), fit using a large, broad dataset to approximately represent all domains. In practice, we use C4, a large Com- mon Crawl-based English dataset (Raffel et al., 2020)' and ParaCrawl's English-French dataset
3

---

Published as a conference paper at ICLR 2023
(Bañón et al., 2020)?, as the data for fitting the background distributions for summarization and translation in our experiments, respectively.
While we use the ground-truth outputs to fit Wit >"),, we decode outputs from the trained CLMs and use those output embeddings to fit the background output Gaussian, .N Nu 28) )
RMDautpal (Wess) = autppll (Wess MD3 (ure(:
(Output RMD OOD score)
where M == MD(wesst p "g" is the MD to the decoded ouiput background distribution N(po 2 ). See Algorithm and for the detailed steps. Using decoded outputs serves two pur- poses: (1) We do not require supervised data (e.g. document-summary pairs) to fit the background Gaussian. (2) Decoded outputs may exhibit increased deficiencies that result from running the model on -ittof-diitribution data, which provides greater contrast with the in-domain ground-truth labels.
The RMD score can be regarded as background contrastive score that indicates how close the test example is to the training domain compared to the background domains. A negative score suggests the example is relatively in-domain, while a positive score suggests the example is OOD, A higher score indicates greater OOD-ness.
Binary classifier for OOD detection Since we have explicitly defined two classes, in in-domain and background/geenrall domain, another option is to train binary classifier to discriminate embeddings from the two classes. We train a logistic regression model and use the un- unnnormalized logit for the background as an OOD score. The Input Binary logits OOD score uses the input embeddings as features, whereas the Output Binary logits OOD score uses the decoded output embeddings as features. A higher score suggests higher likelihood of OOD. The preferred use of the logits over probability was also recommended by previous OOD studies for classification problems (Hendrycks al., 2019). Though RMD generative-model based approach and the binary classifier is a discriminative model, we show that RMD is a generalized version of binary logistic regression and can be reduced a binary classification model under certain conditions (see Section A.5 for details).
3 EXPERIMENTS: OOD DETECTION
3.1 EXPERIMENT SETUP
We run our experiments using Transformer ( (Vaswani et al., 2017) encoder-decoder models trained for abstractive summarization and translation. Below we specify the dataset used for training/fine- tuning (i.e. in-domain) and the OOD datasets.
In the case of summarization, OOD datasets can be intuitively categorized as near or far OOD based on the nature of the documents. For example, news articles from different publishers may be considered as sourced from different distributions, but are closer than news articles are to dialogue transcripts. We also quantitatively showed that using n-gram overalp analysis in Table A. In contrast, the translation datasets we use consist of English-Frenne sentence pairs with less variation between datasets due to the shorter length of sentences.
Summarization model We fine-tuned PEGASUSLARGE (Zhang et al., 2020) on the xsum (Narayan et al., 2018) dataset, consisting of BBC News articles with short, abstractive summaries.
Summarization datasets We use 10,000 examples from xsum and C4 training split to fit in- domain/foreground and background Gaussian distributions, respectively. For test datasets, we have cnn dailymail (Hermann et al., 2015; See et al., 2017), news articles and summaries from CNN and MailyMail; newsroom (Grusky al., 2018), article-summary pairs from 38 major news publi- cations; reddit_tiff (Kim et al., 2018), informal stories from sub-reddit TIFU with author written summaries of very diverse styles; samsum (Gliwa et al., 2019) and forunsum (Khalman et al., 2021), high-quality summaries of casual dialogues.
Translation model We train a Transformer base model (Vaswani et al., 2017) with embedding size 512 on WMT15 English-French (Bojar et al., 2015). The model is trained with Adafactor optimizer (Shazeer & Stern, 2018) for 2M steps with 0.1 dropout and 1024 batch size. Decoding is done using beam search with 10 beam size and = 0.6 length normalization (Wu et al., 2016b). The best checkpoint scores 39.9 BLEU on newstest2014.
hhtp:://www..aaoooooooo////////e erawl
4

---

Published as a conference paper at ICLR 2023
Table 1: AUROCs for OOD detection. For summarization task (a), enn dailymail and newsroom are considered as near shift OOD since it shares news topics xsum, and reddit.tifu, forum- sum, and samsum are far shift OOD. For translation (b), WMT dataset contains various test WMT datasets collected from different years, OPUS contains five different domains (the degree of shift varies), and MTNT contains noisy data from Reddit.
(a) Summarization
Near Shift OOD
Far OOD
Measure
cmn dailymail
newsrnom
reddit cifu
forumsum
samssm
MD
0.651
0.799
0.974
0.977
0.995
RMD
0.828
0,930
0.998
0.997
0.999
Binary logits
0.997
0,959
1,000
0.999
0.998
OUTPUT OOD
Perplexity (haseline)
0.424
0.665
0,909
0.800
0.851
NLI score (baseline)
0.400
0,469
0,709
0.638
0.743
MD
0.944
0,933
0,985
0.973
0.985
RMD
0,958
0.962
0.998
0.993
0.998
Rinary logits
0.989
0.982
1.0000
0.998
0.997
(b) Translation
OPUS
MINT
Mensure
nt2014
mdd2015
nd:2015
law
medical
Koran
sub
INPUT
MD
0,534
0.671
0.670
0,511
0.704
0.737
0.828
0,900
0.668
0,798
0,866
0.389
0.840
0.957
0.959
0.943
Binary logits
0.864
0,904
0.904
0.485
0.813
0,963
0.928
0.950
0.963
OUTPUT OOD
Peplexitt (baseline)
0.570
0.496
0.494
0.392
0.363
0.637
0.343
0.359
0.633
COMET (baseline)
0.484
0.543
0.632
0.724
Prism (bascline)
0.445
0,504
0,505
0,459
0.565
0,716
0.604
0.577
0.699
MD RMD
0.609
0.733
0.739
0.482
0.784
0.838
0.900
0.935
0.794
0.786
0.858
0.861
0.355
0.845
0.939
0.951
0.959
0.922
Binary logits
0.822
0.860
0.865
0.507
0.783
0.942
0.890
0.910
0.931
We use 100,000 examples from WMT15 En-Fr and the same number of examples from ParaCrawl En-Fr to fit the foreground and background Gaussians, respectively. For test, we use newstest2014 (nt14), newsdiscussdev2015 (ndd15), and newsdiscusstest2115 (ndt15) from WMT15 (Bojar et al., 2015) and the law, Koran, medical, IT, and subtitles (sub) subsets from OPUS (Tiedemann, 2012; Aulamo & Tiedemann, 2019). We also use the English-French test set of MTNT (Michel & Neubig, 2018), consisting of noisy comments from Reddit.
Translation datasets
Evaluation metric We use the area under the ROC curve (AUROC) between the in-domain test data as negative and the OOD test data as positive sets to evaluate and compare the OOD detection performance. AUROC 1.0 means a perfect separation, and 0.5 means the two are not distinguishable.
Baseline methods We compare our proposed OOD scores with various baseline methods, includ- ing (1) the model perplexity score, (2) the embedding-baat based Mahalanobis distance. In addition, we also compare with (3) Natural Language Inference (NLI) score (Honovich al., 2022) for summa- rization, and (4) COMET (Rei et al., 2020) and (5) Prism (Thompson & Post, 2020) for translation. NLI score measures the factual consistency by treating the input document as premise and the gen- erated summary as hypothesis. Both COMET and Prism are quality estimation metrics designed to measure translation quality without access to a human reference. More specifically, COMET finetunes the large XLM-R model (Conneau et al., 2020) on human evaluation data, and Prism is the perplexity score from a multilingual NMT model trained on 99.8M sentence pairs in 39 languages.
3.2 RESULTS
RMD and Binary classifier are better at OOD detection than baselines Table shows the AUROCS for OOD detection on the (a) summarization and (b) translation datasets. Overall, our proposed OOD scores RMD and Binary logits outperform the baselines with high AUROCs (above 0.8). The commonly used output metrics, perplexity, NLI, COMET and Prism, have generally low AUROC scores (many have values around 0.5-0.6), suggesting they not suited for OOD detec- tion. Interestingly, we noticed that the output OOD scores perform better for summarization, while the input OOD scores perform better for translation. One possible reason is that when summariza-
5

---

Published as a conference paper at ICLR 2023
tion outputs are low-quality (e.g. producing repeated text or irrelevant summaries) they look very different than reference summaries, making OOD output score more sensitive to the contrast.
Though RMD and Binary logits OOD scores both perform well at OOD detection, RMD OOD score better at distinguishing near-OOD from far-OOD. This can be seen in Figure where near-OOD datasets have scores distributed in between in-domain and far-OOD. In the summariza- tion task, mear-OOD (news articles) datasets emn een_dailllee minddilymaill and newsroom have their RMD scores distributed in the middle of xsum and aaddit_tifu, forumsum and samsuu. In contrast, under the binary logits score, the near-OOD and far-OOD datasets have largely overlapping score distributions making hard to distinguish between the two. practice, RMD OOD score may be better suited for selective generation where domain shifts are expected. We explore this in more detail in Section 4.
648
nowsroom all reddst tims
Deasity
Deasityy
RMD Outgat DOD Score
Bibary Logits Outpat DOD Score
Figure 3: Density of RMD (left) and Binary logits (right) OOD scores evaluated on summarization datasets. RMD better at distinguishing near-OOD from far-OOD.
For the translation task, we additionally note that all methods have small AUROC for law dataset, suggesting that none of the methods are detecting the dataset as OOD. To better understand the special characteristics of the law dataset, we conducted an n-gram overlap analysis between the various test sets including law and the in-domain training data. We observed that law has highest unigram overlap rate (48.8%) and the second highest overall overlap with the in-domain data (Table A.9). This shows that law is close to in-domain data in terms of surface features, which might contribute the low AUROC scores for all tested methods.
We use ParaCrawl instead of C4 for translation because our translation model is trained on the sentence level, unlike the summarization model that takes the document input. To further explore the effect of the background data on the performance, we split C4 documents into sentences and use that the background data to compute the scores. The OOD detection performance using C4 sentences is very similar to that using ParaCrawl, as shown in Table A.3, suggesting that our method is not particularly sensitive the choice of background data.
4 USING OOD SCORES FOR SELECTIVE GENERATION
The most conservative option for deployment of a conditional language model is to completely abstain from generating on inputs that are detected as out-of-distribution, for which we have shown in Section 3 our OOD scores are fairly accurate. However, it is often desirable expand the use of models beyond strictly in-distribution examples, the quality of outputs sufficiently high. In classification, this has been framed as determining when to trust classifier, or selective prediction (Geifman & EI- Yaniv, 2017; Lakshminarayanan et al., 2017; Tran et al., 2022). In this section, we seek predict the quality of generation given an example, which may be out-of-distribution and abstain if the predicted quality is low. We call this selective generation. In practice, abstaining may correspond to hiding the model's generated text, turning off summarizatioo//rasslttio feature.
4.1 EXPERIMENT SETUP
We use the same models and datasets described Section 3.1 but instead of simply detecting out- of-distribution examples, focus now is to predict the quality of generation for examples possibly outside the training distribution.
'e define overlap rate as the percentage of unique n-grams in the test set that are also present in the in- domain data. The overall overlap is defined the geometric mean of all the n-gram overlap rates up 4. All domains/splits including the in-domain data subsampled 1K for this analysis.

---

Published as a conference paper at ICLR 2023
Measuring Translation quality We use BLEURT (Pu et al., 2021) the main metric to mea- sure translation quality. Previous work has demonstrated that neural metrics such as BLEURT are much better correlated with human evaluation, on both the system level and the sentence level (Fre- itag et al., 2021). BLEURT scores range from 0 to 1, with higher scores indicating better quality.
Measuring Summarization quality In general, it is unclear how to automatically measure the quality of summaries generated by a model on -iito---istrrbution examples (in this case, exam- ples from different datasets). The reason summarization datasets have dataset-specifie summary styles that may be difficult to compare. For example, xsum summaries are typically single-sentence whereas maidlmaya summaries consist of multiple sentences. Thus we report ROUGE-1 I score as an automatic measure but primarily use human evaluation to assess the quality. Amazon Me- chanical Turk workers were asked to evaluate summaries generated by the xsum model on a scale of 1-5 (bad-good) using 100 examples from xsum, enn_dailymail, reddit__tiu, and samsum. We collected ratings per example and computed the median. See Section A.3 for more details.
4.2 PERPLEXITY HAS DIMINISHING CAPABILITY IN PREDICTING QUALITY ON OOD DATA
Since the models are trained using negative log-likelihoon as the loss, perplexity (which is mono- tonically related) is a good predictor of output quality for in-domain data. In fact, the Kendall rank correlation coefficient: between perplexity and human judged quality score is 0.256 (See Table 2) for in-domain xsum for summarization. However, when including shifted datasets to test, we found that the perplexity score is worse at predicting quality on OOD data. For example the Kendall's de- creases to 0.068 for OOD dataset samsum (see Table A.4). We observed similar trend in translation, although less severe, as data shifted from in-domain OOD, the Kendall's between perplexity and BLEURT decreases (see Table A.5). Figure 4 further shows the correlation between perplexity and the quality score (ROUGE-I, human rating, and BLEURT, respectively) as function of OOD score. It is clear to see the correlation decreasing as OOD score increases and the trend is consistent for both summarization and translation.
(a) Summarization, ROUGE-1 (b) Summarization, human rating
(c) Translation, BLEURT
0.48
0.53
FDc
vos rigettiant
20
Baz
K
0.25
ganiges
0.0
018
01.33
200
O01 scone (output)
2/06
4,000
1,500 OOD scure (sutput)
adon
5.
-06
OOD spore (mpul)
Figure 4: The Kendall's 7 correlation between perplexity and (a) ROUGE-1, (b) human evaluation median rating, and (c) BLEURT decreases as OOD score increases respectively. Note that we use output RMD OOD score for summarization and input RMD OOD score for translation.
4.3 COMBINING OOD SCORES AND PERPLEXITT
While model perplexity for quality estimation is worse for OOD examples, we observed that our OOD scores and perplexity are complementary in quality prediction. Figure A. shows a 2-D plot between the OOD score and perplexity regarding quality. We can see that neither perplexity nor OOD score can perfectly separate good and bad examples, and the combination of the two can work much better. Our observation echos work in uncertainty estimation in classification models (Mukhoti et al., 2021): perplexity based on softmax predictive distribution is regarded as estimation for aleatoric uncertainty (caused by inherent noise or ambiguity in data), and the OOD distance based on representation estimates the epistemic uncertainty (caused by lack of training data), and combining the two provides a comprehensive estimation of uncertainty.
We propose two simple methods to combine perplexity and OOD scores. (1) A simple linear re- gression, trained on a random 10% data split using ROUGE-I or BLEURT as the quality score, and evaluated on the test split and human evaluation split. scores, i.c. PR = PRaaplexity + PROOD- We sum PRs instead of their raw values because the two
(2) the sum of the percentile ranks (PR) of the
scores are in different ranges, PR (2: R(x) x 100, where R(x) is x's rank in the list of size N.
Table 2 shows the Kendall's 7 correlation coefficient between the various single and combined scores and the quality metric with only in-domain and all examples from all datasets. When all datasets
7

---

Published as a conference paper at ICLR 2023
Table 2: Kendall's r correlation (p-value 0.05 are grayed out) between various measures and human evaluation for summarization and BLEURT for translation. The "All" column shows the correlation when both in-domain and OOD examples are merged. Note for negatively correlated scores (c.g. perplexity (ppx), RMD), we take the negative value of the score for casier comparison.
(a) Summarization
(b) Translation
Measure
In-domain
All
Measure
In-domain
Single Score
Single Score
Perplexity (baseline)
0.309
0.286
Perplexity (baseline)
0.256
0.300
COMET (buseline)
0.184
0.336
NI.1 score (haseline)
0.337
0.381
Prism (baseline)
0.184
0.301
Input RMD
0.015
0.336
Inpuu RMD
0. 147
0.195
Output RMD
0,053
0.385
Output RMD
0.086
0.170
Combined Soore
Combined Score
PR (pp,, input RMD) (pxx, output RMD)
0.186
0.358
(pun, input RMD)
0.321
0.361
0.250
0.415
PR (Pu, outpot RMD)
0.323
0.356
Lincar Reg. (ppx, input & output)
0.235
0.422
Linear Reg. (pps, imput & outpul)
0.318
0.352
are merged, the combined scores significantly improve the correlation over perplexity by up to 12% (absolute) for summarization and 8% for translation, while the gains over the best exter- nal model-based (and much more expensive) baselines are 4% and 3%. The two combination methods perform similarly. See Tables A.4 and A.5 for an expanded table of scores.
4.4 SELECTIVE GENERATION USING THE COMBINED SCORE
In selective generation, our goal is to generate when the model is more likely to produce high- quality output, and abstain otherwise, enabling safer deployment of generative language models. To evaluate that, we propose using the Quality vs Abstention Curve (QA), analogous to accuracy versus rejection curve used for selective prediction in the classification (Chow, 1957; Bartlett & Wegkamp, 2008; Geifman & EI- Yaniv, 2017). Similar concepts were proposed also in Malinin & Gales (2020); Xiao et al. (2020), but they only use automatic quality metrics for the analysis while we consider human evaluation to assess the quality as well. Specifically, at a given abstention rate a, the highest a-fraction scoring examples are removed and the average quality of remaining examples is computed. We want to maximize the quality of what is selectively generated and a better curve is one that tends to the upper-left which corresponds to removing bad examples earlier than good ones.
Figure 5 shows the QA curves for various methods on summarization and translation. Quality is measured by human evaluation for summarization (see Figure A.4 for similar ROUGE-1 plot), and BLEURT for translation. The combined scores have the highest quality score at almost all ab- stention rates for both summarization and translation, while linear regression and PR perform similarly. For single scores, the OOD score performs better than perplexity and NLI scores at almost all abstention rates for summarization. For translation, the OOD score better than perplexity when abstention rate > 0.65 and worse than perplexity when < 0.65. In other words, OOD score better at abstaining slightly far-OOD while perplexity is better at abstaining near-OOD examples. Interestingly, our combined score is even marginally better than COMET that requires a separate neural network trained on human evaluation data. Prism is better than single scores, but much worse than our combined score. Area under the QA curves are shown in Tables A.6 and A.8 for reference.
Figures 5 (b, d) are the corresponding survival curves showing how many examples per dataset are selected for generation a function of abstention rate, based on the PR score. For summarization, the samples from far-OOD datasets reddit tifu and samsum are eliminated first with their sample count decreasing rapidly. The near-OOD dataset enn dailymail and in-domain xsum kept intact until a > 0.3, and in-domain xsum examples survive the longest. Similarly for translation, the oft-of-domain and worst-quality (as seen Table AbI Koran, MTNT, and subtitles examples are eliminated first, and the best-performing law and in-domain datasets are abstained last. The order in which datasets are eliminated corresponds to the aggregate quality by dataset, which we report in Table A.1. Besides the quantitative results, we show few real examples in Section A.14 to better demonstrate how our predicted quality score helps selective generation.
5 RELATED WORK
OOD detection problem was first proposed and studied in vision classification problems (Hendrycks & Gimpel, 2016; Liang et al., 2017: Lakshminarayanan et al., 2017: Lee et al., 2018; Hendrycks et al., 2018; 2019), and later in text classification problems such sentiment analysis (Hendrycks
8

---

Published as a conference paper at ICLR 2023
(b)
75
Terpisaly Iaan
Hummn Eval Median
0.73
[hear PB
Sunvvai Count
ETR dalymal
Abstention Rate
Abstention Rate
(d)
3.600
Per
2375
COMET iBank
Proo (hone)
Survivel Count
820
frgut IND
Livear Seg
0723
8
420
n212015
4615
Karan
3.50
0.6
22
64
Rabe
0.6
Abstention
38
Figure 5: (a) The Quality (human eval) vs Abstention curve for summarization. Combined scores have the highest quality at almost all abstention rates. (b) Survival count of each dataset as a function of abstention rate, using PR (we use output/input RMD for summarization/ranslatooo perplexity). OOD data is abstained earlier than in-domain. (c, d) The same as (a, b) for translation.
to pair with
al., 2020), natural language inference (Arora et al., 2021), intent prediction (Liu al., 2020a; Tran et al., 2022), and topic prediction (Rawat et al., 2021). The widely used OOD methods can be characterized roughly into two categories (1) softmax probability or logits- baste- scores (Hendrycks & Gimpel, 2016; Liang et al., 2017: Hendrycks et al., 2019; Liu et al.. 2020b), (2) embedding- -based methods that measure the distance to the training distribution in the embedding space (Lee et al., 2018; Ren et al., 2021; Sun et al., 2022), (3) contrastive learning based methods which incorporate the contrastive loss into the classification cross-entropy loss to improve representation learning and consequently improve OOD detection (Winkens et al., 2020; Zhou et al., 2021). Though it is not straightforward to extend those classifier scores to CLMs especially for input OOD detection, extend three of them based on understanding as baselines for comparison with our methods. See Section A.6 for details. The results in Table A.2 show that those methods are in general not competitive with our proposed methods RMD and Binary logits, especially on near-OOD) datasets.
OOD detection problem is less studied in CLMs. A few studies explored OOD detection in semantic parsing (Lukovnikov et al., 2021: Lin et al., 2022), speech recognition (Malinin & Gales, 2020), and machine translation (Malinin et al., 2021; Xiao et al., 2020), but many of them focus on ensemble- based methods like Monte Carlo dropout deep ensemble which use the averaged perplexity after sampling multiple output sequences. The ensembling method costs N times of the inference time, which is not feasible in practice. In this work, we focus on developing scores that can be readily derived from the generative model itself, without much increase in computation. We include an ensemble-basee baseline in Section A.6 and show that its performance is worse than our methods.
6 CONCLUSION AND FUTURE WORK
We have proposed lightweight and accurate scores to detect nat-of-distribution examples for con- ditional language generation tasks. For real-world deployment, we have also shown how our OOD scores can be combined with language model perplexity selectively generate high-quality outputs while abstaining from low-quality ones in the setting of input distribution shift.
Although our experiments focus on summarization and translation, our methods do not make any assumptions about the task modality, and we believe our method is widely applicable to other tasks where the model output is a sequence, c.g. image captioning. While our analysis was restricted to conditional language modeling with encoder-decoder Transformers, we expect our method to also work with oecoder-only (Liu et al., 2018) architectures, used by some large language models such as GPT-3 (Brown et al., 2020), PaLM (Chowdhery et al., 2022), and LaMDA (Thoppilan et al., 2022).
Finally, analyzing why certain examples are OOD could lead to insights how to make models more robust. Section 13 presents one possible way to attribute OOD scores to sentences.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGEMENTS
The authors would like to thank Jeremiah Zhe Liu, Sharat Chikkerur, and the anonymous reviewers for their helpful feedback on the manuscript. The authors would also like to thank Colin Cherry, George Foster, and Polina Zablotskaia for their feedback throughout the project.
REFERENCES
Udit Arora, William Huang, and He He. Types of arXiv preprint arXhv:2109.0687,, 2021.
texts and how to detect them.
Mikko Aulamo and Jörg Tiedemann. The OPUS resource repository: An open package for creat- ing parallel corpora and machine translation services. In Proceedings of the 22nd Nordic Con- ference on Computational Linguistics, pp. 389-394, Turku, Finland, September- 2019. Linköping University Electronic Press. URL https///ccanthologg.oryyy 11666666.6.66666...
Marta Bañón, Pinzhen Chen, Barry Haddow, Kenneth Heafield, Hieu Hoang, Miquel Esplà-Gomil Mikel L. Forcada, Amir Kamran, Faheem Kirefu, Philipp Koehn, Sergio Ortiz Rojas, Leopoldo Pla Sempere, Gema Ramírez-Sánchez, Elsa Sarrias, Marek Strelec, Brian Thompson, William Waites, Dion Wiggins, and Jaume Zaragoza. ParaCrawl: Web-scale acquisition of parallel cor- pora. In Proceedings of the 58th Annual Meeting of the Association for Computational Lin- guistics, pp. 4555-4567, Online, July 2020, Association for Computational Linguistics. doi: 3/065//112020.acl-mmm.
URL httpss//cckanhollgy.ooo wrwp:ogg..orr//// ott/20200.accooooooooo acl- 417-
Peter I. Bartlett and Marten H Wegkamp. Classification with a reject option using hinge loss. Journal of Machine Learning Research, 98), 2008
Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Barry Haddow, Matthias Huck, Chris Hokamp, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Car- olina Scarton, Lucia Specia, and Marco Turchi. Findings of the 2015 workshop statistical machine translation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pp. 1-46, Lisbon, Portugal, September 2015. Association for Computational Linguistics. doi: URL https://ccammtbooo..rr/W......
10.18653/1//115-300
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Tlerbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 1877-1901. Curran Associates, Inc., 2020. URL https://moceedigg..uunissscccec Papeee..-......f....
Saikiran Bulusu, Bhavya Kailkhura, Bo Li, Pramod Varshney, and Dawn Song. Anomalous example detection in deep learning: A survey. IEEE Access, 11133330-132444 2020.
Chi-Keung Chow. An optimum character recognition system using decision functions. IRE Trans- actions Electronic Computers, (4):247-254, 1957.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay. Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard. Guy Gur-Ari, Pengeheng Yin, Toju Duke, Anselm Lev- skaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito. David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowyez, Erica Moreira, Rewon Child, Oleksandr Polozov. Katherine Lee, Zongwei Zhou, Xuezhi Wang, Bren- nan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. Palm: Scaling language modeling with pathways, 2022. URL https:/arrii..ng/as///000000
Douglas
10

---

Published as a conference paper at ICLR 2023
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. Un- supervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 8440-8451, Online, July 2020. Association for Computational Linguistics. doi: 10.1665/11/2000.acmmmim.. 747. URL https:
Markus Freitag, Ricardo Rei, Nitika Mathur, Chi-kiu Lo, Craig Stewart, George Foster, Alon Lavie, and Ondiej Bojar. Results of the WMT21 metries shared task: Evaluating metries with expert- based human evaluations on TED and news domain. In Proceedings of the Sixth Conference on Machine Translation, pp. 733-774, Online, November 2021. Association for Computational Linguisties. URL https///acaanhoooogy.oro////
1.73.
Yonatan Geifman and Ran El-Yaniv. Selective classification for deep neural networks, Advances in neural information processing systems, 30, 2017.
Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer. SAMSum corpus: A human- annotated dialogue dataset for abstractive summarization. In Proceedings of the 2nd Workshop on New Frontiers in Summarization, pp. 70-79, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.1 1//39///// URL httph//gs//caatthooggoor/00//1/
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6577 2014.
Ben Goodrich, Vinay Rao, Peter J. Liu, and Mohammad Saleh. Assessing the factual accuracy of generated text. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 119, pp. 166-175, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781 9781450362016.
doi: 10.1145/3295500333995 URL https:
Max Grusky, Mor Naaman, and Yoav Artzi Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies. Proceedings of the 2018 Conference of the North American Chap- of the Association for Computational Linguisties: Human Language Technologies, Volume
(Long Pupers), 2018. doi: 118653/v1/n188 1065.. URL http::/dxx.doi.orgg.. 100 1ee1111.8666//0 18653/11/18--0666
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.021.66 2016.
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. arXiv preprint arXiv: 0088/.206666 2018.
Dan Hendrycks, Steven Basart, Mantas Mazeika, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Song. Scaling out-of-distribution detection for real-world settings. arXiv preprint arXiv:1911.11132. 2019.
Dan Hendrycks, Xiaoyuan Liu, Eric Wallace, Adam Dziedzic, Rishabh Krishnan, and Dawn Song. Pretrained transformers improve catto--distribution robustness. arXiv preprint arXiv:2004.06100 2020.
Karl Moritz Hermann, Tomás Kociský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching machines to read and comprehend. In NIPS, pp. 1693- 1701, 2015. URL http://paapes.nssccccppppeee
httpapppr///--seechhiinnnneeee wwweeee-aacc-----eeeeeee to-aees-tt-rd-d---e cod-compre
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. In International Conference on Learning Representations, 2020. URL https:
Or Honovich, Roee Aharoni, Jonathan Herzig, Hagai Taitelbaum, Doron Kukliansy, Vered Cohen, Thomas Scialom, Idan Szpektor, Avinatan Hassidim, and Yossi Matias. TRUE: Re-evaluating factual consistency evaluation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp- Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/v1/2000 naw.l/nnaacammnn main. 287. URL http://///eaaattoooy.oo/22222.2 naacl- main.287.

---

Published as a conference paper at ICLR 2023
Misha Khalman, Yao Zhao, and Mohammad Saleh. Forumsum: A multi-speakker conversation sum- marization dataset. In Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 4592-4599 2021.
Byeongchang Kim, Hyunwoo Kim, and Gunhee Kim. Abstractive summarization of reddit posts with multi-level memory networks, 2018.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems, 30, 2017.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distributiin samples and adversarial attacks. NeurIPS, 2018.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: Denoising sequence-to-sequeee pre- training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7871 7871-7880, On- line, July 2020. Association for Computational Linguistics. doi: 10. URL https://ccaamhooog..or//222 wclp:///g....o//..aaa.alaoooc acl-main.703.
Shiyu Liang, Yixuan Li, and R Srikant. Enhancing the reliability of out-of-distribution image detec- tion in neural networks. arXiv preprint arXiv: 706.02690, 2017.
Zi Lin, Jeremiah Zhe Liu, and Jingbo Shang. Towards collaborative neural-symbolie graph semantic parsing via uncertainty. In Findings of the Association for Computational Linguistics: ACL 2022, pp. 1160-4173. 2022.
Jeremiah Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax Weiss, and Balaji Lakshmi- narayanan. Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. Advances Neural Information Processing Systems, 011774987712 2020a.
Peter J. Liu, Mohammad Saleh, Etienne Pot, Ben Goodrich, Ryan Sepassi, Lukasz Kaiser, and Noam Shazeer. Generating wikipedia by summarizing long sequences. In International Conference on Learning Representations, 2018. URL http:://ppeueevvwwwwew///mmmotlt
Weitang Liu, Xiaoyun Wang. John Owens, and Yixuan Li. Energy-based out-of distibbbbnt detec- tion. Advances in Neural Information Processing Systems, 33:21464-214777 2020b.
Denis Lukovnikov, Sina Daubener, and Asja Fischer. Detecting compositionally out-of-distribution examples in semantic parsing. In Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 591-598, 2021.
Andrey Malinin and Mark Gales. Uncertainty estimation in autoregressive structured prediction. arXiv preprint arXiv:2002.07650 2020.
Andrey Malinin, Neil Band, German Chesnokov, Yarin Gal, Mark JF Gales, Alexey Noskov, Andrey Ploskonosov, Liudmila Prokhorenkova, Ivan Provilkov, Vatsal Raina, et al. Shifts: A dataset of real distributional shift across multiple large-scale tasks. arXiv preprint arXiv:2107.07455, 2021.
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. On faithfulness and factuality in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1906-19199 1919, Online, July 2020. Association for Computational waeelmmain.111. 173. URL https//hccamthhooggroo0200000000 aallmmmin..7..
Linguisties. doi: 10. 00.18853//1/2020
Paul Michel and Graham Neubig. MTNT: testbed for machine translation of noisy text. In Pro- ceedings the 2018 Conference on Empirical Methods in Natural Language Processing, pp- 543-553, Brussels, Belgium, October-November 2018. Association for Computational Linguis- tics. doi: 10.18653/11/111-10000 URL http:://cccaattoollyyo/0000---00
Jishnu Mukhoti, Andreas Kirsch, Joost van Amersfoort, Philip HS Torr, and Yarin Gal. Deep deter- ministic uncertainty: A simple baseline. arXiv e-prints, pp. arXiv-2102, 2021.
12

---

h/tp://..1111111p1-e... Published as a conference paper at ICLR 2023
Kevin P. Murphy. probml.ai.
Probabilistic Machine Learning: Advanced Topics. MIT Press, 2023. URL
Vaishnavh Nagarajan, Anders Andreassen, and Behnam Neyshabur. Understanding the failure modes of dut-of-distiibution generalization. arXiv preprint arXiv:2010.15775, 2020.
Shashi Narayan, Shay B. Cohen, and Mirella Lapata. Don't give me the details, just the sum- mary! topic-aware convolutional neural networks for extreme summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 1797-1807, 2018. Association for Computational Linguistics. doi:
Brussels, Belgium, 10.18653vv//D18 1206. URL http://aebamhhooog.or//111 1216.
A Nguyen, J Yosinski, and J Clune. Deep neural networks are easily fooled: high confidence predictions for unrecognizable images. arxiv. arXiv preprint 4rX1v:1412.1897, 2014.
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. Advances in neural information processing systems, 32, 2019.
Amy Pu, Hyung Won Chung. Ankur Parikh, Sebastian Gehrmann, and Thibault Sellam. Learning compact metrics for MT. In Proceedings of the 202/ Conference on Empirical Methods in Natural Language Processing, pp. 751-762, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: hutp://aclantooloy..rrr/// wwt.:////////eennll-o... www.eeemmnninnS--... 58in.58.
URL
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with unified text- to-text transformer. Journal of Machine Learning Research, 21(140):1667, 2020. URL hitp:
Mrinal Rawat, Ramya Hebbalaguppe, and Lovekesh Vig. Pnpood: Out-of-distribution detection for text classification via plug andplay data augmentation. arXiv preprint arXiv:2111.00500 2021.
Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. Comet: A neural framework for mt evaluation. arXiv preprint wxXi:20099000025 2020.
Jie Ren, Peter J Liu, Emily Fertig, Jasper Snoek, Ryan Poplin. Mark A DePristo, Joshua V Dillon, and Balaji Lakshminarayanan. Likelihood ratios for outto--distribution detection. NeurIPS, 2019.
Jie Ren, Stanislav Fort, Jeremiah Liu, Abhijit Guha Roy, Shreyas Padhy, and Balaji Lakshmi- narayanan. A simple fix to mahalanobis distance for improving near-ood detection. arXiv preprint arXiv:2106.09022. 2021.
Lukas Ruff, Jacob R Kauffmann, Robert A Vandermeulen, Grégoire Montavon, Wojciech Samek, Marius Kloft, Thomas G Dietterich, and Robert Müller. A unifying review of deep and shallow anomaly detection. Proceedings of the IEEE, 109(5):756-795, 2021.
Mohammadreza Salehi, Hossein Mirzaei, Dan Hendrycks, Yixuan Li, Mohammad Hossein Ro- hban, and Mohammad Sabokrou. A unified survey on anomaly, novelty, open-set, and out-of- distribution detection: Solutions and future challenges. arXiv preprint arXiv:2110.14051, 2021.
Abigail See, Peter J. Liu, and Christopher D. Manning. Get the point: Summarization with wointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), pp. 1073-1083, Vancouver, Canada, July 2017. Association for Computational Linguistics. doi: 10. 10.18653/v//P17-1099 URL.
http:///ccamthoooy..m./P11..00
Noam Shazeer and Mitchell Stern. Adafactor: Adaptive learning rates with sublinear memory cost.
In International Conference on Machine Learning, pp. PMLR, 2018.
13

---

Published as a conference paper at ICLR 2023
Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. arXiv preprint arXiv:2244.06507 2022.
Brian Thompson and Matt Post. Automatic machine translation evaluation in many languages via zero-shot paraphrasing. arXiv preprint arXiv:2004, 14564, 2020.
Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tie Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201 0022999........... 2022.
Jörg Tiedemann. Parallel data, tools and interfaces in OPUS. In Proceedings of the Eighth In- ternational Conference Language Resources and Evaluation (LREC" pp. 2214-2218, Istanbul, Turkey, May 2012. European Language Resources Association (ELRA). URL htp::
Dustin Tran, Jeremiah Liu, Michael W Dusenberry, Du Phan, Mark Collier, Jie Ren, Kehang Han, Zi Wang, Zelda Mariet, Huiyi Hu, et al. Plex: Towards reliability using pretrained large model extensions. arXiv preprint arXiv:2207.074111 0, 2022.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural informa- tion processing systems, 30, 2017.
Jim Winkens, Rudy Bunel, Abhijit Guha Roy, Robert Stanforth, Vivek Natarajan, Joseph R Ledsam, Patricia MacWillims, Pushmeet Kohli, Alan Karthikesalingam, Simon Kohl, et al. Contrastive training for improved out-of-distribution detection. arXiv preprint arXiv:2007.05566. 2020.
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin John- son, Xiaobing Liu, Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang. Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, and Jeffrey Dean. Google's neural machine translation system: Bridging the gap between human and machine translation. CoRR, abs/1609.00 01199.88144 44, 2016a. URL http://arxiv..rg/aas// 609.088144
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. Google's neural machine trans- lation system: Bridging the gap between human and machine translation. arXiv preprint 1rXiv:1609.08144 2016b.
Tim Z Xiao, Aidan N Gomez, and Yarin Gal. Wat zei je? detecting dustt--disttibution translations with variational transformers. arXiv preprint arXiv:2006. 08344, 2020.
Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J. Liu. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. In Proceedings of the 37th International Conference on Machine Learning, ICML'20 200. JMLR.org. 2020.
Wenxuan Zhou, Fangyu Liu, and Mubao Chen. Contrastive out- -ittt--diittibttion detection for pre- trained transformers, arXiv preprint XrXi::2004.08812, 2021.
14

---

Published as a conference paper at ICLR 2023
A APPENDIX
A.1 THE OUTPUT QUALITY FOR SUMMARIZATION AND TRANSLATION DATASETS.
Table A.I: The output quality for summarization and translation datasets. (a) Summarization quality (higher is better) for xsum model. ROUGE-1 is based on all samples in the test split per dataset, while human evaluation is based 100 samples. The raw human evaluation rating ranges from to We normalized the score by dividing 5.0, and toke the median of the ratings over raters to reduce inter-rater noise. The standard deviation among ratings reported in brackets. (b) Translation quality for different datasets (higher is better). All datasets are sub-sampled to 1000 sentence pairs.
(a) Summarization
Datasct
ROUGE-1
Human evaluation
Xsum
0.698 (0. 182)
con.dailymail
0,226
0.624 (0. 145)
reddit_tifu
0.140
0,450 (0.152)
0.210
0.376 (0. (0.147)
(b) Translation
Dataset
BLEURT
BLEU
law
0.781
53.8
holdout
0.674
41.8
ndt2015
0.671
37.9
ndd2015
0.664
30.9
medical
0.643
34.2
FT
MTNT
0.565
32.0
Koran
0.491
12.9
A.2 OOD SCORE AND PERPLEXITY ARE COMPLEMENTARY FOR PREDICTING OUTPUT
QUALITY.
(a) Summarization, ROUGE-1
(b) Summarization, human rating
(c) Translation, BLEURT
nouge1
Percentle rank pespertty
human roting
uc
bleurt
Peronntle rank of parplesty
Percentle ank of perpledity leeliy
B
ht
3
8
count
.00
V 8
nb
D5
88
5
8
8
Porcentle rank of output RWD
20
output
70 80
Percontie
30 4:
10 10 120
Figure A.I: 2D plot between OOD and perplexity. The two scores are self-normalized by its per- centile rank respectively. Each square corresponds subset of samples whose OOD and perplexity scores are within the percentile bin. The size of the square represents the size of the bin where the color indicates the quality of the model's output. The OOD score and perplexity capture different properties of model outputs, and combining both scores can be beneficial for quality prediction.
15

---

Published as a conference paper at ICLR 2023
A.3 AMAZON MECHANICAL TURK ASSESSMENT OF SUMMARY QUALITY
PEGASUSLARGE model fine-tuned on xsum was run on random sample of 100 examples from the test split of four datasets: xsum, enn _dailymail, reduit samsum. Each example rated for general summarization quality on a rating of 1-5 by 3 AMT workers using the template shown in Figure A.2. Workers were required to be Masters located in the US with greater than 95% HIT Approval Rate, with at least 1000 HITs approved and were paid $0.80 per rating.
Read the document below, then rate the summary for quality on scale of 1-5. (1 Poor summary, 5 = Great summary)
When assessing quality consider the informativeness, whether it is falthful the article, and fluency the English quality good, does repeat itself?). Note, some documents and summaries may have adult language having adult language is not enough by itself to consider summary bad.
Document to summarize:
Michael Coe, 35, saw the two 16-year-olds hugging in the street in Newhan, east London, in April and denanded to know if they were Mus lins. Southwark Crown Court heard the Muslin convert then called the girl "whore", before throwing the boy to the ground.
Coe also attacked a passing teacher who had tried to help the couple.
Judge Michael Gledhill OC said the two children had denied they were Muslim when challenged by Coe.
"Why? Because they were trightened of what you would they told you the truth, that they were in fact Mustin," Jude Gledhill said.
added: "At the time of these offences you either held extrenist views or views that were getting very close to extrenist views."
Coe had adnitted "shoving" the boy who half his size claining was acting in self- defence,
convicted August the father in 2007 while serving eight-year for firing shotgun policce during
occasioning actual bodily harm battery. prison by al-Qaeda
Dhiren
Coe was also convicted of religiously aggravated harassment 2013 after seeing Muslim talking to group of and telling her that was against Islan. The defendant, also known Mikaeel Ibrahin, becane close associate of convicted hate preacher Choudary, founder of the banned organisation al-Muhajiroun, of which Coe member.
Prosecutar Jonathan Polnay read victin impact statement from the boy. "He feels the offence has affected his life quite lot," My Polnay said. "He doesn't see his friends outside of school.
"He has also split up with the girl who was his girlfriend at the tine."
Summary:
An associate redical preacher Anjem Choudary who "shoved' a boy who was hugging his girffriend has been jailed for two years.
Rating:
Figure A.2: AMT template for summarization human evaluation.
16

---

Published as a conference paper at ICLR 2023
A.4 ALGORITHM FOR RMD OOD SCORES
Algorithm Fitting Gaussians for input and output embeddings
1: Input: CLM M with encoder fe and decoder fa trained on in-domain train set D
((,y)); A large and background dataset such as C4 or ParaCrawl Du train {(x, ÿ)) where
y = (()).
2: Generate the input embeddings Strain = {z]fe(x)) x E Din Dirain) and Shg = {z)fe(x)), x E
Dbg ).
train
3: Fit Gaussian distribution Mi,² E+) using Srain and a background Gaussian N Eg)
using Shy. traii"
4: Similarly, generate output embeddings Eiain = fwlfa(y). y E Dirain ) and gbg ttrain
(w) fa(ū), E ).
5: Fit a Gaussian distribution ((p"". " using Ethiin and a background Gaussian N(p Eg)
using ebg
Algorithm 2 OOD score inference
Input: In-domain test set Dia {xx 1j)}. OOD test set Dood {xx ()) where ý M(x).
2: Generate input embeddings Siest (x), x Diast and Soed {=lff(()), Deod J.
3: Compute input OOD score RMD mput (z) for z E Sin and Spond respectively. Compute AUROC based on the imput OOD scores.
4: Similarly, generate output embeddings &est
(wlfa()), E Dia Diest and Eood (w (D)) ý E D. Compute output OOD score rurque (w) for w respectively. Compute AUROC based on the output OOD scores.
gin and good
A.5 THE CONNECTION BETWEEN RMD AND BINARY CLASSIFIER
RMD generative model based approach which assumes the distributions of the two classes are Gaussian, while the binary classifier is discriminative model which learns the decision boundary between two classes. Though they have different settings, under certain condition, the Gaussian generative model can be reduced to binary classifier. To see the connection, let us assume the label y = 0 if the sample from in-domain, and y Let us also assume the two classes have balanced sample size without loss of generality p(y 1) = ply = 0). Since the log-proband oog-robbability of log p(y = 1|z) can be rewritten using the Bayes rule log p(y 1 1[2 = log p( zly 1) + log 1) log 2), the logit (log odds) be written as,
the sample from the general domain.
p(y 1|
logit log
log 1|z) log p(y 0[2)
p(y 0|z)
bogp(2zy = 1) - log p(2y 0)
(MD(>; :: My=1, Ey=1) - MD(;; My=0 Eyy=)) + const.
When Σ = Eyy= oy the equation can be further simplified as
logit ( ²²=0)²² =B12 3n-
(uT N- My=1 2 -- uy==)) + const.
Therefore, when assuming the covariance matrices are identical for the two Gaussian distributions, the Gaussian generative model can be reduced to a binary classification model. However, our RMD does not assume the same covariance matrix in both distributions. We estimate the covariance matrix individually for each class. So our RMD is different from binary classifier, and has higher model capacity than the binary classifier.
17

---

Published as a conference paper at ICLR 2023
Table A.2: AUROCS for OOD detection for comparing our proposed method with more baseline
Near Shift OOD
Far Shift OOD
Measure
cmn dailymail
newsroom
reddit tifu
forumsum
INPUT OOD
KNN rorm 100%,, k=10000)
0.887
0.743
0.944
0.961
0.955
MD
0.651
0.799
0.974
0.977
0.995
RMD
0,930
0.998
0.999
Binary logits
0.997
0.959
1,000
0.999
0.998
OUTPUT OOD
NLJ score
0.440
0:469
0.709
0.638
0.743
Perplexity
0.665
0.909
0.851
Meun(MSP)
0.343
0.616
0.877
0.715
0.826
Energy score
0.460
0.960
0.981
Ensemble using MC dropout (N==)
0,496
0.768
0.970
0.937
0.944
Ensemble using MC dropout (N=10)
0.497
0.774
0.976
0.947
0.956
KNN (cem 100%,, k=1000)
0.860
0.791
0.948
0.926
0.968
MD
0.944
0.933
0.985
0.973
0.985
RMD
0.958
0.962
0.998
0.998
Binary logits
0.989
0.982
1.000
0.998
0.997
A.6 COMPARISON WITH MORE BASELINE METHODS
As we discussed in the related works, OOD detection problem was mainly studied in classification problems, and less studied in CLMs. Though it is not straight forward to extend classifier based scores to CLMs especially for the input OOD detection, would like to include many possible methods as we can to present a comprehensive comparison for different methods.
For those methods which rely classification head derived logits, MSP (Hendrycks & Gimpel, 2016), max-logit (Hendrycks et al., 2019), and energy score (Liu et al., 2020b), we simply consider the output decoding process a sequence of classifications over tokens, and take the average of the corresponding score over the generated output tokens ,,.....y/r as the output OOD scores. Therefore we added the following scores for CLMs,
Mean(MSP) - - p(y x)).
Energy score 1 ET= E(x, fi), where ft) log Evey vly-e, æ) is the logit corresponding to the v-th token at the 1-th decoding step, V is the token-vocabulary, and T is the temperature parameter. We set T = since the original paper (Liu et al., 2020b) suggested the energy score can be used parameter-fre free by simply setting 7 =
f
Ensemble estimation of the output perplexity from multiple Monte-Carlo dropout samples. Malinin & Gales (2020); Xiao et al. (2020) propose to turn on the MC dropout layer at the inference time and sample multiple times (N) using different random seeds as way to approximate the Bayesian neural networks. We follow their idea and generate multiple output sequences and use the averaged perplexity as the uncertainty score. Note that the inference time for ensemble based method is N times of that for the single model based score.
KNN-based OOD score. Sun al. (2022) propose to use the distance to the k-th nearest neighbour in the training set in the embedding space as an OOD score. There are two ayper-parametent in the KNN-based method, a and k. is the proportion of training data sampled for nearest neighbor calculation, and k refers to the k-th nearest neighbor. We use the optimal = 1000 and a 100 as suggested by the paper. We also normalize the embedding features since the paper showed the feature normalization is critical for good performance.
Mcan(MSP), energy score, and ensembled perplexity score, are all derived from the logits of the tokens in output sequences, so they are output OOD scores. The KNN-based method can be applied for both input sequence embeddings and output sequence embeddings.
Table A.2 shows the AUROCs for OOD detection for the above newly added baselines. as a com- parison to methods. First, the logits based output OOD scores, perplexity, mcan(MSP), energy score, even the ensembled perplexity score which costs N times of the inference time, are in gen- eral not competitive with our proposed method RMD and Binary logits. Though the energy score
18

---

Published as a conference paper at ICLR 2023
Table A.3: Comparison of the OOD detection performance using two different background data, ParaCrawl and C4 sentence.
WMT
OPUS
MTNT
nt
mdd2015
ndt2015
modical
Koran
INPUT OOD
RMD (ParaCrawl) RMD sent)
0.798
0.866
0,863
0.389
0.840
0.957
0.959
0.969
0.943
0.954
0.924
0.985
Binary logits (ParaCraw) Binary logits (C4 seat)
0.904
0.904
0.485
0.813
0.963
0.928
0,950
0.848
0.916
0.916
0.285
0.808
0.944
0.918
0.987
0.976
OUTPUT OOD
RMD (Para Crawl) RMD (C4 sent)
0.786
0,858
0,861
0.355
0.845
0.939
0.951
0.959
0.922
0.898
0.953
0.947
Binury logits (Para Binary logits (C4 sent)
0.822
0.860
0.865
0.507
0.783
0.942
0.890
0.910
0.931
0.853
0.925
0.919
0.294
0.809
0.964
0.901
0.981
0.975
BASELINES
Input MD
0.534
0.671
0,670
0.511
0.704
0.737
018228
0.900
0.668
Output MD
0.733
0.739
0.482
0.838
0.900
0.935
Perplexity COMET
0,570
0.496
0.494
0.392
0,363
0.657
0.343
0.359
0.633
0.484
0.514
0.525
0.435
0.543
0.632
0.619
0.518
0.724
Prism
0.445
0,504
0,505
0.459
0.716
0.604
0.577
a bit better than perplexity and mcan(MSP). and ensembled score is better than energy score, the performance gap between those methods and our proposed method is still big, especially for the near-OOD datasets. Second, KNN-based methods are not as good as MD and RMD either. Though it is possible that the optimal hypeer-prramtte, suggested by the paper may not be the optimal ones for our problem, searching for the optimal hyper-paramette requires separate validation set. In contrast, our proposed methods have no hyperparameters.
A.7 EFFECT OF THE CHOICE OF THE BACKGROUND DATASET
Our principle for choosing the background data is to make it as general as possible. For summariza- tion we use the C4 dataset, which contains large amount of web crawl documents, to represent broad range of topics. Similarly for translation, we use ParaCrawl dataset, which is also large web crawl of sentences, because translation model is sentence to sentence model, unlike the summa- rization model that takes the document as the input. To further explore the effect of the background data the performance, to compute the scores, and compare that with the version using ParaCraw] dataset. The OOD de- tection performance using C4 sentences is very similar that using ParaCrawl, shown in Table A.3. For example, ParaCrawl-base Koran, IT datasets, while C4 based input score slightly better at the other datasets. Both are significantly better than the baseline methods, and both give the same ranking of datasets on their OOD-ness, so our conclusion remains. Those results verify that our method is robust to the choice of background data.
split C4 documents into sentences and use that as the background data
input OOD score has slightly better performance on medial,
A.8 ROC PLOTS FOR THE CORRESPONDING AUROC SCORES FOR OOD DETECTION
To better visualize the OOD detection performance, we present Figure A.3 to show the ROC plots for the corresponding AUROC scores for OOD detection in Table 1. Each of the OOD measures is used for separating the in-domain test data as negative and the OOD test data as positive sets. The AUROC is defined as the area under the ROC curves. The closer an ROC curve is to the upper left corner, the larger the AUROC value is. AUROC 1.0 means perfect separation, and 0.5 means the two are not distinguishable. AUROC is independent of the choice of threshold, so it can be used for fair comparisons among methods.
A.9 CORRELATION BETWEEN DIFFERENT SCORES AND THE QUALITY METRICS
19

---

Published as a conference paper at ICLR 2023
(a) Input MD
(b) Input RMD
(c) Input Binary logits
asty Fagenn mus
Trun bodtt
on Jy mall
on mall
on myill
newporm
noeeroim_al
neweroam_al read uu
red U Ulu
reod Ulu
frunoum
fouusum samsun
To tumssm
samsurs
s0 mouus
Tate
(d) Output MD
(c) Output RMD
(f) Output Binary logits
onn da Iymal
om da tyttall
cnt da tymall
newroosm_ ail
neweroom, .i
neseroom Jill
redit Itfu
reddt IPu
redd Ith)
ferumuu
fesumum
mumumum
sammoum
sarmoum
mouumm
Mastre
Pass
(g) Perplexity
(h) Mcan(MSP)
ii) NLI score
dellyell
nowsroom
tymail
nssoroam,
fesumuu
fosumum
Partive
Partive
Fasss Postrie Sate
Figure A.3: ROC plots for the corresponding AUROC scores in Table for OOD detection in summarization
Table A.4: Kendall's t correlation (p-value < 0.05 are greyed out) between various measures with human-judged quality of a PEGASUS xsum model decoded summarization datasets. The "All" column shows the correlation when examples from all datasets are included. Note for negatively correlated scores (e.g. perplexity, OOD score), take the negative value of the score for easier comparison. few intra-dataset correlations have p-value < 0.05 due to the small sample size (only 100 examples per dataset sent for human evaluation).
In-domain xsum
Near Shift OOD caildailymai
Far Shift OOD
Measure
reddit_tifa
samsum
All
Single Score
INPUT OOD 0.044
MD
-0.018
-0.017
0.133
0.328
RMD
-0.033
0.017
0.133
0.336
Binary Logits
-0.022
-0.061
0.233
OUTPUT OOD
Perplexity (baseline)
0,256
0.186
0.081
0.068
0.300
NLI score (baseline)
0.337
0.308
0.226
0.132
0.381
MD
0.106
-0.055
0.202
0.352
0.384
RMD
0.053
0.177
0.214
0.314
0.385
Binary logits
0.199
-0.100
0.091
0.026
0.213
Combined Score
PR sum (perplexity, input RMD)
0.186
0.134
0.082
0.109
0.358
PR sum (perplexity, output RMD)
0,250
0.350
0.168
0.237
0.415
PR sum (perplexity. input output RMD)
0.171
0.242
0.158
0.250
0.401
PR sum (perplexity, input binary logits)
0.214
0.079
0.322
PR (perplexity, output binary logits)
0.347
0.086
0.114
0.052
0.330
PR sum (perplexity, input output binary logits)
0.277
0.003
0.127
0.096
0.307
Lineare regression (perplexity, input & output)
0.235
0.402
0.170
0.250
0.422
20

---

Published as a conference paper at ICLR 2023
Table A.5: Kendall correlation (p-value < 0.05 are grayed out) between various measures and quality measured by BLEURT on translation datasets. For easier comparison, we negate the signs of the coefficients OOD score). Within the same dataset, perplexity shows good correlation, but it deteriorates (with the exception of MTNT) as we move to more OOD datasets such as Koran.
measures that are expected to have negative correlation with BLEURT (e.g.,
MINT
202014
Single Soore
INPUT OOD
MD RMD
-0.081
-0.131
-0.129
-0.117
-0.171
0.011
-0.147
-0.093
0.012
-0.117
0.007
0.147
0,091
0.049
0.115
0.197
0.013
-0.071
-0,000
0.098
0.083
0.195
Rintey logits
0.144
0.116
0.341
0.162
0.124
-0.003
0:025
-0,071
0.104
0.161
0.200
OUTPLT OOD
Perplesity (haseline)
0.341
(boselinc)
(boseline)
0.329
0.342
0.370
MD
0.029
00,06
.0.04
-0.04
0.096
0.032
0.105
0.057
0.041
-0,020
0,083
RMD Bintzy logits
0.086
0.049
0.011
0.095
0.135
0.026
00.07
40.05
0.061
0.077
0.170
0.106
0,058
0,005
0.114
0.094
-0.03
-0.013
-0.099
-0.012
0.075
0.151
Cumbined Score
(perplexity, iuput
0.354
somppurpesity, cuiput
0.350
sumppeplleityy input& cupput RMD)
0.250
PR sumppeplexityy imput binury logits)
0.323
0.352
0.372
0.384
0.391
0,211
0.111
0.244
0.359
PR sumpperpexxity outpur binnry logits)
0.318
0.302
0.314
0.350
0.356
0.168
0.162
0.127
0.156
0.293
0.299
PR simpperplsiity. inpui & cutput binery logits)
0.300
0.263
0.288
0.309
0.340
0.125
0.145
0.053
0.163
0.287
0.288
Liese repression (perleeiity,
ounpus)
0.318
0.370
0.355
0.414
0.383
0.243
0.180
0.119
0.368
0.367
0.352
21

---

Published as a conference paper at ICLR 2023
A.10
SELECTIVE GENERATION AND OUTPUT QUALITY PREDICTION
(a) ROUGE-I vs Abstention Rate
(b) Survival count per dataset
0.425
Perplexitt (Base)
400
0.400
NLI (Base)
350
Output RMD
0.375
Reg.
Survival Count
300
0.350
PR sum
250
0.325
Xsum
deilymall
0.300
newsroam_pt all
reddit tiu
50
forumsum
0.200
samsum
Abstention Rate
Abstention Rate
Figure A.4: (a) The summarization quality ROUGE-1 vs abstention curve for single scores, in- cluding input and output RMD OOD scores, output perplexity score, and NLI score, and combined scores, including linear regression machine learning model, percentile sum RMD OOD scores and perplexity score. The corresponding under the curve is in Table A.7. (b) The survival count of each dataset as the joint dataset abstained. Each dataset is sub-sampled to 400 examples for this analysis.
1000
haldout
Survival Count
nt2014
nod2015
600
400
medical
Koran
subtities
0.0
0.2
1.0
Abstention Rate
Figure A.5: The translation survival count of each dataset as the joint dataset is abstained. Complete results for Figure 5 (d).
22

---

Published as a conference paper at ICLR 2023
Table A.6: Area under the quality (human eval) abstention curve for summarization for various single scores and the proposed combined scores.
Measure
Area under the quality (buman eval) vs abstention curve
Single Score
Input OOD
RMD
0.466
Binary logits
Outpor OOD
Perplexity (buseline) NI. score (haseline)
0.458
0.469
MD
0.469
RMD Binary logits
0.441
Combined Score
(prrxxpey input RMD)
0.468
(Prpplpprll output RMD)
0.478
PRemn(perlcxxtty input & ourput RMD)
Prrapperpl input binary logits)
0.461
PRun(perplexity output binary logits)
0.461
Prraepperllevity input & output binary logits)
0.456
Linear regression (perplexity, inpat & output RMD)
0,481
Table A.7: Area under the quality (ROUGE-1) 1) abstention curve for summarization for various single scores and the proposed combined scores.
Measure
Area under quality (rougel) vs abstention curve
Single Score
Input
MD
0,208
RMD
0.214
Rinary logits
0.217
Outpur OOD
Perplexity (baseline)
NLI score (baseline)
0.207
RMD
0.221
Binary logits
0.207
Combined Score
PR.. (perplexity, input RMD)
0.222
PRu. (perplexity, output RMD)
0.228
PR... (peplexxttl input output RMD)
0.224
pu (perplexey,,,, input binary logits)
0.225
PR.. (perpeeiit ourput binary logits)
0.221
PR.. (perplexity, input output binary logits)
0.220
Linear regression (perplexity, input & output RMD)
23

---

Published as a conference paper at ICLR 2023
Table A.8: Area under the quality (BLEURT) abstention curve for translation using various single scores and the proposed combined scores.
Names
Area under the quality vs absiention curve
Single Score
Input OOD
MD
0.383
RMD
0.623
Binary logits
Outpor OOD
Perplexity (buseline)
0.627
Comet (haseline)
0,644
Prism (bascline)
0.638
0.601
RMD Rinary logits
0.618
Combined Score
PRai (perplexity. input RMD)
0.647
PR (perplexity. output RMD)
0.646
(Reappepplextty input & output RMD)
0.641
PR (perplexxx imput binary logits)
0.639
PR... (perplexity, output hinary logits)
0.632
PR (perllexit,, inpur output binary logits)
0.633
Lincar regression (ppx, inpur output)
0,645
24

---

Published as a conference paper at ICLR 2023
A.11 INVESTIGATION OF THE N-GRAM OVERLAP BETWEEN LAW DATASET AND IN-DOMAIN DATASETS
domain/split
overall average
R-gram overlap
holdout
nt2014
4.9
39.0
12.3
2.7
0.5
add2015
40.7
12.9
2.7
0.5
adt2015
4.6
39.0
12.8
2.6
0.3
law
7.7
48.8
16.1
4.2
L1
medical
4,3
33.5
10.7
2,4
0.4
Koran
2.8
32.6
8.7
1.4
0.2
2.2
0.3
sub
2,8
38.6
10.9
1,4
0.1
MINT
31.4
1.2
0.1
Table A.9: n-gram overlap analysis between the various test sets including law and the in-domain training data, we observe that law has the highest unigram overlap rate (48.8%) and the second highest overall overlap (defined as the geometric mean) with the in-domain data.
A.12 QUANTITATIVE ANALYSIS USING N-GRAM OVERLAP TO DETERMINE NEAR- AND FAR-OOD DATASETS IN SUMMARIZATION
To support our claim that the news related test datasets, enn dailymail and newsroom are closer to the in-domain xsum than the other dialogue datasets reddit_tifu, samsum, and forumsum, we compute the n-gram overlap between each of the test datasets and the in-domain dataset. We use Jaccard similarity score, (AA,
[Ang] [AUB|' where A and B the of r-gram in dataset A and dataset B, to measure the similarity between two datasets. Table A.10 shows the similarity scores based on 1 - 4 grams. It is clear to see that maillmila and newsroom have significantly higher similarity with the in-domain xsum data than other three datasets. Therefore, we call the news-related datasets OOD and the other dialogue based datasets far- OOD.
R-gram overlap
domain/split
overall average
Xsum
13.3
4.6
1.4
dailymail
6.2
31.1
12.7
4.0
0.9
newsroom
28.8
11.1
3.3
0.7
reddit_tifu
0.3
forumsum
2.7
18.0
6.5
1.6
0.3
samsum
Table A.10: Jaccard similarity based on n-gram overlap between the various test sets and the in- domain xsum training data. We observe that the news-related datasets cmn dailymail and news- room have significantly higher similarity scores with the in-domain xsum data than the other three OOD datasets reddit_tifu forumsum, and samsum.
A.13 VISUALIZATION OF OOD SCORE ON SHIFTED DATASET
We explore how individual parts of input text contribute the OOD score, which help us visualize which parts the text are OOD. We define the OOD score of each sentence in the text using leave-one-out strategy: For any given sentence, with and without that sentence in it. The negative of the change in the OOD score after removing the sentence denotes the OOD score of that sentence. Intuitively, if removing the sentence decreases the overall OOD score, that sentence is assigned a positive OOD score and vice-versa. Figure A.6 illustrates an example where an article contains noise in the form of tweets with emojis, and the OOD scoring mechanism described above assigns positive OOD scores to those tweets and negative scores the main text.
compute the OOD score of the article
25

---

Published as a conference paper at ICLR 2023
Interne Ticaerre Tosteer inytry make
internet the however, a new Twimer bot trying to make oeth-cree importont part of your
Can Important part your Ii celine The bot, wh ch can hed Twitte
followerss to
their heulth
eoamplne encouraging meexagee
which find Twitter @diycaarbot, offers reminders imp ove their health and wall-owing Some ecamples of their encouraging ng messagss are deeply
dieply pleane" and remember up your screen
last contted by Twitier oner dieayyaun who apptassed in taven that la
your
ralivaat remandera becauson be
gyntered drow lattly
The created Tw tweteer Stormyyun, explained than ago,
some best satfoure tipe
bot eosy and tangible seff care reminders because he was "oocessivdly on twitter boely"
mocommenedd below ast comeone for you need it please hete
2010
cronted four days It's gameros followers some the best udf care recommended below
13ke quick break take deep breaths
plase Tenns your reminier (Sttryceee November to 2010 pippooo
pereh renteder
bosembet
your
you
immndettion ligttryarmbett Movember 15, 6 please remember to 13ke-
B2BAM 2016
break stretoh and take quick walk around the room hon0s
O Realy Share
20166008
Figure A.6: OOD score be attributed to individual sentences highlight the out-of-domain noisy parts of text (red denotes out-of-domain and blue denotes in-domain text), e.g. tweets present in articles scraped from internet. Example taken from Newsroom dataset.
26

---

Published as a conference paper at ICLR 2023
A.14 SUMMARIZATION EXAMPLES WITH LOW/ HIGH PREDICTED QUALITY SCORES
Besides the quantitative results, here we show a few real examples to better demonstrate how well our predicted quality score helps for selective generation out-of-distribution examples. The model here was fine-tuned on xsum but inference was run on examples from enn dailymail.
Figure A.7, A.8, and A.9 show 3 examples in cmn dailymail that have the highest (Pamppeppleeiii output RMD) scores that predict for low quality summaries.
Figure A.10, A.11, and A.12 show examples in ean.dallymil that have the lowest PR (perppleex, output RMD) scores that predict for high quality summaries.
Document: man trying clude police jumped into a Missouri creek overnight wearing only his under- wear - but his daring gambit did not pay off. Responding officers and firefighters followed the fugitive into the murky waters Brush Creek in Kansas City and fished him out early Friday momiig. The 38- year-old suspect has been taken to an area hospital to be treated for injuries to his arm and leg. He may face charges in connection a hit-and-rum crash. Escape by water: A year-orl man stripped down to his skivvies and jumped into Brush Creek in Kansas City, Missouri, after being stopped by police. Up Brush Creek without a paddle: The suspect reached the middle of the creek and spent 10-15 minutes swimming back and forth. According called to a gas station in the 4600 block of Prospect at around 2am after receiving complaints from neigh- bors about blasting loud music. The report states that when police approached the car, grey 2007 Infinity, and asked to see the driver's license, the man smiled, said, 'I'm out!' and took off from the scene. The Infinity promptly smashed into the north side of the Brush Creek bridge, after which the driver got out of the mangled car and jumped into the water. Police say the 38-year-old suspect stripped down to his underwear and spent 10-15 minutes swimming chest-deep water, with officers waiting for him on north and south sides of the creek. Surrounded: When firetighters tried to pull him out, he threatened them with log. Fish out water: Police officers armed with BB gun went after the nighttime bather and apprehended him. The bather was complaining of broken leg, according Fox4KC, so the Kansas City Fire Department's 's water rescue crew were sent in to fish him out. But the half-naked man in the water was not going to go quietly. "The suspect picked up a large log and started swinging it at the firemen so they backed off as to not escalate the situation," the arrest report states. That is when uniformed police officers armed with a BB gun followed the man into the creek, got him in a choke hold and pulled him out of the creek. Police suspect the man may have been under the influence of drugs or alcohol. Prelude: Before he jumped in the water, the 38-year-old driver fied from police and smashed his 2007 Infinity into bridge. Police suspect the man may have heen under the influence of drugs or alcohol at the time, As of Friday morning, the y8-year-old has not been formally charged with any crime.
Kansas City Police Department's arrest report, officers were
Reference Summary: The 38-ycar-old suspect was questioned by Kansas City police after neighbors complained he was blasting music in his 2007 Infinity, Instead of handing over his ID, driver smiled, said 'I'm out!" and took off. After crashing into bridge, the man stripped down to his underwear and jumped into Brush Creek. It took cops armed with a BB gun minutes to fish out the fugitive. Model Summary: All images are copyrighted.
Human rating score (f means high quality): 0.2
PR (perplexity, output RMD) (1 means high quality): 0.67
Examples in enn dailymail that have the highest PRam (perplexit, output RMD) scores Figure A.7:
that predict for low quality summaries.
27

---

Published as a conference paper at ICLR 2023
Document: A crisp fan who gets through 42 bags in week has discovered a skull-shaped deep-fried potato snack in one of his packets. Barry Selby, 54, who lives with his dog in Poole, Dorset, was cating bag of cheese and onion erisps when he made the bizarre discovery, which appears he profile of human skull. The floor- fitter has decided to keep the two inches tall by two-and aad-a-halfi inches wide snack as believes it
more impressive than other oddly- shaped examples he has seen on the internet. Scroll down for video. Spooky find: Barry Selby was eating bag of Tesco cheese and onion crisps when he found the 'skull' snack. Selby said: was shocked when found it. was just eating a bag of cheese and onion crisps from Tesco and when I pulled it out it did take back bit. 'I thought it was worth keeping as don't 't think will ever find one like again. must have been a very weird-shaped potato. 'It's about two inches tall and two-and-a-half inches wide and it's in perfect detail, it even has eye socket. 'I sometimes give my dog, Max, crisps in a bowl, so it's lucky he dida't have this packet or wouldn't 't have found it. Weird snack: Mr Selby has decided to keep the unusual find, which appears to show jaw, nose and eye. Comparison: The 54-year-old said he was 'shocked' to make the discovery, although it is not his first. In the 1990s he came
3D heart-shaped crisp, which he kept until it broke. And it's not the first odd-shaped snack he has come across in the 1990s he found crisp shaped like 3D heart, which he kept for several years until it broke. But says this find was different: 'This one was a big one. I just thought "wow" and wanted to share it. 'I've been keeping it top of my front room, but should in protective box really. 'T I'm going to keep it forever, it's just so spooky. I looked on the internet for other funny-shaped crisps but this is one-off."
computer
Reference Summary: Barry Selhy from Dorset was eating bag of Tesco cheese and onion crisps. The 54-year-old discovered snack shaped like profile of the human skull. He said he was 'shocked' with the find and has decided to 'keep forever' It's not his first weird food find he once discovered a heart- shaped crisp.
Model Summary: All images are copyrighted. Human rating
means high quality): 0.2
(perpleell,,,,,, output RMD) (} means high quality): 0.66
Figure A.8: Examples in cnn dailymail that have the highest "pmrppeeel output RMD) scores that predict for low quality summaries.
28

---

Published as a conference paper at ICLR 2023
Document: Last week was barely showing Demelza Poldark now the proud mother show's 's latest addition. Within ten minutes of tomorrow night's episode, fans will see Aidan Turner' dash- ing Ross Poldark gaze lovingly at new baby daughter. As Sunday night's latest heartthrob, women across the country have voiced their longing to settle down with the brooding Cornish gentleman but unfortunately it seems as if his heart is well and truly off the market. Scroll down for video. Last week she was barely showing but Demelza Poldark is He may have married his red-headed kitchen maid out of duty, but as he tells her that she makes him a better man, audiences is the timeline of the pregnancy. With the climax of the previous episode being the announcement of the pregnancy, it is quite jump to the start of tomorrow's instalment where Demelza, played by Elcanor Tomlinson, talks about being eight months pregnant. Just minutes after once again without any nod to the month of her pregnancy passing in than the
the proud mother the show's latest addition.
have little doubt about his feelings. What rather less convincing, however,
passing of time she giving birth, blink of an eye. With the climax of the previous episode being the announcement of the pregnancy, is quite a jump to start of tomorrow's 's instalment where Demelza, played by Eleanor Tomlinson, talks about being eight months pregnant. As Sunday night's latest heartthrob, women across the country have voiced their longing to settle down with Poldark but unfortunately it seems as his heart is well and truly off the market. Their fast relationship didn't go unnoticed by fans. One posted Twitter: 'If you are pregnant Poldark times expect to have it the next 10 minutes' It is reminiscent of the show's previous pregnancy that Elizabeth. another contender for Ross's affection, go full term in the gap between two episodes. This didn't go unnoticed by fans, who posted on Twitter: 'Poldark is rather good, would watch the next now. Though if you are pregnant in Poldark times expect have in the next 10 minutes.'
Reference Summary: SPOILER ALERT: Maid gives birth to baby on Sunday's episode. Only an- nounced she was pregnant with Poldark's 's baby last week. Model Summary: It's change the world Poldark.
Human rating score (t means high quality): 0.4
PR (perplexxt,,, output RMD) ( means high quality): 0.62
Figure A.9: Examples cnn dailymail that have the highest PR (perplexx, output RMD) scores that predict for low quality summaries.
29

---

Published as a conference paper at ICLR 2023
Document: Rangers boss Stuart McCall says he is already working on a dossier of signing targets for next season even though he may not be around to parade them. The interim Ibrox manager still does not know if he will be in charge beyond the current campaign after being lured back to his old club to kick- start their faltering promotion bid, So fär, everything is going to plan with Gers second in the Scottish Championship table and destined for a semi-final play-off slot. Stuart McCall says he is already looking at transfer targets for next season, though he may not at Rangers. But with 12 players out of contract, McCall knows the Light Blues will need to strengthen if they have any chance of keeping pace with rivals Celtic next season if they go up- and is already piecing together a wish list potential new arrivals. He said: 'ive been speaking to lot of agents and putting things in place for if and when... Even if I'm not here, if I' ' getting players put to mc who would like to come to Rangers regardless of the manager, then build a little portfolio of positions that we will be needing next year. 'It's not a case of standing still and then thinking come June 1. 'Oh we need to get into action'. 'No, there are a lot of agents who come to us and we build little dossicr of players that as staff, we think will he good for next season, regardless of what league we are in. 'It would he slightly naive [if we were not doing that]. If I'm in charge or not, still want the club do well and I will put my view across the hoard on who I think should be coming into the club and who should be here." McCall is compiling dossier on targets as he looks to put the club in the best possible position. Rangers have operated a haphazard transfer policy since re-emerging from the embers of liquidation. The club's team of scouts were jettisoned under the disastrous Craig Whyte regime and former boss Ally McCoist was largely forced to turn to a list of former Ibrox servants he had personal knowledge of when trying to bolster his squad. But McCall revealed the club's 's new board are now starting the process of re-establishing their spying network albeit on a smaller level than before, 'I think there has been discussions behind the scenes with different people,' said the former Motherwell boss. 'I don't think we are at the stage where we were 10 or 15 years ago where we were aiming to get into the Champions League and bringing players in for three and four million yet. 'I don't think Rangers will be at the stage yet next year where we need international scouts everywhere. Rangers have expanded their scouting network after haphazard system over the past few years. 'But certainly a scouting network needs to be put in place. "Having said that, I spoke to Craig Levein at Hearts and they do a lot of their scouting with Jonline service] Wyscout. When I brought Henrik Ojamaa in at Motherwell, that was after I'd seen a clip of him on YouTube. I sold him for /350,000 after signing him for nothing. That was great. 'So you can still do your own background work. Personally I would always like to see the player myself. I've only ever signed one player without watching him first and slightly regretted it. 'So yeah we need scouting network but at this moment where Rangers are, not to the extent where we have scouts all over Europe.' McCall admitted he still does not know if he will rejoin Gordon Strachan's 's Scotland staff for the June 13 Euro 2016 qualifier with Ireland in Dublin. And he also confessed to uncertainties ahead of Saturday's 's match with Falkirk. McCall's side are still in line for promotion, sitting in the play-off positions in the Scottish Championship. Peter Houston's 's Bairns five points behind fourth-placed Queen of the South with two games to play- need unlikely series of results to make the play-offs but McCall says that raises more questions than answers. He said: 'Housty is a wily old fox who has done terrifically don't know what expect. 'It will take a difficult set of results for them get
well in his carcer into the play-offs so don't know if they will come here and think the pressure is off and play care free. "They don't 't lose many goals so we may have to be patient through the 90 minutes. We have had a couple of decent results against them but they have capable players and we will need to be at our best,'
Reference Summary: Rangers are currently second the Scottish Championship. Stuart McCall's 's side are in pole position to go up via the play-offs. But McCall is still not certain of his future at the club next season. Rangers boss says he is still trying to build the squad for next year. Rangers have begun to expand their scouting after several poor years.
Model Summary: Stuart McCall says he is already looking at transfer targets for next season, though he may not be Rangers.
Human rating score (f means high quality): 0.8
PR (perplexxy, output RMD) (L means high quality): 0.10
Figure A.10: Examples in enn dailymail that have the lowest PR (perplexiy output RMD) scores that predict for high quality summary.
30

---

Published as a conference paper at ICLR 2023
Document: An Alberta student who'd dd accidentally left his headlights on all day was greeted by what may have been the world's friendliest note from stranger when he returned to his car. But Derek Murray, University of Alberta law student, found more than just note that cold November day in Edmonton- also found an extension cord and battery charger left by the stranger bring his dead Acura back to life. Now that Murray's life-affirming tale has now gone viral, he says just shows you how such a pure of kindness from one person can just spread through everyone and help make everyone's day little brighter' Good Samaritan: A friendly stranger left this unbelievably friendly letter to Alberta law student Derek Murray in order to help him get his started after he left the headlights on all day. At first, though, he assumed the letter was from an angry fellow motorist, he told the National Post. 'When first saw the note, was expecting to an angry letter from someone telling got somcone just totally brightening my day. My day could have been ruined but, because of this guy, was the highlight of my day.' The note reads, in part:. noticed you left your lights on. The battery will probably not have enough charge to start your vehicle. I left blue extension cord on the fence and battery charger beside the fence
not park there. Instead,
cardboard box. you know how hook it up, use it to start your car. What followed was detailed explanation of how to use the equipment. 'Sure enough,' Derek recalled to National Post," 'I looked over at the house my car was parked beside, and there was blue extension cord plugged into an outlet behind the guy's house with battery charger right there beside it.' Derek was able to get his car started, but when he rang the good Samaritan's doorbell, there was no answer. So, Derek left his the stranger's friendly note to post to Facebook, where it has now gone viral. The note has been viewed millions of times and even Edmonton Mayor Don Iveson retweeted the photo. Derek snapped photo of the note for Facehook and it has since gone viral. e 'It just shows you how such a pure act of kindness from one person can just spread through everyone and help make everyone's 's day little brighter,' Derek said.
note a thank you for the kind gesture. He later snapped photo of
Reference Summary: Derek Murray, University of Alberta law student, could have had his day ruined by the mistake by stranger's kindness brightened it up. Murray posted his story and the note online and random act of kindness has now gone viral.
Model Summary: A Canadian student who accidentally left his headlights on all day was greeted by what may have been the world's friendliest note from a stranger when he returned to his car.
Human rating score means high quality): 0.8
PRum (perplexe,, output RMD) (1 means high quality): 0.11
Figure 11: Examples in emn aon_dailymaiil that have the lowest PR (perplexyt output RMD) scores that predict for high quality summary.
31

---

Published as a conference paper at ICLR 2023
Document: Bayern Munich had to make without FOUR important first-leam stars Pep Guardiola's side attempted to overturn a 3-1 deficit against Porto Tuesday night. Injured quartet Franck Ribery, Mehdi Benatia, David Alaha Arjen Robhen were forced watch on from sidelines as the German giants bid to reach the Champions League semi-fine mini-finals. However, the absence of Robben and Co appeared to make no difference as Bayern raced into a 5-0 lead at half-time before claiming a 6-1 victory to win the tie 7-4 aggregate. Injured trio Franck Ribery, Mehdi Benatia and David Alaba chat ahead of Bayem''s clash with Porto. Injured Ribery acknowledges steward before taking a seat at the Allianz Arena on Tuesday night. Ribery looks
former Roma defender Benatia chats with the France international in the dugout. While Ribery, Benatia and Alaba chatted in the home dugout ahcad of kick-off, Holland international Arjen Robben was in front of the mic doing some punditry alongside Bayern goalkeeping legend Oliver Kahn. Ribery missed the game after failing to recover from a recent ankle injury while for- mer Roma defender Benatia faces another two weeks out with groin problem. Robben was unavailable for the encounter with an abdominal injury. David Alaba, meanwhile, is set for month the sidelines having partially ruptured knee ligaments playing for Austria at the start April. Bayern had just fit players to choose from against Porto in the first leg but tore the Portuguese giants apart at the Allianz Arena to progress, Holland international Arjen Robben was pictured doing punditry alongside Bayern legend Oliver Kahn (right) Bayern Munich wideman Robben unavailable for the Champions League clash with an abdominal injury.
Reference Summary: Bayem Munich beat Porto 6-1 the Allianz Arena Tuesday night. German giants were without Franck Rihery, David Alaba and Mchdi Benatia. Arjen Robben was also sidelined and did some punditry for the tic.
Model Summary: Arjen Robben, Mehdi Benatia, Franck Ribery and David Alaba all missed Bayern Mu- nich's 's Champions League quarter-fine second leg against Porto. Holland international Arjen Robben was pictured doing punditry alongside Bayern legend Oliver Kahn (right) Bayern Munich wideman Robben was unavailable the Champions League clash with an abdominal injury.
Human rating score means high quality): 0.8
PR (perplexxy,, output RMD) ( means high quality): 0.11
Figure 12: Examples in cnn can_dannll dailymail that have the lowest PR (perllextt, output RMD) scores that predict for high quality summary.
32


Reviews:
[{"review_id": "ThQJTQrHvo4", "reviewer": "Reviewer_WGqY", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes an out-of-distribution detection mechanism for conditional language models (LMs). The authors investigate settings such as (degrees of) OOD detection for summarisation and translation, and using OOD detection to refrain from generating with the model on-the-fly.", "summary_of_the_review": "My recommendation is a borderline accept. The reason for \"accept\" is the fact the paper is on a timely research topic and that it includes many varied and interesting experiments. The reson for \"borderline\" is detailed in my comments/questions below.\n\n- The examples of existing works on OOD detection could be greatly improved (2nd paragraph in the introduction). Not sure the cow example is the best one, since it is more an example of \"noisy\" rather than \"out-of-distribution\" input. It is also an example in the vision domain, should you not include an example in the (conditional) language modelling domain?\n\n- Page 2, paragraph 1, last sentence (\"In Section 4, we show that while model perplexity is a reasonable choice for [performing selective generation with] in-domain examples, combining with our OOD score works much better when the input distribution is shifted\") -> Please make it clear that you are talking about selective generation, e.g., I added the text in-between brackets to do that as a suggestion.\n\n- Page 2, section 2: In the first paragraph, it is not clear what is the (practical) relationship between maximum softmax probability (MSP) and perplexity. Please better elaborate your point, the paragraph reads like two more-or-less independent points glued together. A method that comes to mind is the mean over the MSP for a sequential output, for instance.\n\n- Page 3, last paragraph (\"MD0(ztest) := MD(ztest; μz0 , Σz0 ) is the MD to a background Gaussian N (μz0 , Σz0 ), fit using a large, broad dataset to approximately represent all domains.\") -> Even though I understand the point, in your introduction you mention that, compared to OOD detection in a standard classification problem, performing OOD detection for CLMs have the added difficulty that CLMs have virtually infinite, unbounded output spaces. This idea of \"approximately\" representing \"all domains\" goes directly against that point. Please defend why this claim/idea makes sense for OOD detection of CLMs. Is the point made in the introduction not so important afterall?\n\n- Page 4, 2nd paragraph (\"While we use the ground-truth outputs to fit N (μw, Σw), we do not have ground-truth in general for the background examples (e.g. C4). We decode outputs from the trained CLMs and use those output embeddings to fit the background output Gaussian, N (μwδ , Σwδ ).\") -> This sentence is not clear. Please rephrase. Your model requires input/output for the two tasks, i.e., summarisation and translation. You have C4 for summarisation, and ParaCrawl for translation. Please elaborate that ParaCrawl does not have translation pairs, but paraphrases (right?), and that C4 does not include summaries (to the best of my knowledge). Your descriptions seem incomplete and assume that the reader will make the leap about important points that should be made clear in your manuscript.\n\n- Page 4: \"Since we have implicitly defined two classes (...)\" -> There was no implicit definition, I would say this definition of the two classes was explicitly made.\n\n- You are using AUROC as your metric, but I would like to see the plots rather than the summary metric (i.e., the actual curves), perhaps in the appendix. As baselines, you are using the perplexity score but not the maximum softmax probability MSP. Could you not use the mean MSP (over the sequence) as a proxy for OOD? Does it consistently underperform compared to using perplexity?\n\n- Page 6: \"Though RMD and Binary logits OOD scores both perform well at OOD detection, RMD OOD score is better at distinguishing near-OOD from far-OOD.\" -> Why does that matter? That is not what models were trained for. If you had trained models (RMD vs. Binary logits) to distinguish between near-OOD and far-OOD, perhaps the outcome would have been different. This split of near- vs. far-OOD is also a bit arbitrary, if you will make it central in your evaluation/discussion (like you do), perhaps it is a good idea to be more principled about it (e.g., using some objective/concrete measure to make one dataset \"more OOD than\" another). You hint at unigram overlap as a measure of OOD-ness later on, perhaps you should use the \"degree of OOD-ness\" as the percentage of unigram overlap between the two domains, for example.\n\n- Page 6: \"We observed that law has the highest unigram overlap rate (48.8%) and the second highest overall overlap with the in-domain data (Table A.7). This confirms that law is actually not OOD data and explains why no method can detect it.\" -> If your definition of OOD only depends on unigram overlap, you do not need anything other than computing the unigram overlap between domains to perform OOD detection. I find this a very misleading comment, since the fact a dataset has high unigram overlap with another does not automatically rule out this dataset being OOD. You should include more nuance in these statements.\n\n- Page 7: \"Amazon Mechanical Turk workers were asked to evaluate summaries generated by the xsum model on a scale of 1-5 (bad-good) using 100 examples from xsum, cnn dailymail, reddit tifu, and samsum. We collected 3 ratings per example and took the median to reduce inter-rater noise\" -> This is a bad strategy to reduce inter-annotator agreement. You are basically hiding the (possible) disagreement by only using the median, whereas in order to reduce the disagreement (and not mask it) you should instead 1) select high-quality annotators, 2) make the annotation guidelines clear including concrete examples of how to annotate difficult cases, 3) include control examples for annotators, etc. Please at least report the variance of the annotators, so the reader knows how noisy they are.\n\n- Page 8, Section 4.4: \"To evaluate that, we propose using the Quality vs Abstention Curve (QA), analogous to accuracy versus rejection curve used for selective prediction in the classification.\" -> Citation missing. You mention Malinin & Gales (2020); Xiao et al. (2020) later, but it is not clear how exactly they relate to QA. If you are the one proposing this measure, please make it clear.\n\n- Page 8, Section 4.4: \"For translation, the OOD score is better than perplexity when abstention rate α > 0.5 and worse than perplexity when α < 0.5.\" -> From Figure 5, the cut-off point looks more like 0.6 or 0.65 rather than 0.5. Is that so?\n\n- Page 9, Section 5, last paragraph: \"In this work, we focus on developing scores that can be readily derived from the generative model itself, without much increase in computation.\" -> You did not compute how much time it takes to run your proposed models compared to the baselines. I find it could be a limitation of your work, especially in light of the existing work using ensembles mentioned in your related work section. You also did not include any such ensembles as your baselines. Why?", "strength_and_weaknesses": "Strengths:\n- The authors address a timely research question, since LMs popularity is (still) on the rise and LMs are deployed almost as out-of-the-box tools for \"language understanding\" or \"language generation\".\n- Quite a number of experiments on different aspects of OOD detection. I like the in-depth investigation of the different ideas proposed/studied in the paper.\n\nWeaknesses:\n- There are some statements/claims that are either not well developed in the text (leaving to the reader to guess the intention of the authors when saying something) or that seem overeaching/inconsistent. Please refer to my comments for detailed examples of these.\n- The paper can be hard to follow at times, and there are a lot of different experiments. Perhaps the authors would have a perceivingly stronger paper by making it less wide and more deep (i.e., cover less different points but go deeper into the points it does cover).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "Ilnw78FccJg", "reviewer": "Reviewer_U7Pj", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Conditional language models often perform poorly on out-of-domain (OOD) data. This might be a reason why they shouldn't be tasked with generating anything for OOD examples at all: generations might be of poor quality and even more unpredictable than normal outputs. In order to predict which examples a conditional language model can be safely applied to, the authors propose a score to detect OOD examples. The score they introduce is Mahalanobis distance, either using the input or the output embeddings. They further show that the latter can be combined with perplexity to obtain an even stronger score.\n\nThe authors perform two types of experiments (on summarization and translation). First, they show that their score can detect OOD data more reliably than baselines. Second, they show that their score can also indicate low-quality generations. ", "summary_of_the_review": "I believe this paper will be useful for the community. It should get accepted.", "strength_and_weaknesses": "Strengths:\n- The paper has a clear motivation and succeeds at what it's trying to do.\n- The experiments are well designed.\n\nWeaknesses:\n- Nothing major.\n\nThere are a couple of typos. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "B7h-MqEz1h", "reviewer": "Reviewer_f7UA", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper addresses the out-of-domain (OOD) detection for conditional generation tasks. It proposes an OOD score, which is based on the Mahalanobis Distance (MD) to the input and output embeddings of the language model. The MD-based OOD score is computed in relative to a background Gaussian model. Furthermore, the combined OOD score and perplexity correlates well with generation quality, and can be used for selective generation and early rejection based on OOD score. The proposed approaches are validated on summarization and translation tasks.", "summary_of_the_review": "The problem is interesting, the proposed approach is effective, and experiments and analysis are comprehensive.\nNovelty is a bit limited, lack of comparison with other common OOD approaches, and the generalization of the proposed approach is questionable.\n", "strength_and_weaknesses": "Strength:\n(1)\tIt addresses an interesting OOD problem for conditional generation, where OOD errors can get easily accumulated via auto-regressive generation.\n(2)\tIt proposes an effective OOD score. The score is based on the MD to the input and output embeddings of the language model. Relative MD is used to measure the inference time deviation from training data and a background distribution model.\n(3)\tThe OOD score in combination with perplexity provides a good metric for selective generation of near-domain samples with quality control. It correlates well with generation quality. It is simple to deploy to address the domain shift issue at inference time.\n(4)\tExperiments and results are well-explained. Analysis is comprehensive. The analysis of the correlation of perplexity and generation quality with OOD score changes is insightful.\n\nWeaknesses:\n(1)\tNovelty is limited as MD-based OOD metrics have been widely studied and used, though this paper used it in a different setting (non-classification). \n(2)\tIt is not compared with other common OOD detection strategies, such as threshold based, energy based, contrastive based approaches.\n(3)\tIt’s not clear how the proposed method generalizes to other tasks and model structures.\n(4)\tIt is unclear why the embeddings from the final layers are chosen.\n(5)\tIt is unclear how the background data is selected, and how well the background Gaussian is expected to generalize. It seems fitting the background Gaussian is nontrivial and the proposed method is not as light-weight as it claimed to be. More importantly, what if we know nothing about the OOD? It seems the background Gaussian would fail to work when there is no good definition of background data.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: notable-top-25%