RainProof: An Umbrella to Shield Text Generator from Out-Of-Distribution Data
Authors: Maxime DARRIN
Pablo Piantanida
Pierre Colombo
Venue: EMNLP 2023 Main (2023)
Keywords: NLP
OOD detection
natural language generation
TL;DR: We propose an information theoretically grounded OOD detectors for the black-box text generation setting

Abstract: Implementing effective control mechanisms to ensure the proper functioning and security of deployed NLP models, from translation to chatbots, is essential. A key ingredient to ensure safe system behaviour is Out-Of-Distribution (OOD) detection, which aims to detect whether an input sample is statistically far from the training distribution. Although  OOD detection is a widely covered topic in classification tasks, most methods rely on hidden features output by the encoder. In this work, we focus on leveraging soft-probabilities in a black-box framework, i.e. we can access the soft-predictions but not the internal states of the model. Our contributions include: (i) RAINPROOF a Relative informAItioN Projection OOD detection framework; and (ii) a more operational evaluation setting for OOD detection. Surprisingly, we find that OOD detection is not necessarily aligned with task-specific measures. The OOD detector may filter out samples well processed by the model and keep samples that are not, leading to weaker performance. Our results show that RAINPROOF provides OOD detection methods more aligned with task-specific performance metrics than traditional OOD detectors.

RAINPROOF: An umbrella to shield text generators from Out-of-Distribution data
Maxime Darrin'
Pablo Piantanida
Pierre Colombo
ILLS
ILLS
MICS
MILA Quebee AI Institute
MILA - Quebec AI Institute
CentraleSupélec
McGill University
CNRS, CentraleSupélec
Paris-Sacla University
Paris-Saclay University
Paris-Saclay University
Equall, Paris
Abstract
world (Fei and Liu, 2016), where training and test data are assumed to be drawn i.i.d. from a single distribution, known as the in-distribution. However, when deployed, these models operate in an open world (Parmar et al., 2021; Zhou, 2022) where the i.i.d. assumption is often violated. Changes in data distribution are detrimental and induce drop in performance. It is necessary to develop tools to pro- tect models from harmful distribution shifts as it is clearly unresolved practical problem (Arora al., 2021). For example, a trained translation model is not expected to be reliable when presented with another language (e.g. a Spanish model exposed to Catalan, or a Dutch model exposed to Afrikaans) or unexpected technical language (e.g., colloquial translation model exposed to rare technical terms from the medical field). They also tend to be leased behind API(OpenAL, 2023) ruling out many usual features-based OOD detection methods.
Implementing effective control mechanisms to ensure the proper functioning and security of deployed NLP models, from tramslation to chat- bots, is essential. A key ingredient to ensure safe system behaviour is (OOD) detection, which aims to detect whether input sample is statistically far from the train- ing distribution. Although OOD detection is a widely covered topic classification tasks, most methods rely on hidden features output by the encoder. In this work, we focus on lever- aging soft cott-rrobbbliiiess in a black-box frame- work, i.e. but not the internal states of the model. Our contributions include: (i) RAINPROOF Rela- tive informAltioN Projection OOD detection framework; and (ii) a more operational evalu- ation setting for OOD detection. Surprisingly, we find that OOD detection is not necessarily aligned with lask-specific measures. The OOD detector may filter out samples well processed by the model and keep samples that are not, leading weaker performance. Our results show that RAINPROOF provides OOD detection methods more aligned with task-specific perfor- mance metries than traditional OOD detectors.
access the soft- coft-predictions
Most work on Out-O--Distibuttion (OOD) detec- tion focus on classification, leaving OOD detection in (conditional) text generation settings mainly un- explored, even though it is among the most exposed applications. Existing solutions fall into two cate- gories. The first one called Training-awan methods (Zhu et al., 2022; Vernekar et al., 201 2019a,b), mod- ifies the classifier training by exposing the neural network OOD samples during training. The sec- ond one, called plug-in methods aims distinguish regular samples in the in in-distribution (IN) from OOD samples based on the model's behaviour on a new input. Plug-in methods include Maximum Softmax Probabilities (MSP) (Hendrycks and Gim- pel, 2016) or Energy (Liu et al., 2020) or feature- based anomaly detectors that compute per-class anomaly score (Ming al., 2022; Ryu et al., 2017; Huang et al., 2020; Ren et al., 2021a). Although plug-in methods from classification settings seem attractive, their adaptation to text generation tasks is more involved. While text generation can be seen as a sequence of classification problems, i.e.,
1 Introduction
Significant progress has been made in Natural Lan- guage Generation (NLG) in recent years with the development of powerful generic (e.g., GPT (Rad- ford et al., 2018; Brown et al., 2020; Bahrini et al., 2023), LLAMA (Touvron et al., 2023) and its vari- ants) and task-specific (e.g., Grover (Zellers et al., 2019), Pegasus (Zhang et al., 2020) and Dialog- GPT (Zhang et al., 2019b)) text generators. They power machine translation (MT) systems or chat- bots that exposed to the public, and their reli- ability prerequisite for adoption. Text genera- tors are trained in the context of a so-called closed
maxime. darr darrin@mila. quebec
İLLS International Laboratory on Learning Systems Mathématiques et Informatique Centralcsupolec

---

chosing the next token at each step, the number of possible tokens two orders of magnitude higher than usual classification setups.
(Csiszar and Matus, 2003), which relies on refer- ence set to get a data-driven notion of normality.
3. New insights on the operational value of OOD detectors. Our experiments on LOFTER show that OOD detectors may filter out samples that are well processed (i.e. well translated) by the model and keep samples that are not, leading to weaker perfor- mance. Our results show that RAINPROOF improves performance on the end task while removing most of the OOD samples.
In this work, we aim to develop new tools to build more reliable text generators which can be used in practical systems. To do so, we work under 4 constraints: (i) We do not assume we can access OOD samples: (ii) We suppose we are in a black- box scenario: we do not assume we have access to the internal states of the model but only to the soft probability distributions it outputs; (iii) The detectors should be easy enough to use on top of any existing model to ensure adaptability; (iv) Not only should OOD detectors be able to filter OOD samples, but they also are expected to improve the average performance the end-task the model has perform.
4. Code and reproductibility. We release plug- and-play library built upon the Transformers library that implements our detectors and baselines 3
2 Problem Statement & Related Works
2.1 Notations & conditional text generation
Our contributions. Our main contributions can be summarized as follows:
Let us denote Ω, a vocabulary of size [S2] and Ω its Kleene closure*. We denote P(22)
1. A more operational benchmark for text gen- eration 0OD detection. We present LOFTER the Language Out oF disTribution pErformance benchmaRk. Existing works on OOD detection for language modelling (Arora et al., 2021) focus on (i) the English language only, (ii) the GLUE bench- mark, and (iii) measure performance solely in terms of OOD detection. LOFTER introduces more real- istic data shifts the generative setting that goes beyond English: language shifts induced by closely related language pairs (e.g., Spanish and Catalan or Dutch and Afrikaans (Xiao et al., 2020)!) and domain change (e.g.. medical vs news data or vs di- alogs). In addition, LOFTER comes with updated evaluation setting: detectors' performance is jointly evaluated w.n. the overall system's 's performance on the end task.
{ P E [0, 10. D P: = ] the set of probabil- ity distributions defined over Q. Let Derain be the training set, composed of N 2 1 i.i.d. samples {(x', ) E (X x )) with probability law PXY. We denote Px and py the associated marginal laws of pxy. Each x' is sequence of to- kens, and a/. E Zi the jth token of the ith sequence.
X - {0).- , E S2" denotes the prefix of length L. The same notations hold for Conditional textual generation. In conditional textual generation, the goal is to model probabil- ity distribution P. yx over variable-length text sequences (x, y) by finding Py ~ (x, for any (x. y). In this work, we assume have ac- cess to a pretrained conditional language model fo : X x y - R where the output is the (un- normalized) logits scores. fe parameterized pe, i.e., for any (x, y),, (x y) = softmax (foo (/T) where T E R denotes the temperature. Given an input sequence x, the pretrained language fo recursively generate output sequence ý by sam- pling l/+11 ~ P (-xx, for t E [yl] Note that ûo is the start of sentence (< SOS > token). We denote by S(x), the set of normalized logits scores generated by the model when the initial in- put is x i.e., S(x)
2. A novel detector inspired by information pro- jection. We present RAINPROOF: Relative in- formAltioN Projection Out OF distribution detec- for. RAINPROOF is fully unsupervised. is flexible and can be applied both when no reference samples (IN) are available (corresponding to scenario so) and when they are (corresponding to scenario 31). RAINPROOF tackles so by computing the models' predictions negentropy (Brillouin, 1953) and uses it as measure of normality. For $1, it relies upon its natural extension: the Information Projection
[softmmx((fe(x,, (x))
hhttps://gith thw.bbb.tttt....// wtt.iioccciii///eeeeeecccc wwwaanno/oonnnennnniiiioo "This work was performed using HPC resources from GENCI-IDRIS (Grant 2022-AD01 1013945).
Affrikaans is a daughter language of Dutch. The Dutch sentence: "Appelen zijn gewoonlijk groen, geel of rood" cor- responds to "Appels is gewoonlik groen, geel of rooi."
"The Kleene closure corresponds sequences of arbitrary
size written with words in 2. Formally: Ω = U
no

---

Note that elements of S(x are discrete probability distributions over 92
Softmax-baser detectors. These detectors rely on the soft probabilities produced by the model. The MSP (Hendrycks and Gimpel, 2017; Hein et al., 2019; Liang et al., 2018; Hsu et al., 2020) uses the probability of the mode while others take into account the entire logit distribution (e.g., Energy- based scores (Liu et al., 2020)). Due to the large vocabulary size, it is unclear how these methods generalize sequence generation tasks.
2.2 Problem statement
OOD detection, the goal is find an anomaly score a : + R2 that quantifies how far a sample is from the IN distribution. x is classified as IN or OUT according to the score a(x). We then fix a threshold ?y and classifies the test sample IN if a(x) < y or OOD if a(x) > 7. Formally, let denote g(-, 7) the decision function, we take:
OOD detection for text generation. Little work has been done on OOD detection for text genera- tion. Therefore, we will follow (Arora et al., 2021; Podolskiy et al., 2021) and rely on their baselines. We also generalize common OOD scores such as MSP or Energy by computing the average score along the sequence at each step of the text gener- ation. We refer the reader Sec. B.7 for more details.
g(x, 1 if (x) >
0 if (x) 7.
Remark 1. In our setting, OOD examples are not available. Tuning y is a complex task, and it is usu- ally calibrated using OOD samples. In our work, we decided not to rely on OOD samples but on the available training set to fix y in a realistic setting. Indeed, even well-tailored datasets might contain significant shares of outliers (Meister al., 2023). Therefore, we fix y so that at least 80% of the IN data pass the filtering procedure. See Sec. G.3 for more details.
Quality estimation as OOD detection metric. Quality Estimation Metries are not designed to de- tect OOD samples but to assess the overall quality of generated samples. However, they are interest- ing baselines to consider, as OOD samples should lead to low-quality outputs. We will use COMET QE (Stewart et al., 2020) baseline to filter out low-quality results induced by OOD samples.
2.3 Review of existing OOD detectors
0OD detection for classification. Most works on OOD detection have focused on detectors for clas- sifiers and rely either on internal representations (features-bssed detectors) or on the final soft prob- abilities produced by the classifier (softmax based detectors).
Remark 2. Note that (eatures-based detectors as- sume white-box access internal representations, while soflmax-bused detectors rely solely on the final output. Our work operates in black-bor framework but also includes a comparison to the Mahalanobis distance for completeness.
Features- based detectors. They leverage latent representations to derive anomaly scores. The most well-know is the Mahanalobis distance (Lee et al., 201 8a; Ren et al., 2021b), but there are other meth- ods employing Grams matrices (Sastry and Oore, 2020), Fisher Rao distance (Gomes et al., 2022) or other statistical tests (Haroush et al., 2021). These methods require access to the latent representations of the models, which does not fit the black-box sce- nario. In addition, is well known in classification that performing per-class OOD detection is key to get good performance (Leee et al., 2018b). This per-class approach is a priori impossible in text generation since it would have to be done per token or by some other unknown type of classes. We ar- gue that it is necessary to find dependdd solutions, especially when it comes to the Maha- lanobis distance, which relies upon the hypothesis that the data are unimodal; we study the validity of this hypothesis and show that it is not true in a generative setting in Ap. A.
3 RAINPROOF OOD detector
3.1 Background
An information measure I P(S2) X P(L)- + R quantifies the similarity between any pair of dis- crete distributions p.q E P(?). Since Ω is a finite set, will adopt the following notations p p there exist information distances, is, general, difficult to build metrics that satisfy all the proper- ties of a distance, thus we often rely on divergences that drop the symmetry property and the triangular inequality.
... ,Ppj and q = [qi.- ... I. While
In what follows, we motivate the information measures will use in this work. First, we rely on the Rényi divergences (Csiszár, 1967). Rényi divergences belong to the f- divergences family and are parametrized by a parameter a E R {1). They are flexible

---

and include well-known divergences such the Kullback-Leiber divergence (KL) (Kullback, 1959) (when a - 1) or the Hellinger distance (Hellinger, 1909) (when a = 0.5). The Rényi divergence be- tween p and is defined as follows:
3. The last step consists of thresholding the previ- ous anomaly score az (x)). If ar(xx is over a given threshold ",, we classify x as an OOD example.
Interpretation of Eq. 3. az (x measures the average dissimilarity of the probability distribution of the next token normality (as defined by u). (x(x) also corresponds to the token average un- certainty of the model fo to generate ý when the input x. The intuition behind Eq. that the distributions produced by fo. when exposed to an OOD sample, should be far from normality and thus have high score.
1
1
P a-1
(a(pl)q) a log Σ
(1)
4
The Rényi divergence is popular as a allows weighting the relative influence of the distributions' tail.
Second, we investigate the Fisher-Rap distance (FR). FR is distance on the Riemannian space formed by the parametric distributions, using the Fisher information matrix as its metric. It com- putes the geodesic distance between two discrete distributions (Rao, 1992) and defined as follows:
Choice u and I. The uncertainty definition of Egq. 3 depends on the choice of both the reference distribution u and the information mcasure I. A natural choice for is the uniform distribution, i.e., [ि| | which we will use in this work. is worth pointing out that Z(-I|u) yields the negentropy of a distribution (Brillouin, 1953). Other possible choices for u include one hot or tf-idf distribution (Colombo et al., 2022b). For I. rely the Rényi divergence to obtain ap, and the Fisher-Rao distance to obtain afR.
(
arccos Σ Vp x qi-
(2)
It has recently found many applications (Picot et al., 2022; Colombo et al., 2022b).
3.2 RAINPROOF for the no-reference scenario (sg)
3.3 RAINPROOF for the reference scenario (s1)
At inference time, the no-reference scenario (5g) does not assume the existence of reference set of IN samples decide whether new input sample is OOD. Which include, for example, boftmax-based detectors such as MSP, Energy or the sequence log-likelihood
In the reference scenario (s1), we assume that one has access to reference set of IN samples R {xf (x', yx), E Dtrain ) where [ the size of the reference set. For example, the Mahalanobis distance works under this assumption. One of the weaknesses of Eq. 3 is that it imposes an ad-hoc choice when using u (the uniform distribution). In $1, can leverage R, to obtain data-driven notion normality.
Under these assumptions, our OOD detector RAINPROOF comprises three steps. For given in- put x with generated sentence y:
We first use fo to extract the step-by-step se- quence of soft distributions S(x).
Under $1, our OOD detector RAINPROOF follows these four steps:
2. We then compute an anomaly score (ar(x)) by averaging step-by-step score provided by I. This step-by-step score is obtained by measuring the similarity between reference distribution E P(?) and one element of S(x). Formally,
1. (Offline) For cach x' E R. we generate ý and the associated sequence of probability distribu- tions (S(x')). Overall we thus generate Exer |s*)| probability distributions which could explode for long sequences To overcome this limitation, we rely the bag of distributions of each sequence (Colombo et al., 2022b). We form the set of these
ax(x)= [x(x
Σ [ppll),, peS(x)
'It also worth pointing out that projecting at each timestep would require a per-step reference set in addition to the computational time required to compute the projections, therefore we decided to aggregate the prohahility distributions over the sequence.
where [8(x) = |
'The detector based the log-likclihood of the sequence is defined as as (x) = | D log Po po(f++1)x0 xx,

---

bags of distributions:
cause a significant drop in performance. For lin- guistically close languages, we selected closely related language pairs such as Catalan-Spanish, Partuguexe-ppaniin and Afrikaans-Duteen coming from the Tatoeba dataset (Tiedemann, 2012) (see Tab. 8). Domain shifts can involve techmical or rare terms or specific sentence constructions, which can affect the model's ' performance. We simulated such shifts from Tatoeba MT using news, law (EuroParl dataset), and medical texts (EMEA).
== = X
p PCS(x))
(4)
S(xi)
(Online) For a given input x with generated sen- tence y, we compute its bag of distributions repre- sentation:
f(x) = [x(
P. PES(X)
Σ
(5)
LOFTER for dialogs. For conversational agents, wc focused on a scenario where a goal-oriented agent, designed to handle specific type of con- versation (e.g., customer conversations, daily dia- logue), is exposed to an unexpected conversation. In this case, it is crucial to interrupt the agent so it does not damage the user's trust with misplaced responses. We rely on the Multi wOZ dataset (Zang et al., 2020), a numan-to-humme dataset collected in the Wizard-of-On set-up (Kelley, 1984), for IN dis- tribution data and its associated fine- tuned model. We simulated shifts using dialogue datasets from various sources, which are part of the SILICONE benchmark (Chapuis et al., 2020). Specifically, we use goal-oriented dataset (i.e., Switchboard Di- alog Act Corpus (SwDA) (Stolcke al., 2000)), multi-party meetings dataset (i.e.. MRDA (Shriberg et al., 2004) and Multimodal EmotionL Dataset MELD (Poria et al., 2018)). daily communication di- alogs (i.e., DailyDialgg DyDA (Li et al., 2017)), and scripted scenarii (i.e., IEMOCAP (Tripathi et al.. 2018)). We refer the curious reader to Sec. B.5 for more details on each dataset.
3. (Online) For x, we then compute an anomaly score (x) by projecting p(x) on the set S" For- mally, oz (x) is defined as:
(x) min T(plp(x)).
(6)
pES.
We denote p" (x) = arg min I
PES
4. The last step consists of thresholding the previ- ous anomaly score (x). If az(x) is over given threshold y, we classify
OOD example.
Interpretation of Eq. 6. (r(x) relies on a Gen- eralized Information Projection (Kullback, 1954; Csiszár, 1975, 1984) which measures the similar- ity between (xx) and the set S Note that the closest element of S* in the sens of can give insights on the decision of the detector. It allows interpreting the decision of the detector as we will see in Tab. 6.
Choice of T. Similarly to Sec. 3.2, we will rely on the Rényi divergence to define R, (x) and the Fisher-Rao distance (x).
Model Choices. We evaluated methods open-source and freely available language bilin- gual models (the Helsinki suite (Tiedemann and Thottingal, 2020)), on a BLOOM-based instruc- tions model BLOOMZ (Muennighoff et al., 2022) (for which German is OOD). For dialogue tasks, we relied on the Dialog GPT (Zhang et al., 2019b) model finetuned on Multi WOZ, which acts as IN distribution. We consider the Helsinki models they used in production for lightweight appli- cations. Additionally, they are specialized for a specific language pair and released with their asso- ciated training set, making them ideal candidates to study the impact of OOD in a controlled setting." Metrics. To evaluate the performance on the OOD task, we report the Area Under the Receiver Oper-
4 Results on LOFTER
4.1 LOFTER: Language Out oF disTribution pErformance benchmaRk
LOFTER for NMT. We consider a realistic setting involving both topic and language shifts. Language shifts correspond exposing a model trained for given language to another which is either linguisti- cally close (e.g., Afrikaans for a system trained on Dutch) or missing in the training data (as it is the case for german in BLOOM (Scao et al., 2022)). It is an interesting setting because the differences between languages might not be obvious but still
'The minimization problem of Eq. 6 finds numerous con- nections in the theory of large deviation (Sanov, 1958) or in statistical physics (Jaynes, 1957).
*Please note that for the likelihood detector, the transla- tion model is additionally fine-tuned on the development set, ensuring a strong bascline.

---

Table 1: Summary of the OOD detection perfor- mance of detectors (Ours) compared commonly used strong baselines (Bas.). We report the best detector for each scenario bold and underline the best overall. The 4 indicates that for this score, the lower, the better; otherwise, the higher, the better."
0.9
[]
400
800
1200
AUROC
0.7-
2000
Language shifts
Domain shifts
Dialog shifts
AUROC
FPR
AUROC
FPR
AUROC
FPR
Ours
0g.
0.95
0.25
0.85
0.62
0.79
0.64
0.00
dFR
0.93
0.28
0.81
0.67
0.76
0.68
50 Bas.
0MSP
0g
0.87
0.44
0.44
0.79
0.78
0.77
0.66
0.76
0.72
Figure I: Ablation study on RAINPROOF for and refer-
0.78
0.79
0.75
0.88
0.65
0.95
ence set size ((R) for dialogue shift detection. Smaller a emphasizes the tail of the distribution, while a = counts common non-zero elements.
0.71
0.57
0.73
0.88
X
X
di 0.88
0.50
0.52
Ours
Ogg"
0.35
0.69
0.76
0.75
51
Bas.
0.92
0.26
0.78
0.59
0.84
0.55
quence likelihood) by several AUROC points. Inter- estingly enough, even our no-reference detectors outperform the reference-based baseline (i.e., a.M deeper study of this phenomenon is presented in Ap. A). While aD,, achieves similar AUROC per- formance to its information projection counterpart OD; the latter achieve better FPR J. Once again, the COMET QE metric does not yield competitive performance for OOD detection.
Table Correlation between OOD scores and trans- lation metries BLEU, BERT-S and COMET.
BERT-5
BLEU IN
COMET
AlI1
IN
OUT
Al.1.
OUT
ALL1
IN
OUT
Scare
Ourx
-0.45
20:43
10.33
40.33
-0.45
40.55
-0.35
-0.35
-0.25
.0.27
-0.37
+0.38
-0.35
-0.33
-0.25
00.23
-0.26
-0.22
<0.13
0.21
0:25
0.15
0.21
wige
0.00
0.11
0088
0:21
0145
0:45
Ouze
-.31
0.33
01.00
0.20
0.64
0.65
-..00
0.21
.0.15
0.16
4,33
0.35
4.11
0.11
-004
003
-0.15
0.11
4.3 Experiments in dialogue generation
Bax
0.21
0.12
0.29
0.08
0,13
0.65
0.06
0001
0.1
Results on Dialogue shifts (Tab. 1). Dialogue shifts are understandably more difficult to detect, as shown in our experiments, as they are smaller than language shifts. Our no-reference detectors do not outperform the Mahalanobis baseline and achieve only 0.79 in AUROC. The best baseline is the Mahalanobis distance and achieves better per- formance dialogue tasks than on NMT domain shifts, reaching an AUROC of 0.84. However, reference- basedd detector based on the Rényi infor- mation projection secures better AUROC (0.86) and better FPR 4 (0.52). Even our detectors achieve decent results on this task, is clear that dialogue shifts will require further work and investigation (see Ap. F), especially in the wake of LLMs.
ating Characteristic AUROC and the False positive rate FPR J. These methods have been widely em- ployed in previous research on out-of-distribution (OOD) detection. An exhaustive description of the metrics can be found in Sec. 4.1.
4.2 Experiments in MT
Results on language shifts (Tab. 1). We find that our no-reference methods (aD,, and aFR) achieve better performance than common no-reference baselines but also outperform the reference-basee baseline. In particular, ap,,, by achieving an AUROC of 0.95 and FPR of 0.25, outperforms all con- sidered methods. Moreover, while no-reference baselines only capture up 45% of the OOD sam- ples average, ours detect up 55%. In ad- dition, COMET QE, quality estimation tool, per- forms poorly in pure OOD detection, suggesting that while OOD detection and quality estimation can be related, they are still different problems.
4.4 Ablations Study
Fig. shows that RAINPROOF offers a crucial flex- ibility by utilizing the Rényi divergence with ad- justable parameter alpha. RAINPROOF's 's detectors show improvement when considering the tail of the distributions. Notably, lower values of a (close to 0) yield better results with the Rényi Informa- tion projection aD; This finding suggests that the tail of the distributions used in text generation contains contextual information and insights about the processed texts. These results are consistent
Results domain shifts (Tab. 1). We evaluate the OOD detection performance of RAINPROOF on domain shifts in Spanish (SPA) and German (DEU) with technical, medical data and parliamentary data.
For So, we observe that aD,, and OFR outperform the strongest baselines (i.e., Energy, MSP and se-

---

Table 3: Impact of OOD detectors on BLEU for IN data only, OOD data and the combination of both ALL. We report average BLEU (Abs.). BLEU gains (G.s) compared to Se only, and removed subset share (R.Sh.). y set to remove 20% of IN dataset.
5.2 Effects of Filtering on Translation Quality
In this experiment, we investigate the impact of OOD filtering from the perspectives of quality esti- mation and selective generation.
Global performance. In Tab. 3 and Tab. 5, we report the global performance of the system (fe) with and without OOD detectors on IN samples, OOD samples, and all samples (ALL). In most cases, adding detectors increases the average qual- ity of the returned answers on all three subsets but with varying efficacy. aMSP a notable excep- tion, and we provide a specific correlation analysis later. While the reference-based detectors tend to remove more OOD samples, the no-reference de- tectors demonstrate better performance regarding the remaining sentences' average BLEU. Thus, OOD detector evaluation should consider the final task performance. Overall, is worth noting that di- rectly adapting classical OOD detection methods (e.g., MSP or Energy) to the sequence generation problem leads to poor results in terms of perfor- mance gains (i.e., as measured by BLEU or BERT-S). OD, removes up to 62% of OOD samples (whereas the likelihood only removes 15%) and maintains or improves the average performance of the system on the end task. In other words, aD provides the best combination of OOD detection performance and system performance improvements.
OOD
Als.
Cl3
R.5h
Als
G=
Als.
R.Sa
92.
Bos.
44.2%
855
55.3
2588
29
12.5
-1X
30.90
Opm
819
919
40.3
40.2
19.35
19.05
1133
-01
-003
1010
99.99
51.0
51.0
+2.4
+2.4
11.8%
Bo.
53.2
4.0
20.0%
319
-1.1
63.4%
504
+2.2
$3.99
Table 4: Computation time (in sec). Off. (Onl.) stands
for offline (resp. online) time.
Score
Off.
Onl.
"Da
X
2.10-3s s
GMSP
1.10-1s
40s
3.10-*s
apg
9.10
with recent research in automatic text generation evaluation (Colombo et al., 2022b). Interestingly, increasing the size of the reference set beyond 1.2k has minimal influence. We provide an additional study of the impact of the temperature and parame- ter a for the different OOD scores Ap. E.
5 A More Practical Evaluation
Following previous work, we measure the perfor- mance of the detectors on the OOD detection task based AUROC and FPR J. However, this evalua- tion framework neglects the impact of the detector on the overall system's 's performance and the down- stream task it performs. We identify three main evaluation criteria that are important in practice: (i) execution time, (ii) overall system performance in terms of the quality of the generated answers, and (iii) interpretability of the decision. Our study is conducted on NMT because due to the existence of relevant and widely adopted metrics for assessing the quality of a generated sentence (i.e., BLEU (Pap- ineni al., 2002) and BERT-S (Zhang et al., 2019a) and COMET (Stewart et al., 2020)).
Threshold free analysis. In Tab. 2, we report the correlations between OOD scores and quality met- rics each data subset (IN and OUT distribution, and ALL combined). For the OOD detector to im- prove or maintain performance on the end task, its score must correlate with performance metrics sim- ilarly for each subset. We notice that not the case for the likelihood or @MSP. The highest likeli- hood on IN data corresponds to higher quality an- swers. Still, the opposite is true for OOD samples, meaning using the likelihood to remove OOD sam- ples tends to remove OOD samples that are well handled by the model. By contrast, RAINPROOF scores correlate well and in the same way on both IN and OUT, allowing them to remove OOD sam- ples while improving performance.
5.1 Execution time
5.3 Towards an interpretable decision
Runtime and memory costs. We report in Tab. 4 the runtime of all methods. Detectors for so are faster than the ones for sf. Unlike detectors using references, no-reference detectors do not require additional memory. They can be set up easily in a plug&play manner at virtually no costs.
An important dimension of fostering adoption is the ability to verify the decision taken by the au- tomatic system. RAINPROOF offers a step in this direction when used with references: for each in- put sample, RAINPROOF finds the closest sample

---

Table 5: Detailed impacts on NMT performance results per tasks (Domain- or Language-shifts) of the different detectors. We present results on the different parts of the data: IN data, OOD data and the combination of both, ALL. For each, we report the absolute average BLEU (Abs.), the average gains in BLEU (G.s.) compared to a setting without OOD filtering (/o only) and the share of the subset removed by the detector (R.Sh.). We provide more detailed results on each datasct in Ap. G. In addition, we performed this study using different thresholds see Sec. G.3
Domain shifts
Language shifts
IN
OOD
ALL G.
IN
00D
ALL G.
Abs.
G.
Rh.
Abs.
G.
Rh.
Alw.
Rh.
Ahs.
G
Rh.
Abs.
G.
Rb.
Abs.
Rh.
X
46.9
+0.0
0.0%
43.3
40.0
0.0%
46.2
+0.0
0.0%
60.5
4000
0.0%%
18.3
-00
0,72
50.1
400
0.0%
Ours
GFR
49.9
18.2%
+3.5
19.9%
59.7
29.9%
6p,
42.0
12.1
20.014
46.2
13.0
40.9%%
48.0
11.9
27.8%
63.2
12.7
19.85
33.6
15.3
83.2%
61.6
-11.5
35,855
19:1
+2.6
20.0%
45.5
+2.3
18.0%
48.9
+2.8
19.7%
63.9
+3.4
19.9%
18.4
-0.0
16.0%
55.2
+5.1
25.8%
Hus.
62:
50.6
43.8
19.0%
47.5
+4.3
50.9
21.1%
65.6
65.1
19.0%
22.4
-4.
6/6
59.7
+9.6
30.0%
45.6
18.8%
33.4
4.9
45.25
42.1
-.1
29.6%
58.9
-1.7
18.1%
20.01
+1.7
31.3%
50.8
10.7
22.1%
Roger
18.0
+1.2
20.0%
17.1%
20.0%%
51.6
+15
20.9%
46.9
+0.0
19.09
57.4
-5.9
63.1%
26.3
+0.1
35.00%
60.9
40.4
19.5%
24.7
+6.4
178
55.6
+5.55
29.0%
Ours
Gyn'
46.9
+0.0
18.7%
37.4
5.9
63.0%
46.3
+0.1
34.8%
60.9
19.2%
24.7
+6.4
56.2%
55.6
+5.5
28.7%
Bas
46.7
-0.1
20.0%
43.1
-0.1
62.2%
45.4
-07
36.5%
60.6
+0.1
20.0%%
20.6
+1.3
60.15
$5.3
+5.2
30.4%
Table 6: Interpretability Analysis OOD source (S.). their ground-truth (GD.), their generation (Gen.) and projections onto the reference set.
Table 7: OOD detection on BLOOMZ using German as an OOD language for the LLM
AUROC
FPR J
0.50
1,00
Ours
8.
Alir la nil sieem trchallar fins los den.
RD
0.58
0.92
Gd
Taxt night we workel until 10 p.m.
SMSP
0,59
0.90
Gon.
Ahir nit varem treballar fins les dea.
BLBU 3,75
Bas.
dg
0.51
0.97
0.54
0.90
D' (x))
Dar gato por Sebbe.
Score 1.23
0.71
0.80
Qurs
S
Austillia no és Austria.
51
OFR
0.70
Australia Ausria
Bas.
0.66
Gen.
Austrilia is not Austiia
BLEU
late Tatoeba dataset samples into English, focusing on languages known to be within the distribution for BLOOMZ while attempting separate the Ger- man samples from them. From Tab. 7, we observed that our no-reference methods perform comparably to the aMsP baseline, but are outperformed by the Mahalanobis distance in this scenario. However, the information projection methods demonstrate substantial improvements over all the baselines.
p" (xx)
La vida no es ficil.
Score 0.82
(in the sense of the Information Projection) in the reference set to take its decision. Tab. 6 present examples of OOD samples along with their trans- lation scores, projection scores, and projection on the reference set. Qualitative analysis shows that, in general, sentences close to the reference set and whose projection has a close meaning are better handled by fe. Therefore, one can visually inter- pret the prediction of RAINPROOF and validate it.
7 Conclusion
This work introduces a detection framework called RAINPROOF and a new benchmark called LOFTER for black-box OOD detection on text generator. We adopt an operational perspective by not only con- sidering OOD performance but also task-specific metrics: despite the good results obtained in pure OOD detection, OOD filtering can harm the per- formance of the final system, as it is the case for OMSP or a... We found that RAINPROOF succeed in removing OOD while inducing significant gains in translation performance both on OOD samples and in general. In conclusion, this work paves the way for developing text- -eneertttntion OOD detectors and calls for global evaluation when benchmarking future OOD detectors.
6 RAINPROOF on LLM for NMT
As alternative to NMT models, can study the performance of instruction finetuned LLM on translation tasks. However, it is important note that while LLMS are trained on enormous amounts of data, they still miss many languages. Typically, they are trained on around 100 languages (Conneau et al., 2019), this falls far short of the existing 7000 languages. In our test-bed experiments, we decided to rely BLOOM models, which have not been specifically trained on German (DEU) data. There- fore, We can use German samples simulate OOD translation
detection in an Instruction-followii setting, specifically relying on BLOOMZ (Muen- nighoff et al., 2022). We prompt the model to trans-

---

8 Limitations
While this work does not bear significant ethical or impact hazards, it is worth pointing out that is not perfect, absolutely safe solution against OOD distribution samples. Preventing the processing of OOD samples is important part of ensuring ML algorithms' safety and robustness but it cannot guarantee total safety nor avoid all OOD samples. this work, we approach the problem of OOD detection from performance standpoint: we argue that OOD detectors should increase performance metries since they should remove risky samples. However, no one can give such guarantees, and the outputs of ML models should always be taken with caution, whatever safety measures filters are in place. Additionally, we showed that our methods worked in specific setting of language shifts or topic shifts, mainly on translation tasks. While our methods performed well for small lan- guage shifts (shifts induced by linguistically close languages) and showed promising results on detect- ing topic shifts, the latter task remains particularly hard. Further work should explore different types of distribution shifts in other newer settings such different types of instructions or problems given instructionfiietuuee models.

---

References
Geli Fei and Bing Liu. 2016. Breaking the closed world assumption in text classification. In Proceedings of the 2016 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, pages 506-514.
Udit Arora, William Huang, and He He. 2021. Types of out-of-distribution texts and how to detect them. arXiv preprini urXiv22109.06227,
Aram Bahrini, Mohammadsadra Khamoshifar, Hos- sein Abbasimehr, Robert J. Riggs, Maryam Esmacili, Rastin Mastali Majdabadkohne, and Morteza Pase- hvar. 2023, Chatgpt: Applications, opportunities, and threats.
Eduardo Dadalto Camara Gomes, Florence Al- berge. Pierre Duhamel, and Pablo Piantanida. 2022. Igeood: An information geometry approach to out-of-distrrbuttinn detection. arXiv preprint arXiv:2203.07798.
Leon Brillouin. 1953. The negentropy prineiple in- formation. Journal of Applied Physics, 24(9):1152- 1163.
Matan Haroush, Tzviel Frostig, Ruth Heller, and Daniel Soudry. 2021. A statistical framework for efficient out of distribution detection in deep neural networks.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 13:1877-1
Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. 2019. Why relu networks yield high- confidence predictions far away from the train- ing data and how mitigate the problem. 2019 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 41 1--50.
Emile Chapuis. Pierre Colombo, Matteo Manica, Matthicu Labeau, and Chloé Clavel. 2020. Hier- archical pre-training for sequence labelling in spoken dialog. In Findings of the Association for Computa- tional Linguistics: EMNLP 2020, pages 2636-2648, Online. Association for Computational Linguistics.
Ernst Hellinger. 1909. Neue begründung der theorie
guadratischer formen unendlichvielen veränder- lichen. Journal für die reine und angewandte Mathe- matik, 1909(136)2210-27
Dan Hendrycks and Kevin Gimpel. 2016. A baseline for detecting misclassified and out-of-distribution examples neural networks. arXiv preprint 16Xiv:1610.02136.
Pierre Colombo, Eduardo D. C. Gomes, Guillaume Staerman, Nathan Noiry, and Pablo Piantanida. 2022a. Beyond mahalanobis-based scores for tex- tual ood detection.
Dan Hendrycks and Kevin Gimpel. 2017 A baseline for detecting misclassified and out-of aultof-distribution exam- ples neural networks. International Conference on Learning Representations.
Pierre Jean A Colombo, Chloé Clavel, and Pablo Pi- antanida. 2022b. Infolm: new metric to evaluate summarization & data2text generation. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 36, pages 10554-10562.
Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. 2020. Generalized odin: Detecting out-of- distribution image without learning from out-of- distribution data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pages 10951-10960.
Alexis Conncau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Out, Luke Zettle- moyer, and Veselin Stoyanov. 2019. Unsupervised cross-lingual representation learning at scale. arXiv preprint 1rXiv:1911.02111
Haiwen Huang, Zhihan Li, Lulu Wang, Sishuo Chen, Bin Dong, and Xinyu Zhou. 2020. Feature space singularity for out-of-distribution detection. arXiv preprint 2011:20111.145:.
I. Csiszar and F. Matus. 2003, Information projections revisited. IEEE Transactions Information Theory, 49464:14
Edwin T Jaynes. 1957. Information theory and statisti- cal mechanics. Physical review, 106(4):620.
Imre Csiszár. 1967. Information-tye measures of dif- ference of probability distributions and indirect ob- servation. studia scientiarum Mathematicarum Hun- garica, 2:229-318.
John F Kelley, 1984. An iterative design methodology for iiser-fiendll natural language office information applications. ACM Transactions on Information Sys- tems (TOIS), 2(1):26-41.
Imre Csiszár. 1975. I-divergence geometry of proba- bility distributions and minimization problems. The annals of probability, pages 146-158.
Solomon Kullback. 1954. Information theory and statis- tics. Courier Corporation.
Imre Csiszár. 1984. Sanov property, generalized i- projection and conditional limit theorem. The An- nals of Probability, pages 768-793.
Solomon Kullback. 1959. Information Theory and Statistics. John Wiley.

---

Kimin Lee. Kibok Lee, Honglak Lee, and Jinwoo Shin. 2018a. A simple unified framework for detecting out- of-distribution samples and adversarial attacks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Resa-Bianchi and R. Garnett, editors, Advances in Neural Information Processing Systems 31, pages 7167-7177. Curran Associates, Inc.
Marine Picot, Francisco Messina, Malik Boudiaf. Fab- rice Labeau, Ismail Ben Ayed, and Pablo Piantanida. 2022. Adversarial robustness via fisher-rao regular- ization. IEEE Transactions on Pattern Analysts and Machine Intelligence.
Alexander Podolskly, Dmitry Lipin, Andrey Bout, Eka- terina Artemova, and Irina Piontkovskaya, 2021. Revisiting mahalanobis distance for transformer- based out-of-domain detection. arXiv preprint arXiv:2101.03775
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. 2018b. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in neural information processing systems, 31.
Soujanya Poria, Devamanyu Hazarika, Navonil Ma- jumder, Gautam Naik, Erik Cambria, and Rada Mi- halcea. 2018. Meld: A multimodal multi-part dataset for emotion recognition in conversations. arXiv preprint arXiv: 1810.02500.
Yanran Li, Hui Su. Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017. Dailydialog: A manually labelled multi-turn dialogue dataset. In Proceedings of The 8th International Joint Conference on Natural Language Processing (JJCNLP 2017).
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018. Improving language under- standing by generative pre-training.
Shiyu Liang, Yixuan Li, and R. Srikant. 2018. Enhane- ing the reliability of out-of-distribution image detec- tion in neural networks. In International Conference on Learning Representations.
C Radhakrishna Rao. 1992. Information and the accu- racy attainable in the estimation of statistical parame- ters. Breakthroughs in statistics, pages 235-247. Springer.
Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. 2020. Energy-based out-o--distribution detection. Advances in Neural Information Processing Systems.
Jie Ren, Stanislav Fort, Jeremiah Liu, Abhijit Guha Roy. Shreyas Padhy, and Balaji Lakshminarayanan. 2021a. A simple fix mahalanobis distance for improving near-ood detection. arXiv preprint
Clara Meister, Tiago Pimentel, Gian Wiher, and Ryan Cotterell. 2023. Locally Typical Sampling. Transac- tions of the Association for Computational Linguis- tics, 11 11:10-1221.
Jie Ren, Stanislav Fort, Jeremiah Liu, Abhijit Guha Roy. Shreyas Padhy, and Balaji Lakshminarayanan 2021b. A simple fix to mahalanobis distance for improving near-ood detection.
Yifei Ming. Yiyou Sun, Ousmane Dia, and Yixuan Li. 2022. Cider: Exploiting hyperspherical embed- dings for oft-s--iisttiuutiin detection, arXiv preprint arXiv:2203.04433
Seonghan Ryu, Seokhwan Kim, Junhwi Choi, Hwanjo Yu. and Gary Geunbae Lee. 2017. Neural sentence embedding using only in-domain sentences for out- of-domain sentence detection in dialog systems. Pat- tern Recognition Letters, 88:26-32.
Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng-Xin -in Yong, Hailey Schoelkopf, et al. 2022. Crosslingual generaliza- tion through multitask finetuning. arXiv preprint arXiv:2211.01786
Ivan N Sanov. 1958. On the probability of large devia- tions of random variables. United States Air Force, Office of Scientific Research.
Nathan Ng, Kyra Yee, Alexei Baevski, Myle Ott, Michael Auli, and Sergey Edunov. 2020. Facebook fair's wmt19 news translation task submission. In Proc. of WMT.
Chandramouli Shama Sastry and Sageev Oore. 2020. Detecting out-oo-disttibution examples with Gram matrices. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 8491 1491- PMLR.
OpenAL. 2023. Gpt-4 technical report.
Kishore Papineni. Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: method for automatic evalu- ation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Compu- tational Linguistics, pages 31 1-118, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
Teven Le Scao, Angela Fan, Christopher Akiki, El- lie Pavlick, Suzana Iliè, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. 2022. Bloom: A 176b- parameter open-access multilingual language model. arXiv preprint arXiv:221 1.05100..
Elizabeth Shriberg, Raj Dhillon, Sonali Bhagat, Jeremy Ang, and Hannah Carvey. 2004. The iesi meeting recorder dialog act (mrda) corpus. Technical report, INTERNATIONAL COMPUTER SCIENCE INST BERKELEY CA.
Jitendra Parmar, Satyendra Singh Chouhan, Vaskar Ray- choudhury, and Santosh Singh Rathore. 2021. Open- world machine learning: Applications, challenges, and opportunities.

---

Craig Stewart, Ricardo Rei, Catarina Farinha, and Alon Lavie. 2020. COMET deploying a new state-of- the-art MT evaluation metrie in production. In Pro- ceedings of the 14th Conference of the Association for Machine Translation in the Americas ( (Volume 2: User Track), pages 78-109, Virtual. Association for Machine Translation in the Americas.
Tim Z. Xiao, Aidan N. Gomez, and Yarin Gal. 2020. Wat zei je? detecting with variational transformers.
translations
Xiaoxue Zang, Abhinav Rastogi, Srinivas Sunkara, Raghav Gupta, Jianguo Zhang, and Jindong Chen. 2020. Multiwoz 2.2: A dialogue dataset with addi- tional annotation corrections and state tracking base- lines. In Proceedings of the 2nd Workshop on Natu- ral Language Processing for Conversational AI, ACL 2020, pages 109-117.
Andreas Stoleke, Klaus Ries, Noah Coccaro, Elizaheth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Tay- lor, Rachel Martin, Marie Meteer, and Carol Van Ess-Dykema. 2000. Dialogue act modeling for au- tomatic tagging and recognition of conversational speech. Computational Linguistics, 26(3):339-37)
Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. Advances in neural information processing systems, 32.
NLLB Team, Marta R. Costa-jusså, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Hef- fernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Bar- rault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman. Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan. Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Safiyaah Saleem, Holger Sehwenk, and Jeff Wang. 2022. No language left behind: Scaling human- centered machine translation.
Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Pe- ter Liu. 2020. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. In In- ternational Conference on Machine Learning, pages 11328-11339 PMLR.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Wein- berger, and Yoav Artzi. 2019a. Bertscore: Eval- uating text generation with bert. arXiv preprint arXiv: N.Kiv:99040067..
Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2019b. Dialogpt: Large-scale generative pre-training for conversational response generation. arXiv preprint arXiv: 191 000536.
Jörg Tiedemann and Santhosh Thottingal. 2020. OPUS-
MT Building open translation services for the World. In Proceedings of the 22nd Annual Confer- enec of the European Association for Machine Trans- lation (EAMT), Lisbon, Portugal.
Wenxuan Zhou, Fangyu Liu, and Muhao Chen. 2021. Contrastive -uitof-distiibttion detection for pre- trained transformers. In Proceedings of the 2021 Conference on Empirical Methods in Natural Lan- guage Processing. Association for Computational Linguistics.
Jörg Tiedemann. 2012. Parallel data, tools and inter- faces in opus, In Proceedings of the Eight Inter- national Conference on Language Resources and Evaluation (LREC'12), Istanbul, Turkey. European Language Resources Association (ELRA).
Zhi-Hua Zhou. 2022. Open-environment machine leamn ing. National Science Review, 98)): 123.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models.
Qiuyu Zhu, Guohui Zheng, and Yingying Yan. 2022. Effective out-of-distribution detection in classifier based on pedcc-loss.
Samarth Tripathi, Sarthak Tripathi, and Homayoon Beigi. 2018. Multi-mode modal emotion recognition on iemocap dataset using deep learning. arXiv preprint arXiv:1804.05788.
Sachin Vernekar, Ashish Gaurav, Vahdat Abdelzad, Tay- lor Denouden, Rick Salay, and Krzysztof Czarnecki. 2019a. Out-oofdisttibutia detection in classifiers via generation.
Sachin Vernekar, Ashish Gaurav, Taylor Denouden, Buu Phan, Vahdat Abdelzad, Rick Salay, and Krzysztof Czarnccii. 2019b. Analysis of confident-clasiffees for out-of-distribution detection. arXiv preprint arXiv:1904, 12220.

---

A Examining the Limitations of Mahalanobis-Ban Based OOD Detector for Text Generation
In order to evaluate the performance of our meth- ods we will focus and report mainly the AUROC and the FPR 4, we provide more detailed metries and experiments in Sec. B.1.
Area Under the Receiver Operating Charac- teristic curve (AUROC). The Receiver Operating Characteristic curve is curve obtained by plotting the True positive rate against the False positive rate. The area under this curve is the probabil- ity that an in-distribution example Xin has anomaly score higher than an OOD sample Xout: AUROC- (r(a(Xxn) > a(Xout) )). It is given by H (Pr (a(x) > 7 Z = 0), Pr (a(x) > y Z = 1)).
0
2
0
2
(a) deu date on fra model. (b) spa date on fra model.
Figure 2: PCA reduction of encoder's hidden features for IN and OUT distribution samples, with Mahalanobis distance mean (green cross). The plot reveals the multi- modal nature of the distributions.
False Positive Rate at 95% True Positive Rate (FPR 1). We accept to allow only given false posi- tive rate corresponding to defined level of safety and we want to know what share of positive sam- ples actually catch under this constraint. It leads to select a threshold %rr such that the corresponding TPR equals r. At this threshold, one then computes: Pr(a(x) > 7r Z = 0) with 7, s.t. TPR(?+) = r. r is chosen depending on the difficulty the task at hand and the required level of safety.
The main drawback of the Mahalanobis distance is assuming a single-mode distribution. In text classification, this is mitigated by fitting one Maha- lanobis scorer per class. However, in text genera- tion, this assumption is flawed there are multiple modes as illustrated in Fig. 2). PCA of Fig. illus- trate failure case of the Mahalanobis distance in the case of OOD detection.
B Experimental setting
this section, we dive into the details and defini- tions of our experimental setting. First, we present our OOD detection performance metrics (Sec. B.I), then we provide a couple samples for one of the small language shifts (Sec. B.4). We also discuss the choices of pre-trained model (Sec. B.6) and how we adapted common OOD detectors to the text generation case (Sec. B.7).
For the sake of brevity, we present only AUROCand FPR Jmetrics in our aggregated results but we also used Detection error and Area Under the Precision-Recall curve metries and those are presented in our full results section (Ap. F).
B.1 Additionnal details on metrics
Detection error. It is simply the probability of miss-classiffcation for given True positive rate.
OOD Detection is usually unbalanced binary classification problem where the class of interest is OUT. Let us denote Z the random variable corre- sponding to actually being out of distribution. We can assess the performance of our OOD detectors focusing on the False alarm rate and on the True detection rate. The False alarm rate or False positive rate (FPR) is the proportion of samples misclassified OUT. For score threshold 7, have FPR Pr (a(x) > 7/2 Z = 0). The True detection rate True positive rate (TPR) is the proportion of OOD samples that are detected by the method. It is given by TPR- Pr (a(x) > > 1).
Area Under the Precision-Recall curve (AUPR-IWWAUPR-OUT)) The Precision- Recall curve plots the recall (true detection rate) against the precision (actual proportion of OOD amongst the predicted OOD). The area under this curve +> (Pr (Z 1 1/ sXX < Pr captures the trade-off between precision and recall made by the model. A high value represents a high precision and a high recall i.e. the detector captures most of the positive samples while having few False positives.
< 1))

---

B.2
Language pairs
B.4 Samples
In Tab. 10 we provide examples of small shifts in translation between Spanish and Catalan and its impact on spanish to english translation model.
Model
IN data
OUT data
Language shift
DEU-ENG
Tatoeba DEU
News FR
B.5 Dialog datasets
DEU-ENG
Tatoeba DEU
Tatoeba NLD
SPA-ENG
Tatoeba SPA
News FR
Switchboard Dialog Act Corpus (SwDA) is cor- pus of telephonic conversations. The corpus pro- vides labels, topic and speaker information (Stolcke et al., 2000).
SPA-ENG
Tatoeba SPA
Tatoeba CAT
SPA-ENG
Tatoeba SPA
Tatoeba POR
NLD-ENG
Tatoeba SPA
AFR
Domain shift
ICSI MRDA Corpus (MRDA) contains transcript 75h of naturally occuring meetings involving more than 50 people (Shriberg et al., 2004).
DEU-ENG
Tatoeba DEU
EMEA DEU
DEG-ENG
Tatoeba DEU
Eurparl DEU
DEU-ENG
Tatoeba DEU
EMEA DEU
DEU-ENG
Tatoeba DEU
Eurparl DEU
DaylyDialog Act Corpus (DyDA) contains daily common communications between people, cover- ing topic such as small talk, meteo or daily activi- ties (Li et al., 2017).
Table Summary models and studied shifts.
B.3 Dataset sizes
Interactive Emotional Dyadic Motion Capture IEMOCAP)(Triathi et al., 2018) consists of tran- scripts of improvisations scripted scenarii sup- posed outline the expression of emotions.
Dataset Name
Size
Tatoeba
1373
Tattoeba CAT
1630
B.6 Choices of models
Tatoeba DEU
3000
To perform experiments needed models that were already well installed and deployed and that would also support OOD settings. For translation tasks, we needed specialized models for a notion of OOD to be easily defined. It would be indeed more hazardous to define a notion of OOD language when working with multilingual model. The same is true for conversational models.
Tatoeba NLD
3000
Tatoeba POR
3000
Tatoebamt ES
3000
newscommentary DE
3000
newscomeentary ES
3000
newscomeentrry
6000
newscomentary NL
3000
amazonreviewsmullt ti DE
3000
Neural Machine Translation model. We bench- mark our OOD method on translation models pro- vided by Helsinky NLP (Tiedemann and Thottingal, 2020) on several pairs of languages with large and small shifts. We extended the experiment to detect domain shifts. These models are indeed specialized in each language pair and are widely recognised in the neural machine translation field. For our periments we used the testing set provided along these models, so can consider that they have been fine-tuned over the same distribution.
amazonnevviewsmultt ES
3000
dailydialog default t
3000
europar1bilingual DE
3000
europar1bilingual
ES
3000
EMEA DE
3000
EMEA ES
3000
EMEA
3000
multiwoz
3000
silicone dydae
3000
silicone iemocap
805
silicone maptask
2963
Conversational model. We used a dialog- GPT (Zhang et al., 2019b) model fine-tuned on the Multi WOZ dataset as chat bot model. The fine- tuning on daily dialogue-type tasks ensures that get
silicone melds
1109
silicone mrda
3000
silicone oasis
1513
silicone
485
the model is specialized, thus allowing good definition of samples not being in its range of expertise. Moreover, the choice of the architecture, DialogGPT, guarantecs that our results are valid on a very common architecture.
silicone swda
3000
Table 9: Number of samples in each (test) datasets

---

Source sentencee
Expected translation
Translation
BLEU
Tom sgrads tccnologia.
technology.
Tom likes technology. Aci está la teua bossa.
42.73
Acf esta la toua bossa.
Here is your bag.
8.12
et posarà en perill.
put you in danger.
posari perill.
A Londres hi han molts pares bonics.
There are many beautiful purks in Lendon.
To London hi han molts purcs bonics.
6.57
Aquest pa
deliciós.
bread very delicious.
Aquest molt deliciós.
A loss els meus amics els agraden els videojocs.
All my friends like playing videogumes.
A tots els meus umics els agrade els videojocs.
4.20
Agó un peix.
Moltus ielicitats!
Congratulations!
Moltes congrats!
27.52
any
Happy New Year!
Bon any
15.97
Aquell que menteis, robarā.
He that will lic, will steal.
The one who's is mindless, he' steal. clau.
12:22
clau.
who
key.
see qui
En Tom surt trebular cada mati dos quarts de set. que seva casa embruixada.
Tom leaves for work 6:30 every moming.
In Tom surt to pull each matt to two quarts of set.
3.67
cold
house haunted.
mi dit that Aquest is the lloe on néixer el men pare.
haunted.
27.78
Aquest és el lloe on va néixer el meu pare.
This the place where my father was born.
8.30
Table 10: Example of behaviours of language model trained to handle Spanish inputs on Catalan inputs.
B.7 Generalization of existing OOD detectors to Sequence Generation
given conditioning x and temperature T we de- fine the average energy of the sequence:aE(x) Σ log Σ to the normalization term of the softmax function applied on the logits. While it takes into account the whole distribution, it only takes into account the amount of unormalized mass before normalization without attention to how this mass is distributed along the features.
[O] ef((x& It corresponds
this section, we extend classical OOD detection score to the conditional text generation settting. Common OOD detectors were built for classifica- tion tasks and we need to adapt them to conditional text generation. Our task can be viewed as a se- quence of classification problems with very large number of classes (the size of the vocabulary). We chose the most naive approach which consists of averaging the OOD scores over the sequence. We experimented with other aggregation such as the min/max or the standard deviation without getting interesting results.
Mahalanobis distance Following (Lee et al., 201 8a; Colombo et al., 2022a) compute the Ma- halanobis matrice based on the samples of a given reference set R. In our case we are using encoder- decoder models use the output of the last hidden layer of the encoder embedding. Let's denote <(x) this embedding for conditioning x. Let μ and Σ be respectively, the mean and the covariance of these embedding on the reference set. We define (M(x) (1 + (ø(x) - 4) TT==((((x)) -4))
Likelihood Score The most naive approach to build OOD score is to rely solely on the log- likelihood of the sequence. For conditioning we define the log-likelyhood score by (x) |x, &e)). The likelihood is
- log pe(S: logpooSs the same as the perplexity.
1-11
B.8 Computational budget
Average Maximum Softmax Probability score The maximum softmax probability (Hendrycks and Gimpel, 2017) takes the probability of the mode of the categorical distribution as score of OOD, We extend thise definition in the case of sequence of probability distribution by averaging this score along the sequence. For given conditioning we define the average MSP score (x)p(x) max PI (ilx, While it is closely ic|o.K] linked to uncertainty measures it discards most of the information contained in the probability distri- bution. It discards the whole probability distribu- tion. We claim that much more information can be retrieve by studying the whole distribution.
We had budget of 20000h on NVIDIA V GPU. While this is important number it used compute the benchmarks over many pairs and lan- guages. In practice our OOD detectors do not re- quire much addition computation overhead since they only rely on the probability distributions al- ready output by the models.
B.9 Towards an interpretable decision
M
An important dimension of fostering adoption is the ability to verify the decision taken by the automatic system. RAINPROOF offers step in this direction when used with references: for each input sample, RAINPROOF finds the closest sample (in the sense of the Information Projection) in the reference set to take decision. We present in Tab. some OOD samples along with their translation scores, projection scores, and their projection on the refer-
Average Energy score We extend the definition of the energy score described in (Liu et al., 2020) to a sequence of probability distributions by av- eraging the score along the sequence. For a

---

Table 11: OOD inputs, their translations and projections onto the reference set. The first 2 are far from the refer- ence set and not well translated whereas the next 2 are very close the reference set and well translated. We for that matter, notice that the projection is quite close to the input sentence grammatically speaking.
AUROC
FPR 4
AUPR-IN
AUPR-OUTT
Err
Ours
4UR
0.57
0.93
0.50
0.64
0.50
0.58
0.92
0.52
0.63
0.51
dMSP
0.59
0.90
0.64
0.52
0.42
Bas.
ag
0.51
0.97
0.47
0.55
0.57
0z
0.54
0.90
0.44
0.62
0.53
0.71
0.82
0.73
0.64
0.39
Scene
Altin la skone neballl the Four sight eoeked wedil i ocballac deu.
0.70
0.82
0.73
0.64
0.40
Graed
Bas.
0.66
0.89
0.74
0.56
0.42
[*(x)
goto por
Table 13: OOD detection on BloomZ using German an OOD language for instruction model.
Scano
Agpesia oole lhe ellorrd no to
This ool has hus to this andi dusan't sver any good
Gearusl
This tall eicoswn no grot.
BLEU-400
cochass dent.
Score!
Joeme
Aspinda de inti mack cormaya.
Greed-auth wuth
tio way Arange lenix
C.1 Negative results on NLLB
Grwrse
iss card
FIF 2-37
p*(x)
chopato
By the very definition of the No-Language-Leff Behind model, it should be particularly hard to find OOD language to benchmark on. The model still requires special token to be set in the sequence define the source and target languages. We tried to apply our OOD detection methods to situations where the presented language does not correspond to the source language set by the special token. We found that in this scenario the likelihood was by far the best discriminator of OOD samples. It can be explained by the fact that our inputs not actually OOD, they are just consistent with the source language token, but the model is still well calibrated overall on these inputs.
Asersiti Auaros
Grazd-iaa Insh
Auaralia imn Aoaet..
Grwose
Aneriilla not Auanta
F1 M
p((x)
La vab no pu
ence set. We notice that, in general, sentences that are close the reference set, and whose projec- tion has a close meaning, are better handled by fo- Therefore, one can visually interpret the prediction of RAINPROOF, and validate it. This observation further validates our method.
C Scaling to larger models
AUROC
FPR | AUPR-IN
AUPR-OUT
Err
Ours
AUR
0.50
1.00
0.75
0.75
0.51
dps
0.57
0.95
0.60
0.62
0.50
dySP
0.71
0.80
0.71
0.68
0.42
In order to validate our results we perform periments on larger and general-purpose mod- els such as BloomZ (Muennighoff al., 2022), NLLB (Team et al., 2022) and the Facebook WMT16 submission (Ng et al., 2020).
Bas.
0g
0.53
0.97
0.54
0.50
0.51
0g.
0.80
0.59
0.78
0.79
0.32
Ours
Oyx*
0.54
0.93
0.57
0.49
0.46
AM
0.62
0.67
0.54
0.42
Table 14: Performance OOD detection for NLLB,
AUROC
FPR [
AUPR-IN
AUPR-OUT
Err
D Additional OOD features-based
Ours
0.72
0.80
0.54
0,79
0.51
baselines
Bas.
acgg
0.65
0.95
0.74
0.59
0.52
0.47
0.89
0.59
0.50
0.43
Ours.
0ng
0.74
0.87
0.80
0.68
0.34
To further support the point that features-based
Op
0.74
0.88
0.80
0.67
0.35
AC
0.70
0.90
0.80
0.52
0.38
detectors have important flaws when it comes to
Bas.
0.64
0.89
0.54
0.67
0.53
text generation we compare our best performing
0.62
0.81
0.66
0.55
0.38
OOD score SOTA OOD detectors in text such
Table 12: Performance in OOD detection on large trans- lation model Facebook WMT-19 submission. (RUS-
the DataDepth (ap) (Colombo al., 2022a) and
the Maximum Cosine Projection (ac) (Zhou et al., 2021).
DEU).

---

AUROC
FPR
AUPR-IN
AUPR-OUT
F Performance of our detectors in OOD
Ours
App
0.85
0.66
0.69
0.91
0.47
detection
90
Bas.
ACDE
0,76
0.51
0.78
0.71
0.20
0.81
0.55
0.86
0.69
0.20
0.61
0.95
0.82
0.36
0.32
F1
Importance of tails' distributions
Bas.
0.57
0.93
0.35
0.75
0.67
0.70
0.80
0.46
0.84
0.58
0.7
Table 15: Comparison of our best detector a against SOTA features based-ood detectors on close language shifts.
630
622
1250
AUROC
1620
0.4
E Parameters tuning
Figure 5: Impact of on the performance of the Rényi information projection for dialog shifts detection. A smaller a increases the weight of the tail of the distri- bution. An a of O would consist in counting the number of the common non zero elements.
Detectors depend on their anomaly score to make decisions, and these scores can be parametric. First of all, soft probability-basce -bsed scores depend on the soft probability distribution and its scaling. There- fore, the temperature a crucial parameter tune to get the most performance. While a small tem- perature makes the distribution pickier, a higher value spreads the probability mass along the classes. Moreover, the Rényi divergence depends on a fac- tor a. We provide here further results and analysis of those parameters our results.
Our results show that, when it comes to domain shift (domain shifts in translation or dialog shifts), reference- detectors good results. They also show that, the more these detectors take into account the tail of the distribu- tions, the better they are, as displayed in Fig. 5. We find that low values a (near 0) yields bet- ter results with the Rényi Information projection aD; It suggests that the tail of the distributions used during text generation carries context infor- mation and insights on the processed texts. Such results are consistent with findings of recent works in the context of automatic evaluation of text gen- eration (Colombo et al., 2022b).
required to obtain
In Fig. 3, analyse the impact of the temper- ature and a parameter for our Senyi-Negentropy score. Consistently with results for the information projection we find that the tail the distribution is important to ensure good detection of OOD sam- ples for all language shifts. A temperature higher than and lower values of a yield the best results. We recommend using = 0.5 with a temperature of 2.
F22 Summary of our results
In Fig. 6 we present the different performance lev- els all the detectors we studied. We can see that in every task our detectors outperform the baselines but also that in dialog shift, while the Mahalanobis distance outperform clearly our detectors for so, they still outperform baselines for their scenario by far.
We found that our aD, score, the Rényi negen- tropy more stable concerning the temperature and the considered datasets and shifts than the energy- based OOD score and the MSP score. Indeed, in Fig. 4, we show that the baselines do not behave consistently across datasets when the temperature changes. This is problem when deploying these scores in production. Indeed, we cannot fit tem- perature for each possible type of shift OOD samples. By contrast, there exist sets of parame- ters (temperature and a) for which our negentropy- based scores perform consistently across different shifts.
F.3 Detailed results of OOD detection performances
In this section, present the performances of our OOD detectors on each detailed tasks, i.e. for each pair of IN and OOD data with all the considered metrics. Our metrics outperform other OOD detec- tors baselines almost all scenarios.

---

02
06
003
0.5
385
13
98
1.
370
15
1.3
15
13
16
1.9
17
1.7
18
155
1.9
1.7
220
050 A4
200m "A "Rou
AN X"
Tempenttur
Teppeahier
Tenperature
Timpersooe
(a) CAT
SPA-POR
(c) NLD-AFR
(d) SPA-FRA
Figure 3: Effect of the temperature and parameter for an, on the performance on OOD detection terms of AUROC.
Table 16: Detailed results of the performances of our OOD detectors on different language shifts, The first language of the pair is the reference language of the model and the second one is the studied shift.
Aime-20t
11M
jessiáoo
rocall
XUMIC
prodiao
Score
1.00
0.83
0.71
0.30
0.41
1.00
01.99
0.53
IL.71
0.80
0.95
0.37
01.66
0.57
1197
ITB
U33
ILTT
0501
033
11.1
1138
1.01
Hoaditres
0.98
0.384
0.98
11:96
11:78
11:8
012
015
0100
ILT1
0.55
0.6
0:51
0.94
0.90
0.70
01:0
0187
0188
0.52
1.31
1.01
0.50
0.26
0.62
0.50
0.34
0.57
0.42
0.29
0.21
0.76
0.51
0.80
0.25
1.00
0.25
0.27
1.0
130
1.0
0.62
0.53
0100
[LT]
L10
0.28
0337
0.30
0.75
0.30
0.91
0.44
0.57
0.96
0.99
0.50
0.05
0.53
0.05
0.71
0.50
0.39
0.85
0.66
0.74
0.43
0.53
0.59
0.48
Oars
0.99
1.00
0.98
1.00
01.99
1.00
0.62
0.62
0.53
0.5
0.00
0.00
0.71
0.71
0.90
0.50
Outs
0:45
0.58
0.79
0.89
0.63
0.77
0.74
0.79
0.54
0.35
0.82
1.00
0.47
0.41
0.66
0.50
1.00
1.00
0.62
0.53
0.00
0.71
1.0
0.45
0.85
0.99
0.74
0.46
0.52
0.52
0.99
0.97
0.98
0.62
0.52
0.01
0.71
0.58
0.41
0.76
0.56
0.79
0.00
0.59
0.00
0.00
Booelizes
0.30
0.66
0.24
0.11
1100
0183
0.35
1164
011
0.5
1196
0.47
1142
0.62
1166
Busclines
0.31
0:39
0331
0.89
0703
0.21
0..5
0,66
11:01
0.46
0881
0130
0.35
0.4
0.0
0.31
(a) deu-nld
(b) spa-cat
Scenario
Score
Scenati
Oare
0.64
0.67
0.94
0.96
0.63
0.87
0.55
0.58
0.55
0.55
0.67
0.70
0.45
0.45
0.72
1.00
Our
0.97
8.99
0.99
1,00
0.99
1.00
0.04
0.02
0.53
0.83
0..1
0.04
0.71
0.71
1.00
1.00
010
0.34
11.44
01.22
0.37
10.14
11.40
1.00
0.97
037
0.56
0116
0352
0.00
0.71
1.00
Bonditzes
0.1
8.66
0394
0.44
0183
1184
10:2
11.5
01:1
0:31
1.75
1.79
0L:99
11:19
1.80
1.00
Breckanx
0.65
0.95
0.41
0.90
0.99
18:77
0.03
08:98
13611
0883
0.06
018
(1:0
0,71
L.0)
0.29
0.83
0.58
0.76
0:27
0.00
0.25
0.36
0.48
0.85
0.68
0.38
0.37
0.55
0.42
0.55
M00
0.45
034
01.70
1..7
0037
0.37
00.00
1.00
1,00
1.00
0.12
0,33
00000
0.71
1.00
0.37
0.94
0.75
0.69
0.40
0.59
0.35
0.47
0.98
1
0.02
0.33
0.01
1.00
Oars
0.20
0.66
0.93
0.82
0.83
0.55
0,50
01.99
0.00
0.39
0.84
0.97
n.34
0.00
I.46
0.00
Ores
1,00
1,00
1
1,00
1.00
1.00
0.02
0,01
0,83
0.53
0.000
0.00
0.71
0.71
0.50
1.00
0gge
0.47
0.54
0.78
0.47
0.46
0.57
0.30
0.56
1,00
1
1.00
0.02
0.53
0.00
0.71
1.00
Oygene
0.52
0.90
0.75
0.74
0.00
0.50
0.00
0.00
Oygens
0.99
0.99
0.99
0.02
0.00
0.50
Roodlines
0.23
0.38
0.30
01.8
0.74
1117
07.9
0.65
0123
0.39
11100
0.33
0.64
0.73
0401
0101
016
0.29
0131
0197
0.65
03:5
0.30
0/00
(c) nld-afr
(d) deu-fra
NuTs IN
AUTR
ALSOC
15R
(1
TTR:
pestidon muall
AUTe:
Jure ouT
AUROC
pratuiis naall
Somuaio
Soarz
Socmatio
Score
Oar
um
90,
0.99
1.00
1.00
1.00
1.00
1.00
10,0
0.02
0.53
0.53
0,00
0.01
(L.71
0L.71
1.50
1.00
Our
epn
96.
0.91
8.90
8.97
6:97
0.95
0.55
0.20
0.11
0,80
0,80
0.36
0.27
0.70
0.70
0.45
1.00
0.36g
0.30
01.50
11.02
0133
10.00
11.11
1.00
46
0.71
0351
03.30
0.58
13:4
0335
13022
1.107
Roselines
nuse
0r.
0.97
0.53
0.83
0.73
01590
883
0:55
ILT1
0154
1150
Roor
0r.
0.64
0.67
0.66
0.63
0.27
0.58
0.58
0880
088
1.10
1.10
sloge
0.87
01.84
8130
0150
1150
0.7
0.87
1US
(LT1
0.24
0.31
0.60
037
0.66
40%
0.97
0L71
0.50
agg-
0.71
appm
0.57
0.2
ogg
agge
0U8
(LT1
app:
0.22
0.81
0.0
038
0.86
agpae
0.2
0.71
aypme
0.48
0.60
M0
0.37
BacInes
(c) spa-fra
(f) sps-ppppp por-

---

Table 17: Detailed results of the performances of our OOD detectors on different domain shifts. For Spanish (spa) and German (de), present two domains shifts: Technical medical (EMEA) data and legal parlementary texts (parl) against common language emboddied by the Tatoeba dataset (tat),
Scenario
Score
Scevari
Oara
0.57
0.90
0.73
0.76
0.51
0.06
0.43
0.46
0.77
0.51
0.62
0.56
0.79
0.50
0.75
1.00
Our
0/66
0.75
0.67
0:75
0.76
0.70
0.44
0.41
0.87
0.45
0.64
0.76
0.87
0.64
0.55
1.00
0333
0.75
11:3
11:05
1115
1195
11:19
1,00
0.75
0.75
0.71
0.04
010
0133
01/7
[10
Beoolines
0.86
0.73
01:33
0148
8.36
0.91
0:77
0.74
Basslines
0.68
0.61
0.68
0.30
0.67
0.90
0.50
((])
0138
0.76
01855
11:6
8:79
11:0
1:00
0.75
6:75
1871
0.05
13.7
038
(1:1
1101
0.3
0388
16:01
1125
1:48
I181
1136
0/6
0A1
1168
01.8
1100
0B8
1141
0.89
0.83
0.88
0833
0.52
0.62
0.80
0.53
0.67
066
0.67
0.48
0.00
0.90
0.00
0.00
Oars
Rgge
0.88
0.87
0.23
0.73
01884
0.83
1147
0..6
0.38
0.77
0.89
0.91
0.39
0.79
I.77
0.75
Oues
0.66
0.67
0.6
6,7
0.70
0.99
0.64
0.45
0.00
0.00
0882
0.85
0.00
0.00
0.00
0.00
0888
0.70
11:00
11:00
1136
1196
11:77
IL722
0.62
0.44
0868
03.8
(1101
1185
11181
01411
Dyges
0.86
0.67
0.50
0.53
0.75
0.96
0.78
0.72
Opyen
0.65
4.7
0.0
0.43
0.00
0,80
0.80
0.0
Roodires
PM
60
0.87
0.33
0.39
11:01
11:0
11.5
11.00
011
01.0
0.80
018
1.00
Rostlines
100
0g1
057
0.7
0.6
070
0.06
0701
0.30
0.94
01001
0227
0132
008
01001
0227
0.19
0101
(a) dou:noss-FMFA
(b) spa.news-par
Nins IN
AUTR OUT
ALSOC
ESR
11
7TR
pectidto rualll
AUTR:
AUTS oUT
AJROC
praticies nozll
Somuatio
Searz
Soamatio
Score
Oara
Dry
0.61
0.75
0.65
0.75
01:65
11.71
0.45
11.41
11.07
0.42
0.86
01.61
11:06
0.32
1.201
Our
Oyp
0.29
8.92
0.75
8:01
0.85
0.891
0,.77
0.64
0.79
13:73
11.71
0.82
11:11
0.80
0.75
1141
Og
0.75
0.75
01.05
11:55
11.01
10.5
11.66
1.00
0.30
0077
03:00
03.00
13:30
01.55
13:30
1.007
Beoelines
0g
0163
0.58
01:68
0.67
0.96
0:50
Bootlines
0.86
0.73
0.82
0..0
08.66
0.80
0.77
0.74
0.75
01:08
11:7
0::1
0.80
0.80
0.41
0.77
1L.8
0130
[130
0.81
023
0.38
0.29
11.44
01:0
11:8
0.00
0.00
I.:0
0.99
0.31
0.32
0.39
0.22
Oare
orggle
11:05
0105
0.63
11.8
1168
8:00
8100
0100
0.00
0.00
0.00
oege
8.90
0.081
0.77
0.71
0856
0.98
0.33
0.79
0083
0088
01501
0.75
0.79
0183
11:6
810
1136
0100
1.00
dpn!
0.31
0.6
11:8
0.:90
0.96
0.85
0.67
0.00
0.00
0.87
0.50
0.49
0.96
0.78
0.72
60
0.39
0132
Bacine
0.81
0.39
0.31
0.72
(c) dewnnews-parl
(d) spw.ness-
Pai
Por
G NTM performance
2.0
Surprisingly we show that common OOD detectors tend to exclude samples that the model well handles and keep some that are not leading to decreasing overall performance in terms of translation met- rics. Moreover, it seems this phenomenon is more dominant in reference- based detectors. We show that our uncertainty- -based detectors mostly avoid that downfall and provide good OOD detection and improved translation performances.
20.05
ope fis
64
(a) MSP
(b) Energy score
Figure 4: Impact of the temperature used to compute the energy (ag) and MSP (aMSp) OOD scores in terms of AUROC.
G.1 Absolute performances
F4 ROC AUC curves
It is clear (somewhat expected) that NMT models do not perform as well on OOD data as we can see in Tab. 19b. However, we find that our OOD detectors are able to remove most of the worst-case samples and keep enough well-translated samples so that with correct filtering our method actually allows the model achieve somewhat acceptable BLEU scores.
F.4.1 Language shifts
In Fig. and Fig. we present the ROC-AUC curves of our different detectors for language shifts in translation.
F.4.2 Domain shifts
In Fig. 9 and Fig. 10 we present the ROC-AUC curves of our different detectors for topic shifts in translation.
G.2 Gains
In Tab. 20 we give the detailed gain in translation performance based on the BLEU score.
F.4.3 Dialog shifts
G.3 Choice of threshold
In Fig. 11 and Fig. 12 we present the ROC-AUC curves of our different detectors for topic shifts in a dialog setting.
We believe that the choice of the threshold for OOD detection should not require OOD samples because we do not want to assume we have access to all kind

---

Task Language shills
Task shifts
Task Dialog shills
Ours
Baselines
Score
N
AIROC
029
X
D1
4.24
do;
Dmt
2.8
FPR
Figure 6: Trade-offs between AUROCand FPR [for each tasks and metrics
Table 18: Detailed performance results of our OOD detectors on dialog shift against the Multi WOZ dataset
reference set.
precidion
poxtsion
Carc
0.877
0.775
0.811
0.87
11.99
1187
0034
0.57
117%%
11:35
11.19
1.:77
11.00
Clert
0:51
0:41
0.55
0.37
18.72
01:9
0.55
08.52
11:52
0.32
0.75
11,9
11:41
03:59
0.55
11:5
0.25
0.25
0162
0.40
0.5
0.75
0.55
1.00
0.63
0.37
0.63
0.56
0.43
0.34
0.28
1.00
Buaclize
0.81
0.79
0.62
0.39
0.76
0.72
0.75
0.77
Bucclacs
0.40
0.65
0.77
0.73
0.43
1.00
0.27
1.00
0.53
0.64
0.57
0.42
0.22
0.75
0.53
0.22
0.36
0.85
0.67
0.52
0.31
0,69
0.34
0.25
0.00
0.89
0.72
0.43
0:00
0.51
0.72
0.52
0.09
0.93
0.5
0.40
0.63
0.53
0.56
0.75
0.06
0.68
0.70
0.64
0.56
055
0.70
0.47
0:43
0.81
0.67
0.63
0.33
0.55
0.44
0.45
Caar
0.66
0.00
0.61
0.65
1167
0166
11:6
1146
8:47
0:51
083
0.06
1167
0.37
1141
Oee
0.31
0.99
0.47
0.11
066
016
0.6
0.67
01:0
1130
088
11501
11:1
0.40
1136
0.36
0.00
0.89
0.72
0.43
0.00
0.5
0.71
0.51
0:53
0.81
0.75
0.58
0.54
0.77
0.51
0.59
0.55
0.57
0.57
0.88
8:37
891
0.58
0.37
0.31
0.80
0.58
0.66
0.29
0358
0.33
0.66
RC
0188
038
11501
0125
0.66
0186
0.07
0.54
0.89
0170
035
1199
0191
0155
0.66
0387
0384
1187
1135
030
1130
013
0:51
0.07
0.24
0.75
1350
11W
0.55
0.9
(a) dailydialog- default
(b) silicone- melds
reaal
Somerti
Soora
Somorin
Scom
Oars
0.52
0.41
0.95
0.87
0.73
01.09
0.53
0.55
0.37
0.52
0.09
0:73
0.50
01.39
0.54
0.35
Outs
0.79
0.68
0.73
0:30
0.81
0.70
0.36
0.39
0.73
0.49
0.74
0.68
0.75
0.66
0.59
0.39
0.63
8.87
11.63
11.56
0:83
0.74
01:38
1.00
0:75
0.75
0.66
0.:5
0.67
035
(3.1
1(0)
11:40
0.66
11:47
117%%
8:45
1.00
11:77
1.001
8.11
0.71
0.88
(1:0
11:88
0190
11:76
1131
musi
13.50
0389
1167
1U52
8131
1469
1136
11:36
Rus
0:20
0A1
1160
0.85
13.5
01001
13:99
0.45
0.80
093
0185
11:0
0.53
0.55
11:66
ILT5
0.92
091
0.91
0.25
0.3
0.42
1.00
0.67
Opts
0.42
0.84
0.67
0163
0:43
0.5
0.14
1.63
0.00
0.00
0.61
0.27
0.64
0350
0.65
F2
Care
0.39
0.40
0.82
0.83
0.65
0.65
0.65
0.63
0.40
0.29
0.84
0.66
0.41
0.41
0.38
0.38
11
Oer
ap
0.59
0.59
0.59
0.53
0.59
0.80
0.67
0.48
0.41
0.41
0.90
0.90
0.61
0.61
0.31
0.31
0gg
0.53
0.88
0.75
0.55
0.54
0.77
0.51
0.50
opp
0.75
0.74
0.25
0.31
0.65
0.77
0.74
0.5
Oggee
0.52
0.78
0.59
0.08
0:33
0:35
0.39
oppee
0.56
0.59
0.59
0.57
0350
0.59
Hoadires
0.53
0.56
0.88
0.87
01:00
11.74
0199
0176
1.99
0.5
0135
0.64
11:0
RoneBoox
Ag
dal
0.81
0.87
0.01
0.57
0188
0.15
0.00
0180
0031
0131
01:01
0.38
0.58
(c) silicone-melde
(d) calicone-dydac
Aurx-1 IN AUre-UU
A.30C
130
(1
11%:
pestiion
reslll
AUrR-1N
AJMI
1191
prodiiiss
nocll
Semuaio
Scorz
Socmatio
Score
Oars
093
0.71
0.68
0.73
0.74
0..22
0..2
0.36
0.36
0.57
0.63
0.67
0.67
0.:00
0.73
0.55
0.48
Ours
agp
0.70
0.68
0.73
0.30
0.81
0.70
0.36
0.30
0.73
0.29
0.54
0.68
0.78
0.56
0.60
0.59
og
0.75
0.25
01.6
0.39
8.67
0.74
0155
1.00
59
9g
0:75
0.75
0.68
0.35
0.67
0.8
Roselines
0.09
0.08
01:50
01:0
8:67
1.10)
0130
Rous
8.91
0.71
0.38
0.39
0.788
0350
0176
0.51
0.1
0.24
01:1
1100
0100
11:00
0LSP
0.50
061
0166
0.95
030
0.45
0.89
0388
0.50
0.79
IL32
0.22
091
0.23
0.42
1.01
0.57
40%
0.30
0.56
0.30
0.60
0.6
0.47
0.89
Oaza
ougg
nagen
0.66
0.67
0167
0.67
1:49
0.51
0166
0.67
0.39
51
Our
ouge
0.94
0.57
1101
0147
0190
0.92
0M1
dapr
0.50
0.31
0176
0.56
0.25
0.34
0.41
0.77
0.58
Oxgen
0.59
0.38
0.58
Oxpenn
0.33
0.59
0.89
Buadites
60
0.93
0.58
0.50
0.52
0.50
0.54
Bacines
daf
d0
0.55
0.84
0.93
0.87
0.10
0.22
0.21
0.35
0.5g
(c) silicone-swda
(f)
sillicoe-dyddan dydada
Sccsaioo
Score
AUMe-UN
AUM1-0011
AU1OC
(1
FMM
preceice.
recall
Somenn
Scor
NIF6-TH
X199-017 </R0C
1PR
11
TPR
proiicin mvall
Baiclass
Riu.
192.
kg
0.58
0.40
0.71
1.00
0.34
0.32
Our
0jR
06
ni..
0:21
0.72
0.69
6.35
048
0.34
8:77
0.76
0.50
0.:3
0.37
0.36
0.68
0.87
11:0
1.00
0.52
0.49
0.00
Kng
eng:
0.55
0.5
0.29
nuge
9p22
0187
Bailiss
SC"
EVV
0.45
0.42
0.92
0.75
0.50
0.42
Bucines
0.88
(g)
silicone-iemocap
(h) silicone-mrda

---

deu-fr
0.6
0.4
0.0
500-cat
spa-fr
0.8
0.2
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
100
3m
Dgp?
Figure 7: ROCAUC curves for our uncertainty baased metrics compared to common baselines for language shift detection. Baselines are represented dashed lines.
deu-fr
deu-nl
nid-afr
0.8
0.2
spa-cat
spa-fr
spa-por
0.6
0.0
1.0
0.2
8g:
Figure 8: ROC-AUC curves our reference-basee based metrics compared common baselines for language shift detection. Baselines are represented in dashed lines.

---

de:news lanel
1.0
0.8
0.6
0.4
0.2
0.0
spa:tat-EMEA
spa:tat-parl
1.0
0.8
0.2
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0g
aMSP
aL
ap,
Figure 9: ROC-AUC curves for uncertainty-haat baaed metries compared common baselines for domain shift detection. baselines are represented in dashed lines.
de:news-EMEA
de:news- Parr
1.0
0.8
0.6
0.4
0.2
0.0
spa:tat-EMEB
spa:tat-par
1.0
0.8
0.6
0.4
0.2
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
asa*
ap;
ac.
ax
apž
ao;
ap.
Figure 10: ROC-AUC curves for our reference-based metrics compared to common baselines for domain shift detection, baselines represented in dashed lines.

---

daily, defeo
dyda_da
0.6
0.4
0.0
meld
mrda
Swda
0.8
0.2
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
10
Figure 11: ROC-AUC curves for our uncertainty based metrics compared to common baselines for dialog shift detection. baselines are represented in dashed lines.
daily dialog-default
dyda da
lemocap
0.8
0.2
meld_s 5
mnda
Swda
0.6
0.0
1.0
0.2
0.8
agl
Sys
Figure 12: ROC-AUC curves for our reference-basee based metrics compared to common baselines for dialog shift detection. baselines represented in dashed lines.

---

spe-cul
spa-por
nid-afr
spartat-parl
deunewsspprl
spectalEEMEA
deu:news-ENe EMEA
Scenario
Score
X
59.63
59.63
62.38
59.63
34.07
59.63
34.07
63.82
62.41
67.77
64.78
36.68
63.82
36.69
Ours
GFR
62.41
62.41
65.89
63.54
36.10
63.54
36.19
ng
62,99
65.80
62.99
35.87
62.99
35.87
64.52
64.52
67.78
64.52
36.72
64.52
36.72
Basclines
58. 15
58.15
60.31
58.15
33.00
58.15
33.00
aCOB
60.36
60.36
63.19
60.36
35.68
60.36
35.68
59.99
59.99
62.82
59.99
59.99
33.82
Ours
dFR"
59.98
59.98
62.83
59.98
33.78
59.98
33.78
91
60.80
60.80
62.67
60.80
34.48
60.80
34.48
Basclines
GM
59.63
59.63
62.61
59.63
33.85
59.63
33.85
(a) IN
spa-cut
spa-por
nid-afr
spactat-parl
deu:news-parl
spactut-EMEA
deu:news-EMEA
Scenario
Score
X
15.73
15.40
23.79
33.17
28.36
59.38
52, 16
Ours
an,
38.04
36.51
41.80
54.60
dFR
26.41
37.74
33.86
38.17
32.10
62.29
55.02
14.44
12.97
27.64
35.11
31.97
60.52
54.48
5
Baselines
4g
16.62
14.70
35,90
40.23
34.18
61.32
54.41
4MSP
17.36
18.97
23.78
30.81
26.46
41.51
34.64
acQs
17.61
16.70
29.02
33.18
29.36
$9.66
54.41
Ours
ap:
20.16
23.33
30.74
34.16
27.38
44.45
43.62
Apw
20.15
23.35
30.61
34.16
27.39
44.45
43.61
Baselines
4c
16.84
16.72
23.67
33.32
29.11
59.55
52.19
4M
16.76
19.37
25.64
30.83
27.22
59.73
54.71
(b) OOD
spa-cat
spa-por
nld-afr
spartat-parl
deunnews- parl
spa:tat-EMEA
deu:news Geu:A
Scenario
Score
X
50.25
44,89
55.20
50.81
31.21
59.55
43.12
Ours
aD.
61.50
61.35
65.66
59.61
35.35
63.91
46.32
AFR
61.35
63.40
56.70
34.34
63.10
46.11
45
54.86
48.72
61.99
53.66
34.19
62.06
45.78
58.75
54.92
65.56
58.32
35.64
63.35
46.22
Baselines
4MSP
51.26
48.32
52.96
48.65
29.50
56.86
33.37
aCpE
51.69
45.67
57.53
32.88
60.09
44.38
54.33
53.38
59.22
58.34
30.37
59.52
36.91
Ours
54.33
53.39
59,18
58.34
30.37
59.52
36.89
51
53.27
48.88
55.85
50.62
31.77
60.34
43.84
Basclines
53.99
58.40
$3.80
30.42
59.63
37.86
(c) ALL
Table 19: Absolue translation performances in terms of BLEU on the different subset (IN, OOD, ALL.) of each dataset of our translation OOD performance benchmark.

---

spa-cat
spa-por
nid-afr
spactat-parl
deu:news-parl
spa:tar-EMEA
Meu:news-EMER
Scenario
Score
+4.19
+2.78
+5.38
+5.15
+2. 61
+4,19
+2.62
Ours
dFR
+2.78
+2.78
+3.50
+3.91
+2.03
+3.91
+2.11
45
+3.36
+3.36
+3.42
+3.36
+1.79
+3.36
+1.79
+4.89
+4.89
+5.40
+4.89
+2.65
+4.89
+2.65
Baselines
4MS
-1.48
-1.48
-2.07
-1.48
-1.07
-1.48
-1.07
40Q8
+0.73
+0.73
+0.80
+0.73
+1.61
+0.73
+1.61
Ours
aD:
+0.36
+0.36
+0.44
+0.36
-0.26
+0.36
-0.26
ApR*
+0.35
+0.35
+0.45
+0.35
-0.29
+0.35
-0.29
51
+1.17
+1.17
+0.29
+1.17
+0.40
+1.17
+0.40
Baselines
6M
-0.00
+0.23
-0.22
-0.00
-0.22
(a) IN
spa-cat
spa-por
nld-afr
spa:tat-parl
dew:news-parl
spa:tat-EMEA
deu:news-EMBs
Scenario
Score
Ours
OD,
+6.33
+22.68 64
+12.72
+8 62
+5.28
+4. 74
+2.44
OFR
10.68
+22.34
+0.07
+5.00
+3.75
+2,91
+2.85
og
-1.29
-2.43
+3.85
+1.94
+3.61
+1.14
+2.31
50
+0.89
-0.70
+12.12
+7.06
+5.82
+1.94
+2.24
Baselines
OMPP
+1.63
+3.57
-0.00
-2.36
-1.90
17.87
-17.52
ACgE
+1.88
+1.30
+5.23
+1.00
+0.29
+2.25
Ours
0
+4.43
+7.93
+6.95
+0.98
-0.98
-14.92
-8.54
OPR
44.42
+7.95
+6.82
+0.98
-0.97
-1492
-8.56
51
+1.11
+1.32
-0.12
+0.14
+0.75
+0.17
+0.02
Baselines
0
+1.03
+3.97
+1.86
-2.34
-1.14
+0.36
+2.55
(b) OOD
spu-cal
spa-por
nid-afr
spuctat-porl
deu:news-parl
spa:tat-EMEA
deurnews-EMEA
Scenario
Score
Ours
OD,
+11.24
+16.46
+10.46
+8.80 88.80
+4.14
+4.37
+3.20
OPR
+10.55
+16.46
+8.20
+5.89
+3.13
+3,56
+2.99
+6.80
+2.85
+2.98
+2.52
+2.66
+8.50
+10.04
+10.36
+7.51
+4.42
+3.80
+3.10
Baselines
+1.01
+3.43
-2.24
-2.16
-1.72
-2.68
-9.74
ACgE
+1.44
+0.78
+2.33
-0.90
+1.67
+0.55
+1.26
Ours
0Dt +4.08
+4.03
+7.53
-0.85
-0.02
-6.20
OpR
+8.50
+3.98
+7.52
-0.03
-6.23
51
Baselines
OC
+3.02
+3.99
+0.65
-0.19
+0.56
+0.80
+0.72
+3.30
+9.10
+3.20
+22.99
-0.79
+0.08
-5.26
(c) ALL
Table 20: Detailed impact of the OOD filtering on the different subset for each task.

---

aptecal
spa-pux
decnnoo-ppal
decunos EMEA
Dinect Score
000
IN
001
0OO
0O1
ALL
Scenorto
Oues
4p2
34%6
2076
188%
9:2%
563
4415
20/5
10%
93R
265
9756
17%
245
17%
34%
255
20%
10%
57%6
166
195%
17%5
1215
1.4%
1.5%
19%
105
24%
75me
11%
25%
70%
165
2815
705
61%
376
70%
201
3354
335
40%
1966
205
.1%
14%
20%6
4%
27%
194
595
33%
19%
51%
3016
20%
2756
195
10%
305
19%
407
1458
19%5
6%
155
19%6
65
22%
188%
387
27%
187
45%
176
199
10%
1559
18
1256
14F
20%
8%
415
1856
50.
40%
20%
70%
21%
70P4
155g
20%
30P4
2756
30P5
11%
1154
28%
30%
14%
205:
11
205
11%
Outs
9pa*
1718
11%
8.y
59%
Table 21: Share of the datasets removed when taking Y so that we keep 80% of the IN distribution.
of different OOD samples that might occur. There- fore we choose fix the False Positive Rate of our detector by constraining the amount of known IN distribution samples that are classified as OOD.
H Negative results
H.1 Different aggregation of OOD metrics
Most of our detectors are initially classification OOD detectors that we adapted for text generation by averaging them over the generated sequences and using this aggregated score as score for the whole sequence. We experimented with other ag- gregations such as the standard deviation or the min/max along the sequence. If the standard devia- tion gave relatively good results they were still less interesting that the naive average.
ALL
odo
Dalasel
spa-cat
spa-por
nid-afr
Figure 13: Gain in translation performances when filtering OOD samples with method different datasets and language pairs.
I1. Negentropy of bag of distributions
We introduced in Sec. 3.3 the bag of distributions as a way to aggregate a sequence of probability distribution and compare it to set of reference using information projections Sec. 3.3. A natural idea would be apply the Negentropy methods (Sec. 3.2) these aggregated distributions.
More formally given sequence of probability distribution So = [p% (x, &e)) we would compute its bag of distributions:
Table 22: Correlation between OOD scores and translation metrics BLEU and BERT-S on domain shifts datasets.
M po(a, Ver)
(x(
(7)
And then compute as novelty score:
BERT-S
CONET
ALL
IN
OUT
ALL
IN
OUT
ALL
IN
OUT
Score
Oom
0.37
0.31
.1.29
01.25
-0.27
0..18
-0.25
0.19
4.25
0.22
41.19
0.09
.0.34
01.9
.0.33
0.29
-0.25
0.17
JD(p) D(pl(().
(8)
0.39
0121
0.26
0.33
Further experiments have shown that this pro- unable to discriminate OOD samples improve performance translation. We suspect that the uncertainty at each step is key capture the behavior of the language model and that this uncer- tainty information is lost when averaging probabil- ity distribution along the sequence.
0.19
0.39
Fcoe:
-0.01
0.06
0.13
0.14
0.25
046
.0.34
0.01
0.10
-0.08
-0.00
0.19
-0.15
-000
0.13
Ours
.0.21
0.02
0.08
-0.08
0.01
0.15
-0.14
-0.02
0.12
Box.
.0.20
-0.02
0.00
-0.04
0.00
0.09
-0.13
-0.02
.000

---

Table 23: Detailed impacts on NMT performance results per tasks (Domain- or Language-shifts) of the different OOD detectors with threshold defined keep 99% the data. We present results on the different part of the data: IN data, OOD data and the combination of both, ALL., For each we report the absolute average BLEU score (Abs.), the average gains in BLEU (G.s.) compared to a setting without OOD filtering (fo only) and the share of the subset removed by the detector (R.Sh.).
Domain shifts
Language shifts OOD
IN
OOD
ALL
IN
ALL
Abs.
R.Sh
Als.
G.s
R.Sh
G.s
Abs.
R.Sh
Abs.
R.Sh
Abs.
R.Sh
47.1
10.0
43.4
0.0%
45.3
13.0
0.01%
60.5
0.0%
0.0%
43.9
Ours
47.3
+0.2
1.0%
44.2
+0.8
5.3%
45.8
+0.5
3.2%
60.7
10.2
1.0%
21.8
63.7
34.35
49.2
15.4
14.6%
44.1
7.2%
40.4
60.7
1.0%
22.9
15.8%
47.3
+0.2
1.0%
44.0
+0.5
1.9%
43.6
+0.4
1.4%
60.9
+0.4
1.0%
18.7
+0.0
17.6%
45.0
+2.1
7.3%
Bas.
47.3
+0.2
0.9%
44.0
1.9%
+0.4
60.8
+0.3
0.9%
18.4%
46.2
+2.3
7.6%
0M
47.0
-0.1
1.0%
40.3
33.1
14.7%
43.5
-1.8
7.8%
60.4
.0.1
1.0%
18.5
10.3
4.3%
44.3
10..5
2.5%
0pg
47.0
40.3
-1.4
13.7%
60.5
45.3
47.0
-0.1
0.9%
40.3
-3.1
26.6%
43.9
-1.3
13.8%
60.5
0.9%
19.3
11:1
10.5%
45.3
11.4
4.8%
Bus.
47.0
-0.1
1.0%
41.6
18.1%
44.6
-0.7
9.5%
60.5
10%
12.1%
45.3
+1.4
5.9%


Reviews:
[{"review_id": "RHCt1uZdeU", "reviewer": "Reviewer_DVtH", "summary": "", "questions": "1. in line 289-290 \"Softmax-based detectors such as MSP, Energy or the sequence log-likelihood\" does not have a verb.\n\n", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank DVtH for their review. \n\n- **Our methods do not require the true output $y$. They rely on the output $\\hat y$ generated by the model.** Making the methods perfectly suitable for real-world applications. \n- We give in Appendix B.7 the details on how all the baselines are computed. We take the average OOD score along the generated sequence for MSP and Energy. This is the naive most straightforward method to generalize these baselines. In [1] suggested by reviewer j15V, the authors rely on similar generalizations of the baselines across the elements of the beam.\n\nWe will fix the typos you raised to our attention and include the missing references you pointed out.\n\n[1] Vazhentsev A. et al. Efficient Out-of-Domain Detection for Sequence to Sequence Models. in ACL 2023", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The authors propose an interesting task of detection OOD sample based on the input and output pair (x, y) for NLG model.\n\n2. The analysis and discussions in the paper are comprehensive and solid.\n\n3. The proposed OOD detection method is effective.", "reasons_to_reject": "1. It is better to detail how each baseline is implemented? For example, for the baseline MSP, how do you calcuate the detection score. It is better to illustrate this with equations. (since you have mentioned that \"Due to the large vocabulary size, it is unclear how these methods generalize to sequence generation tasks.\")\n\n2. Usually the OOD detection process is performed only on the input x, i.e., if x is an OOD sample, then we should not feed it into the model to produce y. The proposed method need to get access to the input and output pair (x, y). Sometimes, generating the output y for OOD inputs may lead to unpredictable behaviour of downstream systems.", "questions_to_authors": "1. in line 289-290 \"Softmax-based detectors such as MSP, Energy or the sequence log-likelihood\" does not have a verb.\n\n", "paper_topic_and_main_contributions": "This paper try to perform OOD detection on NLG tasks. Specifically, they propose to use an everaged generation probability as an OOD score to determine whether a given (x, y) pair is an OOD sample. Detailed analysis and discussions are given in this paper. The authors also claim that their method can improve the final performance of the NLG model.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "Some reference on the OOD detection tasks are missing:\n\nEstimating Soft Labels for Out-of-Domain Intent Detection\n\nA recent survey can also be found here:\n\nA Survey on Out-of-Distribution Detection in NLP\n"}}, {"review_id": "exDopzUY1r", "reviewer": "Reviewer_fLxa", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We want to thank reviewer fLxa for their review of our work.\n\nWeaknesses:\n- **About the LLM era and relevancy of our work on smaller specialized models**: While we do agree that our results are not satisfying on the most recent LLMs and that our benchmark focuses on traditional NLP tasks and datasets, these larger models cannot be deployed everywhere or by everyone (yet?). Many still rely on smaller specialized architectures. For example, the Marian MT models [1] are such bilingual models optimized for fast translation. In addition, the definition of OOD becomes less clear regarding large language models trained on various tasks and datasets. We would argue that OOD detection especially matters for smaller specialized models intended for specific usage (in our case, translation). Therefore, we believe our work remains relevant if we want to use smaller/more efficient and highly specialized models.\n\n- In addition as shown in [2] OOD samples are still a challenge and an open question for LLM as they significantly impact their performance. Generally speaking, abstention methods are still very much needed and relevant, from smaller specialized models to LLMs. It is also reasonable to believe that the abstention methods for LLMs and smaller models should differ because of the widely different behaviours exhibited by LLMs and smaller models.\n\n- We will correct the typos you pointed out in the camera-ready version of our work.\n\nWe hope our answers have alleviated your concerns, and you might be able to update your score accordingly.\n\n\n[1] Junczys-Dowmunt et al., Marian: Fast Neural Machine Translation in C++, ACL 2018\n\n[2] Wang and al, On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Strength\n- RAINPROOF computes an anomaly score based on the model's token probabilities being different from a reference distribution, indicating the input is OOD. It can work without reference data or use a set of in-distribution examples.\n- Experiments show RAINPROOF outperforms baselines like maximum softmax probability and improves metrics like BLEU by removing OOD samples. It also provides interpretable detections.", "reasons_to_reject": "Weakness\n- Overall, with the development of LLMs, NLP systems are becoming more and more robust against distribution shifts, such as mastering different languages, topics, and scenarios. What are the truly realistic and challenging OOD problems given nowadays powerful models? I view the tasks and datasets in LOFTER as traditional ones, which may not be important in this era.\n- The experiments on LLMs are few and the results are not satisfying. AUROC remains low and FPR is higher than 0.8\n- The performance advantage is marginal on large models according to Table 12, 13, and 14\n- Sentences are incomplete. Line 289: Softmax-based detectors such as MSP, Energy or the sequence log-likelihood. Table 7: Missing ``Ours.''", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper presents a new framework called RAINPROOF for detecting out-of-distribution (OOD) samples for text generation models. The goal is to detect when an input is different from what the model saw during training, which can cause the model to generate poor or unsafe outputs. RAINPROOF introduces a new benchmark called LOFTER to evaluate OOD detection methods in more realistic scenarios like close language pairs (e.g. Spanish-Catalan) or domain shifts (e.g. news to medical text).", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "OG25prxfij", "reviewer": "Reviewer_j15V", "summary": "", "questions": "A. It is not clear why we choose to filter 80% of in-distribution samples. Do we choose it on some held-out data? Again, how do you choose hyper-parameters for the model if you do not use any OOD examples?\n\nB. Why do we choose the uniform distribution as the reference, and what is the intuition behind it? You also mention other choices but do not use them in the experiments, why?\n\nC. Why you didn’t try to use a normalized likelihood score? It seems that the unnormalized score is biased towards the shorter sentences. And how do you aggregate the last hidden layer of the encoder to get the sentence representation for the Mahalanobis score?\n\nD. When you compare computation time in Table 4, do you add the time to generate the sequence? It seems that the Mahalanobis distance doesn’t need generated sequence (y) at all, but the proposed methods need it. Thus, the comparison could be unfair in Table 4.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer j15V for their thorough review of our work. \n\n**Concerning the construction of the benchmark and choice of setting.** We recognize that we missed works [1-3], and we will add them in the revised version of our work. Nevertheless, we can point out several major differences:\n- In [1] Tim Z. Xiao et al., the authors perform experiments on a single pair of close languages (nld-deu to english). In contrast, our work includes many more pairs and OOD type (we consider topic shifts and close language shifts: afr-nld, nld-deu, por-spa, cat-spa). In addition, it is not clear if this work was ever published.\n- We missed reference [2] by Vazhentsev A. et al., however, this work seems to have been released after the submission deadline of EMNLP, which is the main explanation. They propose ensembling methods that rely on different models to assess the uncertainty of a generation and the OOD nature of the sample. In contrast, our methods are added on top of a single existing model. **They report excellent results when languages are widely different (English vs French in terms of OOD, in Table 16 we report similar results for french vs Spanish for example). However, their results on smaller shifts are not as good around 0.75 (Table 9, Reddit dataset), which is consistent with our results on topic shifts (See Table 17), where we achieve similar or higher AUROCs, with methods that do not require ensembling but only the probability distribution output by a single model.**\n\n- In [3] Guerreiro, Nuno M. et al., the authors focus on natural/normal settings, where the inputs are in distribution and not perturbed by focusing on detecting hallucinations. In contrast, our work focuses specifically on handling OOD samples. Although we could test our methods for detecting hallucinations, we believe it would be a very different task.\n\n**Concerning the lack of comparison of our methods in classification tasks.**\n- We focused on generation as OOD detection in text classifiers has already been explored by many high-performing works (See for example, Colombo et al, Neurips 2022 https://arxiv.org/abs/2211.13527). \n- Our work was prompted by the fact that features-based OOD scores rely on being able to compute a score per-class in classification, which is not possible in text generation, and that this assumption is critical but wrong in text generation (see Appendix A). In classification settings, features-based methods that rely on per-class scores achieve near-perfect results, making our methods irrelevant in classification. Thus, the sole focus is on text generation, which has been significantly less explored in previous work.\n\n**Concerning the other works reviewer j15V pointed out:**\n- [4] Podolskiy, A.V., et al, They focus on OOD detection in classification only and not in generation and conduct a study of the performance of common baselines (the same we used MSP, Likelihood and Mahalanobis distance) in transformers for classification.\n- [5] Yiyou Sun, et al. We were not aware of this work from computer vision. The method they propose could very well be leveraged in NLP but it would be a different work and setting as they rely on having white-box access to the model, whereas we only rely on the soft probability output by the model.\n\n**Questions:**\n- Since we do not allow the use of OOD samples in our setting, we decided to calibrate our OOD detection methods using the training data. We assume that a share of the training is of bad quality and should be excluded. We settled on 80% as a reasonable assumption, but we provide results for a threshold of 99% in Table 23.\n- Using the uniform distribution yields the Negentropy of the distribution (Brillouin, 1953), (https://en.wikipedia.org/wiki/Negentropy). We use it as a measure of uncertainty about the prediction. The negentropy consists of the entropy normalized by the size of the support of the distribution. In the case of text generation, while the vocabulary is very large, the tokens with non-zero probability are rare. We are interested in the shape of the distribution over the non-zero probability tokens and not over the whole vocabulary. We did perform experiments using the TF-IDF distributions, but those did not yield positive results, as reported in Appendix H.3.\n- You are right. We made a mistake in reporting the score, we use the normalized likelihood (the likelihood of the sequence as output as a score from the beam search generation). We will revise the paper to correct this inconsistency. \n- We performed experiments taking the average embedding over the input. We also tried using only the embedding of the first token. \n- We only measured the additional time needed to perform the OOD detection when performing the full generation. So, indeed, one could first extract the encoder embeddings, perform OOD detection using the Mahalanobis distance and then proceed with the generation if the input is deemed “IN distribution”. In that case, using the Mahalanobis distance or other “encoder-only based OOD scores” would be more efficient since it eliminates the generation process. However, doing so does not allow us to assess the quality of the generated sentence as our method does.\n\nWe hope our answers have alleviated your concerns and you will consider updating our grade accordingly.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.    The proposed OOD detection approach shows strong performance and doesn’t degrade the performance of the model on the downstream task, moreover, it could improve it by filtering noisy in-distribution instances.\n2.    The method is fully unsupervised, i.e. no OOD examples are needed for it. The method is model and task-independent.\n3.    The method is clearly described and the paper is well-written.", "reasons_to_reject": "1.    Lack of testing on other benchmarks on the OOD detection – NMT or classification. See papers [1]-[3].\n2.    Lack of support for some of the statements and choices (see Questions)", "questions_to_authors": "A. It is not clear why we choose to filter 80% of in-distribution samples. Do we choose it on some held-out data? Again, how do you choose hyper-parameters for the model if you do not use any OOD examples?\n\nB. Why do we choose the uniform distribution as the reference, and what is the intuition behind it? You also mention other choices but do not use them in the experiments, why?\n\nC. Why you didn’t try to use a normalized likelihood score? It seems that the unnormalized score is biased towards the shorter sentences. And how do you aggregate the last hidden layer of the encoder to get the sentence representation for the Mahalanobis score?\n\nD. When you compare computation time in Table 4, do you add the time to generate the sequence? It seems that the Mahalanobis distance doesn’t need generated sequence (y) at all, but the proposed methods need it. Thus, the comparison could be unfair in Table 4.", "paper_topic_and_main_contributions": "The work addresses the problem of out-of-distribution (OOD) input detection for sequence generation tasks. The OOD detection task is crucial for the real-world application of models because the model’s performance could substantially drop on the OOD instances, hence, we shouldn’t provide such examples to the model, i.e. reject them. Authors are concerned with the translation and response generation tasks. They propose to use a new evaluation setting – Language out of distribution performance benchmark (LOFTER) that comprises machine translation (MT) and dialogues datasets. For MT, there are language shifts (e.g. Dutch model exposed to the Afrikaans) and domain shifts (e.g. EuroParl model exposed to the Tatoeba). For OOD detection, a new method is proposed (RAINPROOF) that can be used in two scenarios depending on whether in-distribution examples exist or not. Their method relies on the measurement of how the distribution of the generated instance differs from the proposal distribution (in an unsupervised regime) and from the closest distribution from in-distribution examples. The Renyi divergence and Fisher-Rao distance are used to measure the score. \n\nThe work is well-written, and the method is clearly described. There are a lot of different experiments presented in the paper and the Appendix. The suggested method outperforms the presented baselines on the proposed benchmark. \n\nHowever, some questions arise and drawbacks exist.\nFirst, the authors do not clearly describe why they needed to make a new benchmark and why they didn’t experiment with their approach on other benchmarks. For example, papers [1], [2], [3] propose different setups for NMT OOD detection. So, it seems that the paper lacks a comparison with the baselines and stronger models on other setups. Also, the authors do not consider the classification task with OOD examples like CLINC-150 which are commonly used, focusing mainly on generation tasks. However, it would be good to examine the approach to this setup.\nSecond, it is not clear why we choose to filter 80% of in-distribution samples. Do we choose it on some held-out data? Again, how do you choose hyper-parameters for the model if you do not use any OOD examples?\nThird, why do we choose the uniform distribution as the reference, and what is the intuition behind it? You also mention other choices but do not use them in the experiments, why?\nFourth, why you didn’t try to use a normalized likelihood score? It seems that the unnormalized score is biased towards the shorter sentences. And how do you aggregate the last hidden layer of the encoder to get the sentence representation for the Mahalanobis score? \nFifth, when you compare computation time in Table 4, do you add the time to generate the sequence? It seems that the Mahalanobis distance doesn’t need generated sequence (y) at all, but the proposed methods need it. Thus, the comparison could be unfair in Table 4. \n\nStrengths:\n1.    The proposed OOD detection approach shows strong performance and doesn’t degrade the performance of the model on the downstream task, moreover, it could improve it by filtering noisy in-distribution instances.\n2.    The method is fully unsupervised, i.e. no OOD examples are needed for it. The method is model and task-independent.\n3.    The method is clearly described and the paper is well-written.\nWeaknesses:\n1.    Lack of testing on other benchmarks on the OOD detection – NMT or classification. See papers [1]-[3].\n2.    Lack of support for some of the statements and choices (see Questions)\n\nAfter rebuttal I increase the soundness scor", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "[1] Tim Z. Xiao et al. Wat zei je? Detecting Out-Of-Distribution Translations with Variational Transfromers. https://arxiv.org/abs/2006.08344\n\n[2] Vazhentsev A. et al. Efficient Out-of-Domain Detection for Sequence to Sequence Models. in ACL 2023\n\n[3] Guerreiro, Nuno M. et al. “Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translation.” in EACL 2022.\n\n[4] Podolskiy, A.V., et al. Revisiting Mahalanobis Distance for Transformer-Based Out-of-Domain Detection. in AAAI 2021\n\n[5] Yiyou Sun, et al. Out-of-distribution detection with deep nearest neighbors. In ICML 2022.\n\n[6] Mohammadreza Salehi et al. A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: solutions and future challenges. In TMLR 2022.", "Typos_Grammar_Style_and_Presentation_Improvements": "It would be better to move the baselines description into the main part of the paper from the appendix, so, the paper will be self-contained.\n\nPage 1, line 16 – ODD -> OOD\n\nIn related work line 206 – in classification tasks, the single-class (ID data) Mahalanobis distance can also show decent performance compare to per-class Mahalanobis distance (see [4], [5], [6])."}}]

Author rebuttal:

Meta-review: This paper addresses the OOD problem in generation (machine translation and response generation in dialogue). The paper propose an evaluation setting, called LOFTER, and show that their proposed method outperforms competitive baselines. In the reviews, There have been concerns about (i) comparison with stronger baselines on established OOD benchmarks on the classification task, (ii) the validity and relevance of the OOD problem in the LLM era, (iii) the marginal improvement of the proposed method in some settings. The authors have addressed these concerns, and by revising the paper according to the discussions, the paper seems to be ready for publication.
Decision: Accept-Main