Uncertainty Estimation in Autoregressive Structured Prediction
Authors: Andrey Malinin
Mark Gales
Venue: ICLR 2021 Poster (2021)
Keywords: ensembles
structures prediction
uncertainty estimation
knowledge uncertainty
autoregressive models
information theory
machine translation
speech recognition.

Abstract: Uncertainty estimation is important for ensuring safety and robustness of AI systems.  While most research in the area has focused on un-structured prediction tasks, limited work has investigated general uncertainty estimation approaches for structured prediction. Thus, this work aims to investigate uncertainty estimation for structured prediction tasks within a single unified and interpretable probabilistic ensemble-based framework.  We consider: uncertainty estimation for sequence data at the token-level and complete sequence-level; interpretations for, and applications of, various measures of uncertainty; and discuss both the theoretical and practical challenges associated with obtaining them. This work also provides baselines for token-level and sequence-level error detection, and sequence-level out-of-domain input detection on the WMT’14 English-French and WMT’17 English-German translation and LibriSpeech speech recognition datasets.

Published as a conference paper at ICLR 2021
UNCERTAINTY ESTIMATION IN AUTOREGRESSIVE STRUCTURED PREDICTION
Andrey Malinin
Mark Gales
Yandex. Higher School of Economics am96 (el9699@anddextee
ALTA Institute, University of Cambridge m fggeng. cam. ac. uk
ABSTRACT
Uncertainty estimation is important for ensuring safery and robustness of Al sys- tems. While most research in the area has focused on unstructured prediction tasks, limited work has investigated general uncertainty estimation approaches for structured prediction. Thus, this work aims to investigate uncertainty estimation for autoregressive structured prediction tasks within single unified and interpretable probabilistic ensemble-basee framework. We consider uncertainty estimation for sequence data at the token-level and complete sequence-level; interpretations for, and applications of, various measures of uncertainty; and discuss both the theo- retical and practical challenges associated with obtaining them. This work also provides baselines for token-level and sequence-leve error detection, and sequence- level out-of-domall input detection on the WMT' 14 English-Frence and WMT' 17 Eingish-Geeman translation and LibriSpeech speech recognition datasets.
INTRODUCTION
Neural Networks (NNs) have become the dominant approach in numerous applications (Simonyan & Zisserman 2015 Mikolov al.) 2013 2010: Bahdanau et al., 2015 Vaswani et al., 2017 Hinton et al. 2012 and are being widely deployed in production. As a consequence, predictive uncertainty estimation is becoming an increasingly important research area, it enables improved safety in automated decision making (Amodei et al. 2016). Important advancements have been the definition of
baseline tasks and metrics (Hendrycks & Gimpel 2016) and the development of ensemble approaches.
such as Monte-Carlo Dropout (Gal & Ghahramani 2016) and Deep Ensembles (Lakshminaray
al. [017 Ensemble-basee uncertainty estimates have been successfully applied to detecting misclassifications, nut-of-disttibution inputs and adversarial attacks (Carlini & Wagner 2017, Smith Gall [2018) Malinin & Gales, 2019) and to active learning (Kirsch et al. a [209)]. Crucially, they allow total uncertainty to be decomposed into data uncertainty, the intrinsic uncertainty associated
with the task, and knowledge uncertainty, which is the model's uncertainty in the prediction due Estimates of knowledge uncertainty
to lack of understanding of the data (Malinin| 2019 2
particularly useful for detecting anomalous and unfamiliar inputs (Kirsch et al., 2019 Smith & Gal
2018 Malinin & Gales 2019 Malinin 2019).
Despite recent advances, most work on uncertainty estimation has focused on unstructured tasks, such as image classification. Meanwhile, uncertainty estimation within general, unsupervised, probabilistically interpretable ensemble-based framework for structured prediction tasks, such as language modelling, machine translation (MT) and speech recognition (ASR), has received little attention. Previous work has examined bespoke supervised confidence estimation techniques for each task separately (Evermann & Woodland 2000; Liao & Gales 2007 Ragni et al. 2018 Chen et al. 2017; Koehn, 2009; Kumar& & Sarawagi 2019) which construct an "ettor-detcttoon model on top of the original ASR/NMT system. While useful, these approaches suffer from a range of limitations. Firstly, they require token-level supervision, typically obtained via minimum edit-distance alignment to a ground-trui transcription (ASR) or translation (NMT), which can itself by noisy. Secondly, such token-level supervision is generally inappropriate for translation, it doesn't 't account for the validity of re- re-arrangements. Thirdly, we are unable to determine whether the error is due to knowledge or
1 An in-depth comparison of ensemble methods was conducted in (Ashkkha et al. 2020 Ovadia ct al. 2019
Data and Knowledge Uncertainty are sometimes also called Alcatoric and Epistemic uncertainty.

---

Published as a conference paper at ICLR 2021
data uncertainty. Finally, this model is itself subject to the pitfalls of the original system domain shift, noise, etc. Thus, unsupervised encertainty-estimaate methods are more desirable.
Recently, however, initial investigations into unsupervised uncertainty estimation for structured prediction have appeared. The nature of data uncertainty for translation tasks was examined in (Ott
al. 2018a). Estimation of sequence and word-level uncertainty estimates via Monte-Cart Dropout ensembles has been investigated for machine translation (Xiao et al.] 2019 Wang et al. 2019] Fomicheva al. 2020). However, these works focus on machine translation, consider only a small range of uncertainty adhoc measures, provide limited theoretical analysis of their properties and do not make explicit their limitations. Furthermore, they don't 't identify or tackle challenges in estimating uncertainty arising from exponentially large output space. Finally, to our knowledge, no work has examined uncertainty estimation for autoregressive ASR models.
This work examines uncertainty estimation for structured prediction tasks within a general, probabilis- tically interpretable ensemble-based framework. The five core contributions are as follows. First, we derive pntommttion-hhoreele measures of both total uncertainty and knowledge uncertainty at both the token level and the sequence level, make explicit the challenges involved and state any assumptions made. Secondly, we introduce novel uncertainty measure, reverse muiual information, which has a set of desirable attributes for structured uncertainty. Third, we examine a range of Monte-Carlo approximations for sequence-level uncertainty. Fourth, for structured tasks there is choice of how ensembles of models can be combined; we examine how this choice impacts predictive performance and derived uncertainty measures. Fifth, explore the practical challenges associated with obtaining uncertainty estimates for structured predictions tasks and provide performance baselines for token- level and sequence-level error detection, and out-of-domain (OOD) input detection on the WMT' 14 Mnglish-Frencee and WMT' 17 English-German translation datasets and the LibriSpeech ASR dataset.
2
UNCERTAINTY FOR STRUCTURED PREDICTION
In this section we develop an ensemble- based uncertainty estimation framework for structured predic- tion and introduce novel uncertainty measure. We take Bayesian viewpoint ensembles, it yields an elegant probabilistic framework within which interpretable uncertainty estimates can be ob- tained. The core of the Bayesian approach to treat the model parameters 0 as random variables and place a prior p(0) over them to compute a posterior P(O via Bayes' rule, where D is the training data. Unfortunately, exact Bayesian inference is intractable for neural networks and it is necessary to consider an explicit or implicit approximation q(0) to the true posterior (D) to generate an ensemble. A number of different approaches to generating ensembles have been developed, such as Monte-Carlo Dropout (Gal & Ghahramani] (016) and DeepEnsembles (Lakshminarayanan et al., 2017]. An overview is available (Ashukha et al. 2020 Ovadia et al. 2019).
Consider an ensemble of models (P(y)æ:; |z; ((m))) m= sampled from approximate posterior q()). where each model captures the mapping between variable-length sequences of inputs {z1, .T]
E X and targets {y1, , yz } y V, where 2, E {wi,-.. ). Y1 {wi," ,,k The predictive posterior is obtained by taking the expectation over the ensemble:
,wk
M
P(ylæ 2e, D) = Eqqo) [P 0)] ~
Σ
P(ylar olm)), @(m) ~ ~ x(0)))
(1)
M
mm
The total uncertainty in the prediction of y is given by the entropy of the predictive posterior.
H(y(yy D)] = Ep(yle, P(y /z. D)]
Σ
P(ylæ, D) In Pnylo, D)
(2)
Total Uncertainty
wEy
The sources of uncertainty can be decomposed via the mutual information I between & and
[n P æ P <z
I [y, DJ =Eg(0) Ep(y)om))
]]
?((P(y)z,
D)] Eg(o) [ 0)]]
Know: Uncertainly
Total Unsertainty
Expected Data Unvertainty
Mutual information (MI) is measure of 'disagreement" between models in the ensemble, and therefore a measure of knowledge uncertainty (Malinin 2019). It can be expressed as the difference between the entropy of the predictive posterior and the entropy
entropy of each model in the ensemble.
2

---

Published as a conference paper at ICLR 2021
The former is a measure of total uncertainty and the latter is measure of data uncertainry (Depeweg et al. 2017). Another measure of ensemble diversity is the expected pairwise KL-divergence (EPKL):
P æ. P( x. 0)
01, D] = - 0())) (
In
q(0) = p(0)D)
(4)
where q(0) = q()) and θ is dummy variable. This measure is upper bound the mutual information, obtainable via Jensen's 's inequality. A novel measure of diversity which we introduce in this work is the reverse mutual information (RMT) between each model and the predictive posterior:
P(y a. D) P 0)
Myy 0 DJ = Eq(0)) (Pllyln
In
q(0) = p(0ID)
(5)
This the reverse-Ki divergence counterpart to the mutual information 33. and has not been previously explored. As will be shown in the next section, RMI is particularly attractive for estimating
uncertainty in structured prediction. Interestingly, RMI is the difference between EPKL and MI:
Myy 0|æ, DJ K. K[y, 0|2, D) I [[y 0|x, 20
While mutual information, EPKL and RMI yield estimates of knowledge uncertainty, only mutual information 'cleanly' decomposes into total and data uncertainty. EPKL and RMI do not yield clean measures total and data uncertainty, respectively. For details see appendix A
Unfortunately, we cannot in practice construct model which directly yields distribution over an infinite set of variable-length sequences y E y. Neither can we take expectations over the this set. Instead, autoregressive models are used to factorize the joint distribution over y into product of conditionals over a finite set of classes, such as words or BPE tokens (Sennrich et al.) 2015).
L
P(y|2x 0) = II P( yet. x; 0), De E [Wi,".. wy }. y1 E fw1,"" "o,,""
(7)
Here the distribution over each y conditioned on all the previous y-t = (11, ... Ut-13 1- which we shall refer to as the context. This set of conditional independence assumptions allows us define a model on the finite space Vd. This formulation describes all machine translation (Bahdanau et al. 2015 Vaswani et al. 2017). end-to-end speech recognition (Chan et al., 20155) and other related tasks. Alternative factorization orders, representing different conditional independence assumptions, can be considered. However, without loss of generality, will use the standard factorization.
For these models uncertainty estimation can be examined at two levels the token level, which considers uncertainty in the prediction of single y), and the sequence level, which considers the uncertainty of predicting the entire sequence y. Token-level uncertainty estimation for autoregressive models is isomorphic to un-structured uncertainty estimation with additional conditioning on the context Y<ı. and so is presented in appendix [A.]] Instead, we focus on discussing sequence-level uncertainty estimation in autoregressive models. We have chosen to focus on autoregressive models as they present interesting challenges, are more general than models with stronger conditional independence assumptions, and are widely applied to tasks of practical value. We emphasize that the proposed ensemble-based approach is general and can be applied to structured tasks with different conditional independence assumptions (Graves et al. 2006 Gales & Young 2008
Gu et al. 2017)
MONTE-CARLO APPROXIMATIONS
The key challenge of autoregressive models is that expressions (2-(5) are intractable to evaluate. Specifically, all expectations over are intractable to evaluate due to the combinatorial explosion of the hypothesis space there are a total of |K| possible L--length sequences in )/ where K is the vocabulary size, and it is necessary to do forward-pass through the model for each hypothesis. This is an issue which was ignored in prior work (Wang et al. 2019: Fomicheva et al., 2020 Xiao et al. 2019). Clearly, it is necessary consider Monte-Carlo approximations to make this tractable. key desiderata of these approximations is that they should be obtainable at no extra cost on top of standard
inference from the ensemble. We examine two types of Monte- Carlo (MC) approximations for expressions -55 which are identical in the limit, but have different attributes given a finite sample size. Properties of these approximation are detailed in appendix A
3

---

Published as a conference paper at ICLR 2021
A result of the auto-regressive conditional independence assumption is that distributions over long sequences can have higher entropy than over short ones. To compare uncertainties sequences of different lengths, in accordance with the desiderata in the introduction, consider length normaizedd "rate' (Cover & Thomas 2006) equivalents of all uncertainty measures, denoted by 'A'.
The simplest Monte-Carlo estimation for entropy is to approximate using S samples:
MC P(y] D)]
InP (r, y P(yla,, D) S L(
Σ
(8)
where y is a realization of the random variable Alteratively, we can approximate as sum of conditional entropies via the entropy chain-rule (Cover & Thomas 2006):
(
C.C P(yla, D)J
(s) a D)J, HP
VyLa (5)
C Cyll P(yla, D)
(9)
Given set of consistent contexts Yet y L, this approximation reduces to averages of token-level uncertainty estimates as a consequence of the entropy chain-rule.
Approximations ( and (9) both yield exact estimates of total uncertainty (2) in the limit S co.
However, 8 only considers the probabilities of individual tokens yi along a hypothesis y()) while (9) considers the entire conditional distribution over each yt. Consequently, (9) may yield more stable approximation using a smaller number of samples. At the same time, while (8) yields a noisier estimate for finite S, is sensitive to the particular set of hypotheses considered.
Monte-Carlo approximations can also be considered for mutual information
S
(2 0l-))
(S) MC y, 0 0r
D
L(s) In
(5)) P(y) O(*),, 0 ~ ( (10)
Unfortunately, this requires sampling from each model individually obtaining this estimate does not come 'for free' with standard ensemble inference. An efficient approximation be obtained via the relative-entropy chain-rule (Cover & Thomas 2006):
S
20
T-Me y. 0 D ~
Σ s=1 ((s)
Σ
Tl. 2 D, Vyla C Cy P(ylz, D)
(11)
S
Unlike (1O). uses samples from the predictive posterior and reduces averages of token-level mutual information Z[yn, 0 0ly x. D). Thus, it is obtained at no extra cost with standard ensemble inference. However, it will not yield (3) as S Do. Nevertheless, this estimate may still be useful in practice. An approximation MCMC for EPKI. with identical properties described in appendix A
Asymptotically exact joint-sequence and chain-rule MC estimates of knowledge uncertainty can be obtained 'for free' during inference from P (, D) by considering the new measure RMI 5
( L D) P P xz 0).
M (9) M. [y,
0 D
In
y(()) P(yl(a,
(12)
5
[(a)
M y. 02 D
Mlyn 0 D.
Vy (3)
C ( ~Pyy/,
(13)
Similar to and 11b. 13) is also average of token-level RMI M[gr, 0
03y(2) 2 x, DJ. and like
8
[2) is sensitive to particular set of hypotheses, while (13) is more stable
Practical Considerations Before applying the proposed Monte-Carlo approximations, two practi- calities need to be considered. Firstly, due to the vastness of the hypothesis space y² Monte- Carlo Monee
sampling requires prohibitively many samples and the introduction of decision rule, whose approx- imation may require a costly computation and whose effectiveness is not yet fully understood, to
find a good set of hypotheses (Eikema & Aziz 2020 Holtzman et al. 2019). Instead, beam-search
4

---

Published as a conference paper at ICLR 2021
is typically used for inference, as it efficiently finds high-quality hypotheses. With regards to the Monte-Care estimators above, beam-search can be interpreted as a form of pooortacce-sapllngg which yields hypotheses from high-probabillin regions of the hypothesis space. As each hypothesis is seen only once during beam-search, the uncertainty associated with each hypothesis ()6)) within beam B in the MC estimators above must be ceeprraancewweggheee in proportion to P(y (8) c D).
B
exp + In P (b)
x D)
A H ((y)),
Tt L
In (2 (), yy ()) E B, Tb
(14)
Σx exp In (ny (z,
B
L
HCW P(y
Σ
Th H|P
(8) y v21 D)1
(15)
L
Note that here we introduce temperature calibration T, which allows us to 'soft-adjust" the contribu- tion of the lower-preg swerrprobabbility hypotheses to the resulting measures of uncertainty. Higher temperature make the importance weights more uniform across the hypotheses. The effects of this are detailed in
appendix G Equivalent expressions for I K.C.v M and M (B) are provided in appendix A
Second, we must consider how to obtain the predictive posterior for an ensemble of autoregressive models. The models can be combined either as expectation-P) of-oroducts or productt-ffeexpecaaiioss
L Pro(yla æz D) = (gy(() II P(ullee 2 0) Ppe(y]ae, D) = II Ey( P(Wllee,, 11 2,, 0)]
L
(16)
The former represents sequence-level Bayesian model averaging, while the latter token-level Bayesian model averaging. Both are methods to do model combination but only the former is fully consistent with the sequence-leve uncertainty measure defined in section as they assume that all tokens the sequence y are generated from the same ((m), However, it is not clear a-priori which combination yields superior predictive performance given a of samples of parameters @(m) from particular q()) and an inference method. If hypotheses are obtained via beam-search, which is typically sequence of token-level decisions, considering Ppe(y//ax D) may be advantageous. The choice of combination also affects how the token-level predictive posterior P(yr Ve x, D) is obtained:
PEp PErluIEE x, = Eg(o) W1 y 22 x. 0)] Bq(0) [P <1 0)]
(b)
Pre(yn 221 D) = Eq(o) (P vel x,0)) (17)
This choice affects measures derived from the sequence and token-level predictive posteriors. How- ever, all measures can still be calculated for both forms at the same time, regardless of which was used for inference. Thus, the choice of combination depends on which yields superior performance.
4 EXPERIMENTAL EVALUATION
The current section provides performance baselines on three applications of structured uncertainty estimates: sequence-level and token-level error detection, and out-of-distribution input (anomaly) detection. Additional analysis is provided in appendices C We also compare performance to prior heuristic ensemble-base approaches. This work only considers ensembles of autoregressive neural machine translation (NMT) and speech recognition (ASR) models generated by training identical models from different random initializations [Lakshminarayanan et al. 2017). This approach was shown to consistently outperform other ensemble generation techniques using exponentially smaller ensembles (Ashukha et al. 2020 Ovadia et al., 2019 Fort et al. 2011). Ensembles of 10 transformer- big Vaswani al. 2017) models were trained on the WMT' 17 English-to-Gerran (EN-DE) and (EN-FR) translation tasks and evaluated on the newstest14 14 (nwt14)
WM English-to-Frenn
dataset. All models were trained using the configuration described in (Ott et al. 2018b). Ensembles of 6 GGG-Transformer (Mohamed et al., 2019) models were trained on the LibriSpeech (Panayotov) al. 2013) (LSP) ASR dataset. Standard Fairseq (Ott et al. 2019 implementations of all models are
used. Details of model configurations are available in appendix B Note that no comparison is made
IIn the current Fairseq OOtt 2019 implementation ensembles are combined as product-ofeeppecaatinn
5

---

Published as a conference paper at ICLR 2021
to supervised uncertainty estimation techniques for NMT/ASR, such as those described in (Liao & Gales 2007 Koehn 2009). for two reasons. Firstly, the focus of this work is general, unsupervised uncertainty estimation approaches based on ensemble methods. Secondly, our knowledge, they have not been applied to autoregressive models and doing so is beyond the scope of this work.
Table 1: Predictive performance in terms of BLEU, % WERR and NLL on newstest14 and LibriSpeech.
NMT BLEU
ASR % WER
NMT NLL
ASR NLL
Madel
EN-DE
EN.FR
LTO
EN-DE
EN.FR
LTC
Single
28.8 +02
45.4 +
5.6 40.
14.7 +0.5
1.46 0.00
1.10 0.01
0.34 +0.00
0.86
#:10
ENS PrFF
30.1
46.5
4.2
11.3
1,3
1.04
0.20
0.48
ENS-ExPr
29.9
46.3
4,5
12.6
1.36
1.05
0.23
0.58
Choice of Ensemble Combination As discussed in section 3 ensembles can be combined as an expectation--o of of--ucucccsss (ExPr) or as a prrouuctof-eeppetaaiiie
(PrEx) (T6). Therefore, it is necessary to evaluate which yields superior predictive performance. We evaluate EN-DE and EN-FR NMT models on newstest14 44 and the ASR models on LibriSpeech test-clean (LTC) and test-other (LTO).
Results in table 1 show that product-of higher translation BLEU and lower ASR word-error--ate (WER) in beam-search decoding for all taske Beam-width for NMT and ASR models is 5 and 20, respectively, We speculate that this is because beam-search inference, which is a sequence of greedy token-level decisions, benefits more from token-level Bayesian model averaging. At the same time, both combination strategies yield equivalent seucher-forcing mean length- normalized negative-log lag-liikehhood on reference data. This may be because the models in the ensemble yield consistent predictions on in-domain data, in which case the two combinations will yield similar probabilities. Further experiments in this work will use ensemble combination, as it yields marginally
combination consistently yields marginally
hypotheses obtained from better predictive performance. Additional analysis and results are available in appendix
Table 2: Sequence-level Error Detection % Prediction Rejection Ratio in
decoding.
ENS-PrEx TU
ENS-ExPr TU
ENS-PrEx KU
ENS-ExPr KU
Task
R(11)
H111)
AM
Ar(3))
F.w
61.2
65.9
60.5
59.0
61.0
55.1
60.2
LSP
LTO
68.8
71.7
67.0
67.9
67.4
64.2
63,2
64,8
57.5
63.2
62.7
61.4
AMI
57.2
66.8
52.3
61.5
54.2
51.8
50.6
63.5
25.9
49.2
49.0
56.4
ENDE ENER
28.1
45.8
27.8
45.5
27.3
26.3
25.6
28.9
15.9
23.9
26.2
25.4
nwt14
25.9
39.0
29.8
32.4
20.3
29.6
Sequence-level Error Detection We now investigate whether the sequence-level uncertainty mea- sures can be used to detect sentences which are challenging to translate or transcribe. In the following experiment model's 1-best hypotheses are sorted in order of decreasing uncertainty and incremen- tally replaced by the references. The mean Bestencce-LEU (sBLEU) sentence- WER (sWER) is plotted against the fraction of data replaced on a rejection curve. If the uncertainties are informative, then the increase in sBLEU or decrease in sWER should be greater than random (linear). Rejection curves are summarised using the Prediction Rejection Ratio (PRR) (Malinin 2019: Malinin et al. 2020), describe in appendix D.2] which is 100% if uncertainty estimates perfectly correlate with sentence BLEU/WER, and 0% if they are uninformative. In these experiments information only from the 1-best hypothesis is considered While the 1-best hypotheses obtained from product-of- expectation combination, we consider uncertainty estimates obtained by expressing the predictive posterior both product-of-expeetatios (ENS-PrEx) and oopectation-o-pproucce (ENS-ExPr).
Table 2 shows several trends. First, measures of total uncertainty yield the best performance. Furthermore, joint-sequence estimates of total uncertainty consistently outperform chain-rule based estimates. This is because, unlike chain-rule approximations, joint-sequence approximations do not account for probabilities of non-generated tokens and only consider probabilities along the 1 --est hypothesis, and therefore assess its quality. This is consistent with results for unstructured- prediction (Malinin 2019). Sccond, measures derived from a product-off-expecce exxeectaaii predictive
"BLEU was calculated using sacrebleu (Post 2018 and WER using sclite.
A Asessment of uncertainty derived from all hypotheses in the heam are analyzed in appendix

---

Published as a conference paper at ICLR 2021
posterior tend to yield superior performance than their expectation-of-prant propduttss counterparts. However, this does not seem to be a property of the 1-best hypotheses, as results in appendix [ on hypotheses obtained from a expectation sorroonnof-ooooooo -f---oduucss ensemble show a similar trend. Third, out of all measures of knowledge uncertainty, joint-sequence RMI performs best. Finally, the performance gap between chain-rule and joint-sequence estimates of total uncertainty is larger for NMT. This is because compared to ASR, NMT is task with intrinsically higher uncertainty, and therefore more irrelevant information introduced by considering the probabilities of non-generated tokens.
The results also show that uncertainty-based rejection works better for ASR than NMT. The issue lies in the nature of NMT it inherently difficult to objectively define a bad translation M While WER is an objective measure of quality, BLEU is only a proxy measure. While high sBLEU indicates a good translation, low sBLEUU does not necessarily indicate poor one. Thus, a model may yield a low uncertainty, high-quality translation which has little word-overlap with the reference and low sBLEU, negatively impacting PRR. A better, but more expensive, approach to assess uncertainty estimates in NMT is whether they correlate well with human assessment of translation quality.
Table 3: Token-level Error Detection AAPP for LibriSpeech in Ream-Search Decoding regime.
Test Data
ENSM-PIEX TU
ENSM-EXP: TU
ENS-PIEx KU
ENS-ExPr KU
% TER
H
P
H
P
I
K.
MM
I
KC
M
LTC
34.7
36.3
32.4
33.4
32.7
28.0
27.6
26.5
25.9
27.4
3.8
LTO
42.4
43.3
39.0
39.1
40.9
37.1
36.1
30.8
33.6
35.3
10.2
AMI
71.7
74.6
68.3
70.4
71.8
67.9
68.7
59.2
64.4
66.6
41.2
Token-level Error Detection We now assess whether token-level uncertainties can be used to detect token-level errors in the models' 1-best hypotheses. Note that token-level error labelling is ill-posed for translation, where correct tokens can be mislabelled as erтors due valid word re-arrangements and substitutions. Thus, token-level error detection is only investigated for ASR. Ground-truth urror-labels are obtained by aligning the hypotheses to the references using the SCLITE NIST scoring tool and marking insertions and substitution&] Performance is assessed via area-under a Precision-Recdt curve. Random performance corresponds the baseline recall, which equal to the token error rate. Results in table 3 are consistent with the previous section. First, measures of total uncertainty outperform measures of knowledge uncertainly. Second, estimates derived from conditional log- scores P of the generated token outperform the entropy H. of the levenllevel predictive posterior. This is because the latter relates to probability of an error at this position, while the former relates to the probability of this particular token being an error. Finally, deriving uncertainties from product-of-expectatin token-level predictive posterior PEz(31 1y 2;, D) yields superior results.
(1)
Table 4: OOD Detection % ROC-AUC in Beam-Search decoding regime for ASR and NMT.
Tisk
ODD
Data
T
R(B))
ENS-PIEx TU
C-W
ENS-ExPr TU
(B) S:w
(B FCow
((B)
CCWW
ENS-PrEx KII
(2(
C--W
Jig (2) S-WW
R(B) L-WW
ENS-ExPy KII
1TT
1
76.7
76.3
75.5
76.4
76.2
77.0
75.0
76.4
77.0
76.6
77.0
76.6
76.1
73.9
74.8
74.0
76.7
76.3
76.9
76.4
73.4
ASR
AMI
96.5
97.5
97.6
97.9
96.4
97.0
97.9
97.2
96.4
94.9
96.2
94.8
96.2
97.4
96.4
93.0
90.1
94.8
95.7
95.9
94.8
95.8
C-FR
99.9
99.9
99.7
99.8
99.7
99.9
99.9
99.9
99.8
89.7
81.0
99.8
99.8
90.1
98.8
LTC
10
65.7
71.8
64.7
72.8
72.5
72.2
73.3
57.1
68.4
71.8
74.1
65.0
73.2
72.9
72.6
75.0
58.1
69.3
72.1
73.9
NMT
PRM
10
83.9
82.7
82.7
80.7
83.5
84.5
96.4
96.7
96.6
96.9
96.7
96.9
96.2
96.5
69.3
72.1
95.0
93.9
96.7
96.4
95.2
94.2
L-FR
18.5
22.2
72.1
70.9
22.7
69.9
27.1
21.7
23.0
22.9
65.2
71.2
74.8
79.6
23.4
50.1
73.8
80.7
L-DE
10
40.8
28.7
34.9
35.1
36.1
38.3
76.1
74.4
78.0
79.9
80.1
82.1
89.2
41.9
41.1
73.3
78.4
81.2
88.8
Out-of-Domaie input Detection We now consider out-of-domain input (anomaly) detection. The goal is use uncertainty estimates to discriminate between in-domain test data and a selection of out-
Provided that the model is high-performing in general.
Detecting deletions is, in general, far more challenging task.
7

---

Published as a conference paper at ICLR 2021
of-domain (OOD) datasets. Performance is assessed via area under ROC-curve (ROC-AUC), where 100% is ideal performance, 50% is random and below 50% indicates that the model yields lower uncertainty for the OOD data. Results are presented in table 4 additional results in appendices |
F
First, let's examine OOD detection for speech recognition. Three OOD datasets are considered, each covering a different form of domain shift. First, LibriSpeech test-other (LTO), which represents set of sentences which are more noisy and difficult to transcribe. Second, the AMI meeting transcription dataset (Kraajj et al. 2005), which represents spoken English from a different domain, mismatched to LibriSpeech, which consist of books being read. Finally, we consider speech in different language (French), taken from the Common Voice Project (Ardila et al. 2019). The results show that OOD detection becomes easier the greater the domain mismatch. Curiously, there is marginal difference between the performance of measures of uncertainty. This likely because ASR models tend to be very 'sharp' and are naturally more entropic in mismatched conditions.
Now let's consider OOD detection for WM Anglish-German machine translation. The following OOD datasets are considered. First, the LibriSpeech test-clean (LTC) reference transcriptions, which are OOD in terms of both domain and structure, as spoken English is structurally distinct from written English. Second, newstest1 14 sentences corrupted by randomly permuting the source-tokens (PRM). Third, French and German source-sentencee from newstest14 (L-FR and L-DE). Results show that discriminating between spoken and written English is challenging. In contrast, it is possible to near-perfectly detect corrupted English. Interestingly, detection of text from other languages is particularly difficult. Inspection of the model's output shows that the ensemble displays pathological copy-through effect, where the input tokens are copied to the output with high confidence. As result, estimates of total uncertainty are lower for the (OOD) French or German data than for (ID) English data. Notably, estimates of knowledge uncertainty, especially reverse mutual information (RMI), MCR. and M are affected far less and discriminate between the in-domain and OOD data. This effect true in general, but is especially pronounced when the copy copy-through effect is triggered. This highlights the value of the RMI, for which asymptotically exact approximations can be obtained.
Clearly, ASR ensembles are better at OOD detection than NMT ensembles. This is expected, as ASR models receive a continuous valued input signal which contains information not only about the content of the speech, but also the domain, language, speaker characteristics. background noise and recording conditions. This makes the task casier, as the model conditions on more information. This is also why ASR has low intrinsic data uncertainty and why the best OOD detection performance for ASR is obtained using measures of total uncertainty. In contrast, NMT models only have access to a sequence of discrete tokens, which contains far less information. This also highlights the value of knowledge uncertainty, as it disregard the high intrinsic data uncertainty of NMT.
An interesting effect, which is fully explored in appendix G is that when considering all the hy- potheses within the beam for uncertainty estimation, it is beneficial for NMT to use a higher importance- imporrance-eeightingg temperature (T 10), increasing the contribution from competing hypotheses. In contrast, this detrimental for ASR, and temperature is kept at T = 1. We hypothesise that this may be an artifact of the multi-modality of translation multiple hypotheses could be equally good and contribute valuable information. In contrast, in ASR there is only one correct transcription, though not necessarily the 1-best, and considering competing hypotheses is detrimental.
The results also show that chain-rule and joint-sequence approximations yield similar performance and that, with the exception of M using information from the full beam yields benefits mi- nor improvements compared during using just the 1-best hypotheses. Uncertainties derived from PEP (3) x. D) and PpE(3) æx D) yield comparable performance, with the exception of mutual infor- mation and EPKL, where PEr (y jz. D) yields consistently poorer performance. This suggests that PeR ( az D) yields more robust uncertainty estimates.
Comparison to heuristic uncertainty measures We close with a comparison of the proposed measures of knowledge uncertainty to the "heuristic' measures describes in (Wang et al. 2019; Fomicheva et al. 2020; Xiao et al., 2019). These measures, and our modifica- tions thereof, are detailed in appendix!! We examine the variance of the length-normalize probability and log-pno pog-obbability of hypotheses across the ensemble, as well as the cross-hypothesis WER/BLEU. The use of more than the b-best hypothesis is our extension for the variance-based measures.
information-thhoone
8

---

Published as a conference paper at ICLR 2021
All of these measures aim to evaluate the diversity of ensemble of models in different ways. In this regard they are all measures of knowledge uncertainry. Their main limitations, as originally presented, are the following. All measures focused on the diversity in the probabilities surface forms of only the 1 1-best hypothesis. While sufficient in some tasks, such as sequence-error detection, this prevents them from fully capture information about the behavior of the space of possible crasslaions/transciiiionnt In this regard, the information theoretic measures presented in our work are an advantage, they naturally allow to do this. We attempt address this for the variance-based measures by considering the importance -epporaace-weighted variance of the probability/log- probaabiiiy of each hypothesis the beam. While not strictly rigorous, this nonetheless attempts address the problem. Such an extension to Cross-BLEU/WERR is not possible, as it is not clear how to match up Pross-BLLEU/WFR have
different hypotheses across all decodings of each model in the ensemble. the additional limitation of needing a separate decoding of each model in the ensemble, which is undesirable and expensive. Finally, it is likely that there is bias towards longer hypotheses as being more diverse, as there is a greater chance of surface form mismatch.
The results in table D show that information-theoreti measures consistently yield better performance, BLES/WBEE typically yields the worst performance,
though sometimes only marginally. especially for NMT. Finally, including information from competing hypotheses can be advantageous for the variance based measures. However, sometimes the performance is degraded this is because the information was integrated in an adhoc, rather than theoretically meaningful, fashion. We also show in appendix [ that length- nongghiaommliiaiinn, which was used inconsistently in prior work, is important both for these measures, and appendix H for the cetematiio-thhooretee ones.
Table 5: Comparison of info-theoretic and heuristic measures on OOD detection (% ROC-AUC).
OOD
Info: Theor.
leurisic
Task
Data
T
B
(2)
M
(23)
GL [n P]
X-DEEU
X-WER
LTD
77.0
76.6
76.1
73.9
72.0
72.7
74.6
72.7
74.3
71.8
ASR
AMI
94.8
96.2
96.1
97.4
85.6
87.3
96.7
95.8
95.9
95.8
C-FR
99.9
99.8
9988
82.0
77.6
98.4
98.0
99.5
99.7
LTC
10
72.2
73.3
75.0
68.6
68.7
72.3
65.1
NMT
PRM
10
96.7
96.9
96.5
96.2
88.7
88.8
93.0
93.4
80.8
73.5
72.1
70.9
76.4
74.8
79.6
77.7
72.2
5 CONCLUSION
This work investigated applying general, probabilistically interpretable ensemble-based uncertainty estimation framework to structured tasks, focusing on autoregressive models. A range of information- theoretic uncertainty measures both at the token level and sequence level were considered, including a novel measure of knowledge uncertainty called reverse nntuullinformation (RMI). Two types of Monte-Carlo approximations were proposed one based the entropy chain rule, and the other on sequence samples. Additionally, this work examined ensemble combination through both token-level and sequence-level Bayesian model averaging. Performance baselines for sequence and token-level error detection, and out-of-domain (OOD) input detection were provided on the WMT' 14 English- French and WMT' 17 English-Germa German translation datasets, and the LibriSpeech ASR dataset. The results show that ensemble-base based measures of uncertainty are useful for all applications considered. Estimates of knowledge uncertainty are especially valuable for NMT OOD detection. Crucially, it was shown that RMI is consistently the most informative measure of knowledge uncertainty for structured prediction. Notably, was found that token-level Bayesian model averaging consistently yields both marginally better predictive performance and more robust estimates of uncertainty. However, it remains unclear why this is the case, which should be investigated in future work. Future work should also investigate alternative ensemble generation techniques and compare ensemble- based uncertainty estimates to the task -sec-fie soffidence-scon estimates previously explored for ASR and NMT. Another interesting direction is assess the calibration of autoregressive ASR models.

---

Published as a conference paper at ICLR 2021
REFERENCES
Dario Amodei, Chris Olah, Jacob Steinhardr, Paul F. Christiano, John Schulman, and Dan Mané. Concrete problems in Al safety. lattp://axxi..ort htlgg//bs/166000 06565) 2016. arXiv: 1606.06565,
Rosana Ardila, Megan Branson, Kelly Davis, Michael Henretty, Michael Kohler, Josh Meyer, Reuben Morais, Lindsay Saunders, Francis M Tyers, and Gregor Weber. Common voice: A massively- multilingual speech corpus. arXiv preprint 1iXix:191220667000 2019.
Arsenii Ashukha, Alexander Lyzhov, Dmitry Molchanov, and Dmitry Vetrov. Pitfalls of in-domain uncertainty estimation and ensembling in deep learning. In International Conference on Learning Representations, 2020. URL httt:::::// /ttpst http:///ppeeeviieeeeee
net htw.eeewweeeeoooeeeeeeeeeeee net::foooumm.ddBJ335g......
Dzmitry Bahdanau. Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning align and translate. In Proc. International Conference on Learning Representations (ICLR), 2015.
Nicholas Carlini and David A. Wagner, Adversarial examples are not easily detected: Bypassing ten detection methods. CoRR, 2017.
07263
William Chan, Navdeep Jaitly, Quoe V. Le. and Oriol Vinyals. Listen, Attend and Spell. http: 1/arxiv. wwwp/r/////baooooo 0 1211 2015. 1508.011
Zhehuai Chen, Yimeng Zhuang, and Kai Yu. Confidence measures for ete-based phone synchronous decoding. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4854. IEEE, 2017.
Thomas M Cover and Joy A Thomas. Elements of information theory. John Wiley & Sons, 2006.
Stefan Depeweg, José Miguel Hernández- Finale Doshi-Velez, and Steffen Udluft. Decompo- sition of uncertainty for active learning and reliable reinforcement learning in stochastic systems. star, 1050:11, 11, 2017.
Bryan Eikema and Wilker Aziz. Is map decoding all you need? the inadequacy of the mode in neural machine translation. arXiv preprint arXiv:2005. 10283, 2020.
G. Evermann and P.C. Woodland. Large vocabulary decoding and confidence estimation using word posterior probabilities. In Proc, of IEEE Intl. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2000.
Marina Fomicheva, Shuo Sun, Lisa Yankovskaya, Frédéric Blain, Francisco Guzmán, Mark Fishel, Nikolaos Aletras, Vishrav Chaudhary, and Lucia Specia. Unsupervised quality estimation for neural machine translation. arXiv preprint arXiv:2005. 10608, 2020.
Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan. Deep ensembles: A loss landscape perspec- tive. arXiv preprint
2019.
Yarin Gal and Zoubin Ghahramani. Dropout as Bayesian Approximation: Representing Model Uncertainty in Deep Learning. In Proc. 33rd International Conference on Machine Learning (ICML-16), 2016.
Mark Gales and Steve Young. The application of hidden Markov models in speech recognition. Now Publishers Inc, 2008
Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning. pp. 369-376, 2006.
Jiatao Gu, James Bradbury, Caiming Xiong, Victor OK Li, and Richard Socher. Non-autoregressive
neural machine translation. arXiv preprint arXiv: 1.02811 2017.
Dan Hendrycks and Kevin Gimpel. A Baseline for Detecting Misclassified and Out-of- 02136 2016.
Distribution Examples in Neural Networks. arXiv:1610.02136.
10

---

Published as a conference paper at ICLR 2021
Geoffrey Hinton, Li Deng, Dong Yu, George Dahl, Abdel rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara Sainath, and Brian Kingsbury. Deep neural networks for acoustic modeling in speech recognition. Signal Processing Magazine, 2012.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. arXiv preprint wrXhv:1904009755 2019.
Andreas Kirsch, Joost van Amersfoort, and Yarin Gal. Batchbald: Efficient and diverse batch acquisition for deep bayesian active learning, 2019.
Philipp Koehn. Statistical machine translation. Cambridge University Press, 2009.
Wessel Kraaij, Thomas Hain, Mike Lincoln, and Wilfried Post. The ami meeting corpus. 2005.
Aviral Kumar and Sunita Sarawagi. Calibration of encoder decoder models for neural machine translation. arXiv preprint arXiv 0000000000002.... 2019.
B. Lakshminarayanan, A. Pritzel, and C. Blundell. Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles. In Proc. Conference on Neural Information Processing Systems (NIPS), 2017.
Hank Liao and Mark JF Gales. Uncertainty decoding for noise robust speech recognition. In Proceedings of Interspeech, volume 37. Citeseer, 2007.
A. Malinin, A. Ragni, M.J.F. Gales, and K.M. Knill, Incorporating Uncertainty into Deep Learning for Spoken Language Assessment. In Proc. 55th Annual Meeting of the Association for Computational Linguistics (ACL), 2017.
Andrey Malinin. Uncertainty Estimation in Deep Learning with application to Spoken Language Assessment. PhD thesis, University of Cambridge, 2019.
Andrey Malinin and Mark JF Gales. Reverse kl-divergence training of prior networks: Improved uncertainty and adversarial robustness. 2019.
Andrey Malinin, Bruno Mlodozeniec, and Mark JF Gales. Ensemble distribution distillation. In International Conference Learning Representations, 2020. URL httppss///peereeeieee
Sott/Eorum?id-232 BygSP 6Vtvr
Tomas Mikolov, Martin Karafiát, Lukás Burget, Jan Cernocky, and Sanjeev Khudanpur. Recurrent Neural Network Based Language Model. In Proc. INTERSPEECH, 2010.
Tomas Mikolov et al. Linguistic Regularities in Continuous Space Word Representations. In Proc. NAACL-HLT, 2013.
AbdeIrahman Mohamed, Dmytro Okhonko, and Luke Zettlemoyer. Transformers with convolutional context for ast. arXiv preprint 1 1660, 2019.
Kevin P. Murphy. Machine Learning. The MIT Press, 2012.
Myle Ott, Michael Auli, David Grangier, and Mare' Aurelio Ranzato. Analyzing uncertainty in neural machine translation. arXiv preprint 1803. 00047,007770 2018a.
Myle Ott, Sergey Edunov, David Grangier, and Michael Auli. Scaling neural machine translation. arXiv preprint arXiv: 1806.0008 2018b.
Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli, fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-HLT 2019: Demonstrations, 2019.
Yaniy Ovadia, Emily Fertig, Jie Ren, Zachary Nado, Sculley, Sebastian Nowozin, Joshua V Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. Advances in Neural Information Processing Systems, 2019.
11

---

Published as a conference paper at ICLR 2021
Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. Librispeech: an corpus based on public domain audio books. In 2015 IEEE International Conference on Acoustics, Speech IEEE, 2015.
and Signal Processing (ICASSP), pp.
Matt Post. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Pupers, pp. 186-191, Belgium, Brussels, October 2018. As- sociation for Computational Linguisties. URL https://wwtttt hwwp::wwwwwwww.............. http::/wwww.we..../attttt....ooe wrt.h/oooo..///.ooo.oooooooooo//
118-6319
Anton Ragni, Qiujia Li, Mark JF Gales, and Yongqiang Wang. Confidence estimation and dele- tion prediction using bidirectional recurrent neural networks. In 2018 IEEE Spoken Language Technology Workshop (SLT), pp- 204-211. IEEE, 2018.
Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. arXiv preprint
2015.
Karen Simonyan and Andrew Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recognition. Proc. International Conference on Learning Representations (ICLR), 2015.
L.. Smith and Y. Gal. Understanding Measures of Uncertainty for Adversarial Example Detection. In UAI, 2018.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998-6008, 2017.
Shuo Wang, Yang Liu, Chao Wang. Huanbo Luan, and Maosong Sun. Improving back-translane with uncertainty- based confidence estimation. arXiv preprint arXiv:1999.00157 2019.
Tim Z Xiao, Aidan N Gomez, and Yarin Gal. Wat heb je gezegd? detecting out-of-distribution translations with variational transformers. 2019, URL http: http:://bayssiaddeeeeeaaaainn orq/2019/papeeze pdf
12

---

Published as a conference paper at ICLR 2021
A DERIVATIONS OF UNCERTAINTY MEASURES
The current appendix details token-level measures of uncertainty for autoregressive models and provides the derivations of the sequence-level measures of uncertainty discussed in section B as well as extended discussions of their theoretical properties.
A.1 TOKEN-LEVEL UNCERTAINTY ESTIMATES
As was stated in section token-level ensemble-based uncertainty estimates for autoregressive models are isomorphic to un-structured uncertainty estimates (Malinin 2019). However, for completeness, they are described in the current section.
First, let's consider the predictive posterior P(n |1., æ. D). As discussed in section 3 the token-level predictive posterior of an autoregressive models can be obtained in two ways. One corresponds to token-level Bayesian model averaging, while the other corresponds to sequence-level Bayesian model averaging. The first be expressed follows:
Pl D) Eq(o)
1
Σ
P(llNet, 0(m)), ((m)) q())
(18)
M
While the latter is expressed like so:
P( D) P(t,, yc1,, x. D) P(yct, 2:, D
E (0) P(u Met. 2, 0) 1
Σm P(u,, Mー1. ((m))
M
(19)
Eg(e) P(y)) x.
Em P(y , x, 0lm!)
Clearly, leven-level BMA is more consistent with estimating the uncertainty in the prediction of the current token yl, regardless of how the context tokens were generated. In contrast, sequence-level BMA considers how the entire sequence was generated.
Regardless how the predictive posterior is obtained, the estimate of total uncertainty will be given its entropy:
P(
Σ P(y = wklyen D) P Wh 2, D)
(20)
Total Uncertainty
Furthermore, by considering the mutual information between yt and 0 we can obtain measures of total uncertainty, knowledge uncertainty and expected data uncertainty:
P(y yet, az; 0)
I Ily 0 01y,1,,,,11 DD ( Priwileee
P(yl yel. x:; D)
Knowledgee Uncertainty
(21)
H (P(Nllye,,,
HP yzcs x. 0)]
Total Uncertainty
Expocted Data Uncertainty
Alternatively, the expected pair-wise KI-divergence (EPKL) between models in the ensemble at the token level can also be considered:
K[y, 0 flyzi, D) =
E (()))(()) [LL P 20 0)]
Knowledge Uncortainty
(22)
Ep(ulveei. [BqrO] In P(wller,, θ)] E.(()) [H[P Ver 0)]]
Expocted Uncertainty
where q()) = @)). This yields alternative measure of ensemble diversity which Jensen- derived upper bound on mutual information. Both EPKL and mutual information yield the same estimate of data uncertainty. We can also consider novel measures of diversity, and therefore knowledge uncertainty, called reverse mutual information (RMI) M defined as follows:
P(n F-t. ac; D)1
M [3) ye x D] = Eg(0) (p(xnleen):
In
P(u (e æ;; 0)
Knowledge Uncertainty
(23)
Ep(yi (ee.., [gcoo] |- In P P( yel, x; 0)]] [P 2DD
Tocal Uncertainty
13

---

Published as a conference paper at ICLR 2021
This is effectively the reverse-KL KL divergence counterpart to the mutual information, which is the mean KL-divergence between each model and the predictive posterior. It yields the same estimates of total uncertainty. Just like for the sequence-level measures of uncertainty, it is trivial to derive relationship between these token-level measures of ensemble diversity:
(24) Thus, RMI is the difference between the EPKL and mutual information. Consequently, while mutual information, EPKI. and RMI all yield estimates of knowledge uncertainty, only mutual information "cleanly' decomposes into total uncertainty and data uncertainty. In contrast, EPKL does not yield clean measures of total uncertainty and RMI does not yield clean measures of data uncertainty.
M 33, O1 xt D
K. I Olyc,, D I Ilyr. 01y-c1,0, x, 2)
All these measures of uncertainty considered above use information from the full distribution over tokens y/, However, we also examine measures which only consider the probability assigned to the predicted token Wy. Firstly, we can examine the log- likelihood of the predicted token under the predictive posterior:
P In P y/ = wollee.. x, D)
(25)
This is a measure of total uncertainty. Alternatively, we can consider the mean negative Point-wise Mutual Information (Murphy. 2012) between the model ((r) and the prediction y) across all models:
P ( ان y<l F. P wi y-l, x. D)
Mwi
In
(26)
Pointwise Mutual Information
This is effectively the RMI at the point prediction, rather than expectation over all classes.
Mylicelft
Pork
Popke"l
Poye- 2"1
(a) Mutual Information
(b) Reverse Mutual Information
(c) Expected Pairwise KL
Figure 1: Measures of Ensemble Diversity Direction of KL Divergence
For autoregressive models, these measures represent uncertainty in the prediction given specific combination input æ and context y=a- However, at the beginning a sequence token-level measures of uncertainty are more sensitive to the input and the end of a sequence become more sensitive to the context y-t--
A.2 DERIVATION OF SEQUENCE-LEVEI MONTE-CARLO APPROXIMATIONS
In the current section we detail the derivations of joint-sequence and chain-rule derived Monte-Cari approximations of sequence-level measures of uncertainty defined in Crucially, they make use of the chain rules of entropy and relative entropy (Cover & Thomas 2006):
1
HPP x 0)]
H[P( ( x. yei. 0)]
(27)
L
([ = Σ
1
Ep( (L x
(28)
1=1
We omit the derivation for the entropy of the predictive posterior they are straightforward. Instead, we focus measures of knowledge uncertainty. First, lets consider a direct joint-sequence Monte- Carlo estimate for mutual information:
S
xx ((+))
(S) y. 0 D] ~
xr, D) (0)) P(yl (())), q())
(29)
S
[(e)
P P
14

---

Published as a conference paper at ICLR 2021
Clearly, this is inference inefficient, as it requires independently sampling from each model. As result, we canmot obtain this estimate of mutual information free during sampling (or Beam-Search decoding) from the ensemble's 's predictive posterior. However, is possible to obtain inference efficient approximation by consider the chain rule of relative entropy:
[yy, a. D Bq(())
Σ 1=1
Ver, 2, yet, x. D)]]
(30)
I
Here, have expressed mutual information However, by replacing the expectation with respect to each individual model by the expectation with respect the predictive posterior, we obtain the following approximation:
of expected token-level KI.-divergences.
i y. 0 01 DJ x L 10)
Ep D)
00 0) œ, D)[]
ap( [T 0,y,,,,, 2D]
(31)
L
L Ilyn 2 D), Vyli C y P(ylar, D)
16
This reduces to the sum of token-level mutual information along hypotheses drawn from the ensem- ble's predictive posterior. However, this approximation will not longer yield equation i the limit as S - Nevertheless, this approximation, while inexact, may still be useful in practice. We also examine sequence-level EPKL. Similar to the exact Monte-Carlo estimate of mutual information, exact Monte-Carle estimation for EPKL is also inference in-efficient, as it requires sampling for all models individually:
Ply m. 0(+)) P(y x., 0(o)) ( P(ylaa, 0(),, (())) -()))
(())
y. D,
D
5
(32)
As before, the chain-rule of relative entropy and replace sampling from each individual model with sampling from the predictive posterior:
R y. 0 D) E
(((0)1(0)
I
Ep(ver (m (
0) y= x, 0]]]
2 E (())
Ep () o O 0]]]
T,
1=1
(33)
E Ep pD) [Cly, 01y
(
K(mn, 0 (1 DJ, Vy C cy(() ~ P(ylæ, D)
This approximation becomes the sum of token-level EPKL along hypotheses generated by the ensemble's 's predictive posterior.This approximation will also not yield equation in the limit as 00.
However, while asymptotically exact inference-cffician Monte-Carlo estimates of sequence-level knowledge uncertainty cannot be obtained via mutual information and EPKL, they can by considering the novel measure RMI equation [ which defined as an expectation with respect to the predictive posterior. A direct seint-sequence Monte-Carke approximation
be obtained follows:
(5) x ! p
P P
(S) y, 0le D] ~ Eg(o)
o D) (x))
P x,, D)
(34)
5
[(a)
s=1
15

---

Published as a conference paper at ICLR 2021
Similarly, also obtain an asymptotically exact chain-rule approximation:
((5) y 0 x. DJ
(0)
P
KL PX) 0]]]
0)]]]
1
Bp [M[y. 0/y,,,,0),,,,1))
(35)
L
1
Myn 0y. a, Yy ~(ylee
5
A.3 IMPORTANCE WEIGHTING
As discussed in section For the Monte-Carlo approximations for sequence-level measures of uncertainty be used with beam search, they need to be adjusted such that uncertainty associated with each hypothesis (b) within the beam B is weighted proportion to it's probability:
beam-scarch decoding can be interpreted as a form of importance sampling.
exp + In P(y x. D)
y (b) B, TTP
(36)
ΣΦ exp + In P(y x D)
All chain-rule derived measures of uncertainty will be expressed as follows:
B
2
HP x. 92 D)
(37)
b=1
B
f(B) 0 D]
RV
Tb
Ily,, 0 2/ 2, D)
(38)
(b)
b=1
B
R(B) y, 0/, D]
Th
K/yi,, 0l x. D)
(39)
Lb
b=1
B
(y.00,0
2
Th
M|y 0 21 D1
(40)
6
b=1
Joint-sequence measures uncertainty are similarly modified:
B
(B) P(yla,, D)
In (2)
(41)
[(b)
d=1
P (1) D) In
M 0|x)
(42)
P(y (b) x 0)
B EXPERIMENTAL CONFIGURATION
The current section of the appendix provides both a description of the datasets and details of the models and experimental setups used in this work.
B. ASR MODEL CONFIGURATION
In this work ensembles of the VGG-Transformer sequencceto--sequeec ASR model (Mohamed et al. 2019) were considered. An ensemble of 6 models was constructed using different seed for both initialization and mini-batch shuffling in each model. We used ensembles of only VGG-Transformer models for inference. We used the Fairseq (Ott et al. 2019) implementation and training recipe for this model with modifications. Specifically, models were trained fixed learning rate for 80
16

---

Published as a conference paper at ICLR 2021
Table 6: Description of ASR Datasets
Dataset
Subset
Hours
Utterances
Words Utterance
Domain
Train
960
281.2K
33.4
Dev-Clean
5.4
2703
17.8
Librispeech
Dev-Other
5.3
2864
18.9
Story Books
Test-Clean
5.4
2620
20.1
Test-Other
5.1
2939
17.8
AMI
Eval
12643
7.1
Meetings
Common- Voice FR
14760
9.5
Test
General
epochs, where an epoch is a full pass through the entire training set. Checkpoints over the last 30 epochs were averaged together, which proved to be crucial to ensuring good performance. Training took 8 days using 8 V100 GPUs. Models were trained on the full 960 hours of the LibriSpeech dataset (Panayotov et al. 2015) in exactly the same configuration described in (Mohamed et al.) 2019). LibriSpeech is a dataset with 1000 hours of read books encoded in 16-bit, 16kHz FLAC format. The reference transcriptions were tokenized using a vocabulary of 5000 tokens, as per the standard recipe in Fairseq for the "GG-traasformer (Ott et al. 2019) Mohamed et al. 2019). For OOD detection we considered the evaluation subset of the AMI dataset (Kraijj et 2005), which is a dataset of meeting transcriptions, as well as the Russian and French datasets of the Common Voice Project (Ariila et al. 2019), which consist of people reading diverse text from the internet. AMI encoded in 16-bit, 16Khz WAV format. Common Voice data was stored as 24kHz. 32-bit MP3 files which were converted into 16-bit 16kHz WAV format via the SOX tool. WER was evaluated using the NIST SCLITE scoring tool.
B.2 NMT MODEL CONFIGURATION
Table 7: Description of NMT Datasets
Dataset
Subset
LNG
Sentences
Words / Sent.
Domain
En
29.2
WMT' 14 EN-FR
Train
40.8M
Fr
33.5
Policy, News, Web
En
26.2
WMT' 17 EN-DE
Train
4.5M
De
24.8
Policy, News, Web
En
27.0
Newstest
Fr
3003
32.1
News
28.2
En
19.0
Khresmoi-Summary
Dev+Test
Fr
1,500
21.8
Medical
De
17.9
This work considered ensembles of Transformer-Bip (Vaswani et al. 2017) neural machine translation (NMT) models. An ensemble 10 models was constructed using a different seed for both initialization and mini-batch shuffling in each model. NMT models were trained on the WMT" 14 English-Frente French and WMT' 17 Geeeninh-German datasets. All models were trained using the standard Fairseq (Ott et (2019) implementation and recipe, which is consistent with the baseline setup in described in (Ott al. 2018b). The data was tokenized using a BPE vocabulary of 40,000 tokens as per the standard recipe (Sennrich et al. 2015). For each dataset and translation direction an ensemble of 10 models was trained using different random seeds. All 10 models were used during inference. Models trained on WMT' 17 Anglish-German were trained for 193000 steps of gradient descent, which corresponds to roughly 49 epochs, while WMT' 14 English-Frent French models were trained for 800000 steps of gradient descent, which corresponds to roughly 19 epochs. Models were checkpoint-average the last 10 epochs. All models were trained using mixed-precision training. Models were evaluated newstest14 14, which treated in-domain data. OOD data constructed by considering
across
17

---

Published as a conference paper at ICLR 2021
BPE-token permuted and language-flipee versions of the newstest14 dataset. Furthermore, the prresmoi-sunmman medical dataset as well the reference transcriptions of the LibriSpeech test-clean and test-other datasets were also used OOD evaluation datasets. All additional datasets used consistent tokenization using the 40K BPE vocabulary.
C PREDICTIVE PERFORMANCE ABLATION STUDIES
The current section provides additional results assessing the cediitiveeperformaane and negative log- rig-likelihood of ensembles of autoregressive NMT and ASR models. Additionally, we include an ablation study of how the number of models in an ensemble affects the performance in terms of BLEU and NLL. Tables 8 and include expanded set of results. Crucially the results show that for all languages, tasks and datasets product-of-exppecaaiin exception) in beam- search decoding and and a consistently lower NLL on reference transcriptions and translations.
yields superior performance (with one
Table 8: Predictive performance in terms of BLEU, %WER and NLL on newstest14 and LibriSpeech.
NWT' 14 BLEU
MED BLEU
NWT' NLL
MED NLL
Model
EN-DE
EN-FR
EN-DE
EN-FR
EN-DE
EN-FR
EN-DE
EN-FR
Single ANS-PrEx
28.8 #0.2
45.4 0.3
29.9 +05
51.3 +0.5
1.46 = 0.02
1.10 0000
1.29 = 0.01
0.82 000
30.1
46.5
32.1
52.7
1.33
1.04
1.16
0.77
ENS-ExPr
29.9
46.3
31.4
52.8
1.36
1.05
1.19
0.78
Table 9: Predictive performance in terms of BLEU, % WER and NLL newstest14 and LibriSpeech.
ASR WER
ASR NLL
Model
LTC
LTO
AMI
LTC
LTO
AMI
Single
5.6 +0.2
14.7 20.5
78.7 1114
0.34 10:00
0.86 1002
5.78 20.32
RNE-PrEx
4.2
11.3
50.4
0.20
0.48
4.05
ENS-ExPr
4.5
12.6
53.4
0.23
0.58
4.62
Finally, we present an ablation study, which shows how the predictive performance varies with the number of models in an ensemble. The ablation shows several trends. Firstly, both BLEU/WER and NLL begin to shown diminishing returns for using more models. This suggests that using 4-6 NMT models and 2-3 ASR models will allow most of the gains to be derived at half the cost of a full 10 6-model ensemble. Secondly, shows that the advantage of a product-of-ext expectttatisns combination is remains consistent with the number of models. This shows that regardless of the number of models available, is always better to combine as a product-of-expectatiin
32.0
1.45
PHEX NWT'14
NWT14
31.5
PHEX MED
31.0
1.35
30.5
1.30
30.0
20.5
MEX
1.25
ExPr NWT114
20.0
PrEx MED
1.20
28.5
1.15
8
10
2
4
10
Ensemble Size (models)
Ensemble Size (models)
(a) NMT EN2DE BLEU
(b) NMT EN2DE NLL
Figure 2: BLEU and NL.I ablation study. Shading indicates +2
18

---

Published as a conference paper at ICLR 2021
5.75
FNS-IFFx LTC
0.34
ENS-PFFx LC
5.50
ENS EXPr LTC
0.32
ENS ExPr LTC
5.25
0.30
5.00
0.28
0.26
4.75
0.24
4.50
0.22
4.25
0.20
Ensemble Size (models)
Ensemble Size (models)
(a) ASR LTC WER
(h) ASR L.C. NI.
15
ENS- PrEx LTO LTD
0.85
ENS-PYEX LTO ENS EXPr LTO
0.80
14
0.75
0.70
13
0.85
12
0.80
0.55
11
0.50
0.45
5
6
2
3
4
5
Ensemble Size (models)
Ensemble Size (models)
(c) ASR LTO WER
(d) ASR NI.I.
ENS- AMI
6.0
ENS- AMI ENS-EXP AMI
ENS- AMI
5.5
% WER
70
5.0
60
4.5
4.0
Ensemble Size (models)
Ensembie (models)
(e) ASR AMI WER
(I) ASR AMI NLL
Figure 3: WER and NLL ablation study. Shading indicates 120
D SEQUENCE-LEVEL ERROR DETECTION
The current appendix provides a description of the Prediction Rejection Ratio metric, the rejection curves which correspond to results in section 4 and histograms of sentence- WER and BLL which provide insights into the behaviour of the corresponding rejection curves.
D.1 PREDICTION REJECTION RATIO
Here we describe the Prediction Rejection Ratio metric, proposed in (Malinin, 2019 Malinin et al., 2017), which in this work is used to assess how well measures of sequence-level uncertainty are able to identify sentences which are hard to translate/rranscribe. Consider the task of identifying misclassifications - ideally we would like to detect all of the inputs which the model has misclassified based on a measure of uncertainty. Then, the model can either choose to not provide any prediction for these inputs, or they can be passed over or 'rejected' to oracle (ie: human) to obtain the correct prediction (or monnsaationrranscrptiionn The latter process can be visualized using a rejection curve depicted in figure 4 where the predictions of the model are replaced with predictions provided by an
19

---

Published as a conference paper at ICLR 2021
Oracle
Oracle
Classification Error (%)
Uncertainty
Classification Error (%)
Uncertainly
0
6
Random
Random
2
20
40
80
100
0
20
40
60
80
100
Percentage of examples rejected to oracle
Percentage of examples rejected to oracle
(a) Shaded area ARRre-
(b) Shaded area Afbms
Figure 4: Example Prediction Rejection Curves (Malinin 2019
oracle in some particular order based on estimates of uncertainty. the estimates of uncertainty are uninformative, then, expectation, the rejection curve would be straight line from base error rate to the lower right corner, given the eπror metric is linear function of individual errors. However, if the estimates of uncertainty are 'perfect' and always bigger for misclassification than for a correct classification, then they would produce the 'oracle' rejection curve. The 'oracle' curve will go down linearly to 0% classification error at the percentage of rejected examples equal to the number of misclassifications. A rejection curve produced by estimates of uncertainty which are not perfect, but still informative, will sit between the 'random' and 'oracle' curves. The quality of the rejection curve can be assessed by considering the ratio of the area between the 'uncertainty' and 'random' curves ARu (orange in figure 4 and the area between the 'oracle' and 'random' curves ARore (blue in figure 4h. This yields the prediction rejection area ratio PRR:
ARuun ARore
PRR
(43)
A rejection area ratio of 1.0 indicates optimal rejection, ratio of 0.0 indicates 'random' rejection. A negative rejection ratio indicates that the estimates of uncertainty are 'perverse' they are higher for accurate predictions than for misclassifications. An important property of this performance metric is that it independent of classification performance, unlike AUPR, and thus it is possible to compare models with different base егrоr rates. Note, that similar approaches assessing misclassification detection were considered in (LLkshminarayanaal et al.) 2017; Malinin et al, 2017 Malinin 2019),, In this work instead of considered misclassificatios assess whether measures of uncertainty correlate well with sentence- BLEU or WER. The overall 'error' is then the average sentence-level BLEU/WER over the test-set.
D.2 REJECTION CURVES
The rejection curves for all NMT models on newstest14 and the ASR model on LibriSpeech test-clean and test-other are presented in f The main difference between the NMT and ASR curves is that the 'oracle' rejection curve for the former is not much better than random, while the rejection curve for the latter far better than random. This can be explained by considering the histograms of sentence-level BLEU and sentence-level WER presented in figure 6 Notice, that the sentence-level BLEUS are varied across the spectrum, and very few sentences reach BLEU of 100. contrast, 55-75% of all utterances transcribed by the ASR models have a sentence- WER of 0-10%, and then there are a few utterances with a much larger WER. Thus, if the measures of uncertainty can identify the largest errors, which contribute most to the mean WER over the dataset, then large decrease can be achieved. Hence the shape of the 'oracle' WeR-rejection curve. In contrast, the contributions from each sentence mean sentence-BLEE BLEU are more evenly spread. Thus. it is difficult to significantly raise the BLEU rejection curve for NMT.
BLEU by rejecting just few sentences. Hence the shape the 'oracle'
20

---

Published as a conference paper at ICLR 2021
Oracle
Dracle
90
Joint-Seq
Joint-Seq
Expected Random
90
Expected Random
80
80
70
60
50
40
20
40
60
80
100
20
40
86
100
Percentage Rejected
Percentage Rejected
(a) NMT EN2DE
(b) NMT EN2FR
Oracle
Joint
Expected Random
Expected Random
WER (%)
WER 0%)
4
2
20
60
80
100
0
20
60
100
Parcentage Rejected
Percentage Rejected
(c) ASR Test-clean
(d) ASR Test-other
80
Orade
Jeint-Seq TU
Expected Random
60
50
WER
40
20
10
Percentage Rejected
(e) ASR AMI
Figure 5: Sequence-level rejection curves for NMT and ASR.
Figure 66 shows that the Sentence-WER AMI eval is distributed more like the sentence-Bii BLUU for NMT tasks few correct sentence and much more uniform distribution of error. Thus, the corresponding 'oracle' rejection curve's shape is more similar to the NMT 'oracle' rejection curves. This clearly shows that the shape of the oracle curve not determined by the task (ASR/NMT), but the error (BLEU/WER) distribution across dataset.
The second trend the results provided in section 4 is that score-based measures of uncertainty work better than entropy-based measures NMT tasks, while ASR they perform comparably. The justification provided states that NMT models yield far less confident predictions, and therefore entropy-based measures suffer due probability mass assigned other tokens. In contrast, ASR
21

---

Published as a conference paper at ICLR 2021
0.40
1.0
ENDE
0.35
best-other
0.30
A3M1
Dengiy
0.25
Density
0.6
0.20
0.15
0.4
0.10
0.05
0.00
0.0
20
40
60
100
20
40
60
80
100
120
Sentence BLEU
Sentence WER (%)
(a) ENDE 14
(b) ASR
Figure 6: Sentence BLEU and WER Histograms.
models yield more confident predictions, shown in figure 7 Notably, on AMI and Common Voice datasets the ASR model also yields less confident predictions, and thus the score-based measures of uncertainty do better than entropy-based ones the AMI rejection curve in figure 5 These results show that on tasks where it is important to determine which particular translation/trascrriiioo hypotheses are worse, score-based measures of uncertainty do as well as or better than entropy- based measures. This result is consistent with confidences being better measure of uncertainty for misclassification detection unstructured prediction tasks (Malinin 2019).
1.0
1.0
NMT
LibnSpeech
Common Voice
Densty
0.6
0.6
0.4
0.0
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Confidence
Confidence
(a) NMT
(b) ASR
Figure Histograms of predicted-token confidence for ASR and NMT.
D.3 ADDITIONAL RESULTS
The current section provides additional sequence-level error detection results. We examine sequence- level error detection of hypotheses produced by ensemble combined expectation-o) products. Results presented in table serve to confirm the previously observed trends and illustrate that the superior performance of measures of uncertainty derived from an product-offexpeecaatios posterior does not depend on the nature of the hypotheses.
E TOKEN-LEVEL ERROR DETECTION
Current appendix provides additional results for token-level error detection. Notably, we present results using score-based measures of knowledge uncertainty M well results on hypotheses derived from an expectation of- of-puuds ensemble combination. Results table [T that the new measures of uncertainty consistently outperforms token-level mutual-in innormmaiioo and RMI.
Results in table [2 show that the observed trends do not depend from which ensemble-combination the hypotheses were obtained.
22

---

This confirms Published as a conference paper at ICLR 2021
Table 10: Sequence-lever Error Detection PRR in Beam-Search decoding using PEP (2) az, D).
ENS-PrEx TU
ENS-ExPr TU
ENS-PrEx KU
ENS-ExPr KU
Task
Text
Rew
Ryw
H
f(1)
R() Kepaw
M
M
Z(1) FC-ww
M
M
LTC
64.7
68.5
63.5
67.3
62.8
61.1
60.5
64.3
52.8
60.6
60.3
64.2
LSP
LTO
73.5
76.0
68.8
72.0
72.3
69.5
68.6
70.5
36.0
68.3
68,3
70.9
57.9
66.8
53.5
63.1
16.7
62.8
ENDE
nwr14
29.9
28.6
29.1
28.1
30.4
5.6
22.4
28.1
30.6
MED
31.1
45.5
28.9
41.7
37.1
36.4
35.8
38.9
-2.1
26.8
36.7
38.6
nwtl4
26.3
38.7
25.9
38.4
31.3
30.8
30.3
32.3
16.9
27.6
30.1
30.2
ENER
MED
17.3
40.1
16.4
39.2
35.3
35.1
34.9
39.9
14.1
29.2
35.4
37.1
Table 11: %AUPR for LibriSpeech in Beam-Search Decoding regime using PPE PpE(3) x. D).
Test
ENSM-PrEx TU
ENSM-ExPr TU
ENS-PrEx KU
ENS-ExPr KU
% TER
Data
71
P
Mw;
I
K
M
MM,,
LTC
34.7
36.3
32.4
33.4
32.7
28.0
27.6
33.4
26.5
25.9
27.4
30.8
3.8
LTO
42.4
39.1
40.9
36.1
41.5
30.8
33.6
35.3
37.4
10.2
AMI
71.7
74.6
68.3
70.4
71.8
68.7
67.9
72.3
59.2
64.4
66.6
67.3
41.2
F OUT-OF-DISTRIBUTION INPUT DETECTION
In the current section additional OOD input detection results are provided for En-De and En-Fr NMT models and the ASR model in Beam-Search decoding regime. Additionally, we provide results for En-De and En-Fr models on reference hypotheses teacher-forcing regime.
F.1 ADDITIONAL RESULTS
Table 13 provides full of OOD detection results using ensemble of EN-FR translation models. All hypotheses are derived from a product-of-exppee expeecctainsstn ensemble. These results tell essentially the same story OOD detection roughly ten times larger than WMT' 17 En-Fr, OOD detection in some cases, notably LTC, PRM and L-FR easier. However, L-DE are significantly worse. Note that for En-FR, L-DE is the heldout language, while L-FR is more familiar. One explanation is that the copy-through effect is so strong on an unfamiliar language that even measures knowledge uncertainty are drastically affected. This suggests that it is necessary to eliminate this regime, it strongly compromises the measures of uncertainty.
En-De models. However, seem that because WMT' 14 En-Fr is
Table 14 provides full set of OOD detection results on hypotheses generated from ensemble combined an expectation-of oot-oofo-rruss.....m The results essentially tell the same story those obtained on hypotheses generated from an ensemble combined as croduct-of-expectaaiin the trend that measures of uncertainty derived by expressing the predictive posterior as a product- expeecccttsonss typically yield marginally better performance, with expceptions where they yield significantly better performance, regardless of the nature of the hypotheses.
F.2 TEACHER-FORCING
Table provides of OOD detection results for EN-DE/EN-FI translation models evaluated in Seacher-forcing regime, where 'references' are fed into the decoder. The aim is to further explore
Table 12: %AUPR for LibriSpeech in Beam-Search Decoding regime using PEP ( a. D).
Test Data
ENSM-PrEx TU
ENSM-ExPr TU
ENS-PrEx KU K.
ENS-ExPt KU
% TER
H
p
H
P
M
Mw;
I
K.
M
M.,
LTC
38.0
43.2
34.1
37.2
37.0
32.9
32.0
40.5
27.7
32.1
33.9
39.6
4.1
LTO
50.8
56.4
44.5
46.5
50.3
46.2
45.2
54.1
32.9
45.1
48.1
53.5
12.0
AMI
76.5
72.1
73.9
76.9
74.0
73.2
77.9
59.5
73.4
75.5
77.6
23

---

Published as a conference paper at ICLR 2021
Table 13: OOD Detection % ROC-AUC Beam-Search decoding regime for ASR and NMT. 10
Task
00D
Data
ENS-PIFE PiFx TU Row
R
ENS-ExPE TU:
2(1))
ENS-
KU PrEx
(19
M
2(8) -LWW
ENS R
ExPr KU
(1)
64.1
77.3
63.3
77.2
78.5
78.4
78.3
81.7
65.2
75.3
78.0
78.9
LTC
65.4
79.2
64.7
79.2
79.1
78.9
78.8
83.6
66.7
76.8
78.6
82.0
92.7
91.6
90.9
91.6
98.7
98.6
98.6
98.5
61.3
94.0
98.6
97.7
ENFR
PRM
93.3
92.1
91.7
98.7
98.7
62.6
33.4
L-FR
57.2
42.1
53.4
46.0
88.1
89.6
90.4
94.8
60.1
85,9
90.3
94.4
L.DE
12.4
6.9
11.2
7.6
35.1
38.2
40.4
39.2
19.7
27.6
40.1
44.4
13.1
14.5
11.8
15.6
38.9
42.7
45.4
67.8
19.4
32.0
46.6
67.6
Table 14: OOD Detection % ROC-AUC in Beam-Search decoding regime for ASR and NMT.
OOD
ENS- PrEx
ENS ExPr TU R
ENS PrEx
KU
ENS ExPr KL
Task
Data
T
B
Rgm
((B) *CoW
R(B)
M
(2 5.w
(B)
R(B)
M
(B)
SJW
76.5
75.4
75.6
74.6
76.2
76.5
76.4
73.9
68.5
76.0
76.1
74.1
LTO
77.0
77.0
76.3
75.5
76.8
77.1
77.1
77.0
71.8
76.9
77.0
77.2
97.5
97.4
97.3
97.1
96.3
96.2
96.1
96.2
79.7
96.1
96.0
96.2
AMI
96.6
97.9
96.7
97.3
95.1
95.1
95.0
97.7
86.7
95.2
95.2
97.7
C-FR
99.7
99.9
99.9
99.9
20
100.0
99.7
99.9
98.9
99.9
99.9
99.9
99.9
38.9
99.9
99.9
99.9
65,3
72.0
63.7
70.1
72.3
72.0
71.7
72.8
49.5
66.4
71.0
71.2
LTC
65.4
74.8
63.8
72.3
72.7
72.4
72.1
75.0
50.6
67.4
71.4
73.9
83.0
85.2
78.6
79.4
96.4
96.7
96.7
95.9
45.3
92.3
96.3
94.3
ENDE
PRM
79.3
79.5
96.7
96.9
97.0
93.5
96.6
L-PR
10
27.1
20.4
21.8
16.5
63.3
68.9
72.2
69.9
19.8
43.7
69.4
69.4
28.0
23.9
22.7
20.2
65.9
71.7
75.2
78.6
18.9
47.5
73.4
76.8
34.6
24.4
78.5
80.5
67.8
78.3
L-DE
42.6
39,8
35.7
31.2
76.9
80.6
82.6
88.6
33.3
72.2
81.2
85.5
63.1
78.0
61.6
76.5
78.1
78.0
77.9
81.6
$6.2
73.0
77.5
79.2
LIC
64.4
80.1
63.1
78.3
78.7
78.6
78.5
83.8
58.7
75.0
78.2
82.2
93.2
92.7
90.7
90.6
98.7
98.7
98.6
98.5
38.2
88.8
98.6
98.1
ENFR
PRM
29.5
I.FR
58.1
48.0
52.2
38.8
88.3
89.7
90.5
94.8
46.7
84,8
90.2
93.4
I-DE
12.5
7.6
11.0
6.0
35.0
38.1
40.3
38.9
20.6
27.6
39.8
41.3
13.2
15.4
11.6
13.6
39.2
43.1
45.8
67.6
18.6
30.8
46.8
66.6
Table 15: OOD Detection ROC-AUC in Teacher-Forcin regime
Task
0OD
Data
ENS Paxx TU R(
R1)
(1)
ExP.
2013 -c-w
A(1
C-C-WW
ENS PrEx KU
2(13
(-11)
ENS EaPr KU
Ciw
L-DEEN
98.3
98.9
98.0
99.0
99.4
99.5
99.5
99.4
55.2
97.6
99.5
96,7
ENDE
L-DEDE
22.4
11.3
11.2
58.7
64.0
67.1
41.0
41.3
54.7
61.4
48.7
LNENEN
62.2
58.3
56.0
56.7
79.6
81.6
82.7
76.6
31.2
59.6
79.9
82,9
L-FREN
100.0
99.8
100.0
99.8
98.1
97.2
96.2
94.1
71.6
94.6
96.1
85.6
L-FREN
99.6
98.4
99.5
98.6
99.9
99.9
99.8
99.6
67.6
98.7
99.9
97.2
L-FRER
28.8
12.3
25.7
12.7
75.1
78.2
80.0
$8.9
55.5
72.2
76.9
61.5
ENER
L-ENEN
39.4
40.5
37.7
40.7
84.3
86.2
87.4
54.2
68.1
82.2
84.8
56.2
D.DN
100.0
99.9
100.0
99.9
97.6
97.0
96.4
95.3
76.2
95.2
96.7
88.0
24

---

Published as a conference paper at ICLR 2021
OOD detection of foreign languages and the copy effect. Here, in-domain data is En-De and En-Fr newstestl4 for En-De and En-Fr models, respectively. As OOD data we also consider newstest14 data, but where the either source, target both languages are changed. The results show that when the source and target languages are both changed (L-DEEN, L-FREEN,, then this is an easy detect scenario, copy-though is forcibly avoided. We also consider situations where we forcibly initiate copy-through. Here, we have matched pairs of source-source or target-target language. Measures of total uncertainty fail, while measures of knowledge uncertainty do not. However, the effect is more severe when source sentences are copied through, rather than target sentences. We speculate that this an affect of the decoder being familiar with target sentences and still trying to do something sensible. These results clearly show that if the copy-through effect somehow eliminated, then detection OOD sentences by NMT models because as easy as for ASR models, where copy-though cannot occur by construction. Finally, we note that again, deriving uncertainties from a product-of-exyy efpexcctati predictive posterior yields marginally better results. Additionally, chain-rule RMI seems to be the best measures of knowledge uncertainty overall. Note that in forcing we have access to only single hypothesis per input, which is regime where joint-sequence estimates of knowledge uncertainty, specifically RMI, tend perform worse than chain-rule derived estimates.
G SENSITIVITY OF MC ESTIMATORS TO NUMBER OF SAMPLES AND CALIBRATION
In this section explore in detail the effect of using more than the 1-best hypothesis in the Monte- Carlo estimation of entropy and reverse mutual information (RMI). We consider both chain-rulc and joint-sequence estimators. Performance evaluated the tasks of sequence error detection and OOD detection. Figure 8 shows that for sequence error detection using more hypotheses within the beam either has little effect, or is detrimental. This expected, as this makes the estimate less sensitive to the particular hypothesis which is being assessed. Notable, joint-seeeensseeee estimates are affected the most, while chain-rule estimators demonstrate more stable behaviour.
45.0
36
42.5
34
40.0
NWT N:
NWT Alc
NWT 7s
NWT Aks
MED it
MED Mc
35.0
MED fts
MED Ms
28
30.0
26
27.5
3 Number beam hyptheses considered
2
1
Number bear hyptheses considered
100
95
LTO de
LTO Mr
ROC-AUC AUC
LTO is
ROC-AUC
LTO Mis
AMI itc
AMI Atc
AMI fts
AMI Ats
%
85
C-FR N.
C-FR Mr
C.FR Rs
C.RR At.
75
4
8 1011 1415 17
2
4 10 17111415111711122
Number of beam hyptheses considered
Number of beam hyptheses considered
Figure 8: Sensitivity of uncertainty measures to number samples on NMT and ASR sequence error detection.
25

---

Published as a conference paper at ICLR 2021
Figure 8 shows that considering more hypotheses is generally beneficial for OOD detection, especially for joint-sequence estimates RMI. This is also expected, as for OOD detection it useful to have information about the effect of the input on more than just the 1-best hypothesis. The performance gain is, ultimately, unsatisfying.
80
PRM Ric
95
fis
70
LTC
90
PRM Ass
PRM Mr
ROC-AUC
60
ROC-AUC
LTC Mc
Ne
85
LTC. Mrs
H,
L-FR. Arc
80
L-FR Ats
L-DE Mrc
L-DE Art
20
70
Number of beam hyptheses considered
Number of beam hyptheses considered
95
LTO Re
LTO. Mrc
% ROC-AUC
LTO 84
% ROC-AUG
LTO. Ses
AMI Ne
AMI Aic
AMI Rt
Mis
85
C-FR Ne
C-FR Mc
C-FR Ns
C-FR Mrs
80
2 4 1011 1213 1415 18111
2 4 1011 15 16 1711999200
Number of beam hyptheses considered
Number of beam hyptheses considered
Figure 9: Sensitivity of uncertainty measures to number samples on NMT and ASR OOD detection.
We consider what happens if we increase the importance weighting temperature T, which increases the contribution for the remaining hypotheses the uncertainty estimate. Figure [O]shows, unsurprisingly that this is extremely detrimental to sequence еттоr detection, as it introduce even more irrelevant information. However, figure shows that for OOD detection, the is unexpectedly interesting behaviour. Using higher temperature for NMT always leads to significantly performance, especially for joint-sequence estimates of RMI. However, what is especially surprising, that for ASR the OOD detection performance degrades. It is not entirely clear why up-weighting information from competing hypotheses is detrimental to performance. Unfortunately, investigating the underlying cause of this effect is beyond the scope of this work. Ultimately, it suggests that analysis of the calibration of autoregressive structured prediction models is an interesting area future research.
26

---

Published as a conference paper at ICLR 2021
44
36
42
40
Mc
30
Ne Mss
Me
34
28
32
30
20 Temperature
23
25 Temperature
70
60
60
50
LTC.
40
50
LTC.
Nc
% PRR
LTO
30
LTO M
20
Rc
Mc
10
AMI No
AMI
30
-10
10
Temperature
Temperature
Figure 10: Sensitivity of uncertainty measures importance weighting calibration.
PRM Nic
PRM Mr.
PRM Ks
90
PRM Mi
ROC-AUC
60
LTC
K-ROC-AUC
Mc
50
L-FR Ric
LTC
85
L-FR, Mlc
Ae
40
L-FR. Als
L-FR, Mrs
L-DE fec
L-DE Mc
30
L-DE As
L-DE Ms
75
10
15
20
25
30
35
40
10
15
20
25
30
35
40
50
Temperature
Temperature
100
100
LTO
LTO
ROC-AUC
80
LTO
ROC-AUC
LTO ABs
AMI Mt:
AMI Aec
AMI fts
AMI Ats
CJFR N.
C-FR Mr
60
C.FR As
C.FR Ar
50
65
10
15
20
25
30
35
40
45
50
10
15
20
25
30
35
40
45
50
Temperature
Temperature
Figure 11: Sensitivity of uncertainty measures to importance weighting calibration.
27

---

Published as a conference paper at ICLR 2021
H EFFECTS OF LENGTH-NORMALIZATON
In section 3 we make the claim that length- noathhh-omaliziionn in important in this section we provide evidence in support that claim. Here we compare sequence error detection and OOD detection performance for WMT" 17 EN-DE NMT and LibriSpeech ASR systems when using the standard length-normalized versions of the uncertainties as well as their non-length-normalizee counterparts. In these experiments only consider translation/raascriptioo hypotheses obtained via beam-search decoding from product-of-expecattiie ensemble, and uncertainty measures obtained using the same combination approach.
Table 16: Effect of length-normalizatien on sequence-level Error Detection % PRR.
ENS-PrE: TU
ENS- PrEx KU
Task
Test set
len-norm
HCw
H
7(1)
C-WW
(-(1)
C-W-W
(B)
M (1)
S-IW
48.8
56.3
47.6
46.7
46.4
54.3
LTC
64.7
68.5
62.8
61.1
60.5
64.3
LSP
49.6
54.4
48.2
46.6
46.0
53.0
LTO
73.5
76.0
72.3
69.5
68.6
70.5
6,3
26.6
-1.2
-1.2
-1.2
25.4
AMI
57.9
66.8
53.5
51.2
50.1
63.1
20.2
30.7
24.2
23.9
23.6
27.8
nwt14
29.9
46.6
29.1
28.1
27.4
30.4
ENDE
21.0
33.6
28.6
28.5
28.3
32.7
MED
31.1
45.5
37.1
36.4
35.8
38.9
The results in tables 17 definitively show, with one exception, that noaggt-normallzztioon consis- tently boosts the performance all tasks. The only exception in for OOD detection when French (L-FR) input sentences are given to the ensemble. In this case there a large improvement from not using length normalization. This seems odd, given than when German input sentences using (in and En-De system) the same effect does not appear. Not that both these cases the pathological copy-through effect appears. However, it seems likely that the copy-through effect is far more pronounced for L-FR. The likely reason for length-norm proving detrimental for long sentences as the length of the translation increases, does the chance of token which is not copied successfully, on which the models will yield far higher estimate of ensemble diversity. Length-normaaizztion would mask such an effect relative.
Table 17: Effect of congth-normalizatioon on OOD Detection % ROC-AUC for ASR and NMT.
OOD
ENS-PrEx TU A(A)
ENS-PrEx KU
Task
Data
T
B
Len-Norm
(B)
(B) C-IW
((B) Ca
M
M
73.2
73.5
73.4
73.9
74.0
74.3
LTO
1
20
76.9
76.3
76.6
77.0
77.0
76.1
ASR
85.2
89.8
81.4
81.2
81.1
86.9
AMI
1
20
96.5
97.9
94.9
94.9
94.8
97.4
99.4
99.4 99.4
99.2
99.2
99.2
99.5
C-FR
1
20
99.9
99.7
99.9
99.9
99.9
99.8
52.1
55.5
60.4
60.8
61.2
66.4
LTC
10
5
66.0
74.1
73.2
72.9
72.6
75.0
65.6
68.7
86.8
88.3
89.1
92.1
PRM
10
5
NMT
82.9
82.7
96.7
96.9
96.9
96.5
63.7
50.9
78.4
81.3
83.0
87.7
L-FR
10
5
27.1
21.7
65.2
71.2
74.8
79.6
44.9
37.1
70.1
73.8
76.1
85.8
L-DE
10
5
40.8
34.9
76.1
79.9
82.1
89.2
28

---

Published as a conference paper at ICLR 2021
1 COMPARISON TO HEURISTIC MEASURES OF UNCERTAINTY
In this work we considered a range of information-theoretii measures of uncertainty, describe them theoretically in section _ and provide Monte- Carlo estimators in section [3] However, (Xiao) et al. 2019; Fomicheva et al., 2020; Wang et al. 2019) a range of other measures was considered, though their properties were not analyzed. Firstly, (Xiao et al. 2019) considered computing the cross-BLEU or ABLEU-Variance of ensemble. Here, the average square of the complement of
pairwise sentence-BLEL between 1-best hypotheses ((m) produced by each individual model in the ensemble used as a measure of uncertainty:
X-BLEU- M(M 1)
Σ Σ
1100-BLES (LEUGB)) )(()))²
(44)
Similarly, an equivalent measure of ASR can be derived, which we will call cross- WER:
M
M Σ
X-WER- M(M - 1)
Σ
(WER(
( (m)) g(o)))
(45)
m==1 gyfm
These two measure of uncertainty assess the diversity between the 1-best hypotheses of each model in an ensemble. In this way they are conceptually related to measures of knowledge uncertainty. Notably, as they are only sensitive to the 1-best hypothesis, they are 'hard' versions of ensemble diversity, as they operate on the surface forms, rather than over probabilities. While a heuristically sensible measure, the main detraction that this requires doing heam-search inference of each model in an ensemble, which is expensive and undesirable. Especially if the final prediction is derived through inference of the
Additionally, Fomicheva et al. (2020); Wang et al. (0019) consider the variance of the sentence- level probability and length-normalized log-probability of the 1-best hypothesis across models in an ensemble:
V(P) = Va(0) P(y) best x., 0)]
(46)
V ||n P| = (o(
} In P(y 0)]
1
(47)
These are also measures of diversity, and therefore knowledge uncertainty. However, they are not strictly information-theoretiee meaningful and it is not how to relate them to concepts, such as entropy. Given these two measures, it is unclear how to include information from other hypotheses and whether nonmalitomaliiiionnn is necessary. First address the latter consider a length- normalized version of variance, and the non-length-normen normalized variance of the probbabbbbil.t.
//P] = Vq(0) (( xc, 0)
(48)
In P] = Va(0) In P best x 0)]
(49)
(50)
Secondly, we extend all four of these measures to average variances of the hypotheses within the beam follows:
=
Σ V V 2()) (Py (b) x 02,0)], (b) B
(51)
V (0) PP 0) ক 0)] ( e B
(52)
d=1
((8) P]
my Th (()) PIn (h) ac, 0)] (M) B
(53)
b=1
B Σ
(1 P| =
V (
(n)
x 0)),0) y (6) E B
(54)
(b)
29

---

Published as a conference paper at ICLR 2021
Thus, explore the effect of considering additional hypotheses and whether these measures need length-nowe noigth--ormalization or not.
Table 18 explores the utility of these uncertainty measures and compares them to the best-performing measures of knowledge uncertainty uncertainty reverse mutual information on the task of OOD input detection. The results show that, with exception, estimates of reverse mutual information outper- form these 'heuristic' measures. With regards to the measures it is clear that length-normaliizttio, with one exception, improves performance. We can also see that these heuristic measures can be improved by considering importance-weighte However, these gains are inconsistent, and sometimes this is detrimental. This highlights both the value of information theoretic measures, whose behaviour is far more consistent.
averages across the hypotheses within the beam.
Table 18: Comparison of information thhooreti and heuristic measures on OOD detection (% ROC- AUC).
Task
OOD
Data
C
(R)
Info, Theor.
M
S.JW
(12)
(PP)
9(/) (P)
yC02 P)
Heuristic
(9 [lo P
X-BLEU
X-WER
LTO
20
1
77.0
76.5
73.9
76.1
51.2
52.6
72.0
72.7
73.8
71.0
74.6
72.7
74.3
71.8
AMI
20
1
94.8
96.2
97.4
96.4
42.3
41.5
87.3
85.6
84.0
86.1
96.7
95.8
95.9
95.8
C-RR
99.9
99.9
99.8
12.5
7.9
77.6
82.0
97.6
98.4
98.0
99.5
99.7
22.2
73.3
46.0
68.6
68.7
65.6
72.3
65.1
58.2
NMT
96.2
33.7
32.7
88.7
88.8
90.4
93.0
93.4
73.5
T-FR
70.9
59.2
764
85.1
52.7
58.0
77.7
76.0
82.7
38.7
82.1
89.2
69.8
90.0
86.2
J CHECKPOINT ENSEMBLES
This work focused mainly on ensemble of models constructed by training from different random initializations. While this tends to yield the best ensembles, this is an extremely expensive process, especially for large transformer models trained on industrial scale corpora. Thus, in this section we conduct preliminary investigation of checkpoint ensembles ensembles of models constructed by considering different checkpoints within single training run. In this work we consider the checkpoints for the 10 last epochs for NMT model training. For ASR, we checkpoint average checkpoints at 5-checkpoint intervals within the last 30 epochs, yielding an ensemble of 6 checkpoints, where each an average of five checkpoints. Results tables 20 shows that checkpoint ensemble yield better predictive performance than single models (CPT-AVG), on average. However, random-init ensembles (those considered in the main work), yield consistently superior performance.
Table 19: Predictive performance in terms of BLEU, &WER and NLL on newstest14 and LibriSpeech.
NWT' 14 BLEU
MED BLEU EN-DE
NWT NLL
MED NLL
Model
EN-DE
EN-FR
EN-FR
EN-DE
EN-FR
EN-DE
EN-FR
CPT-AVG CPT-ENS
28.8 102
45.4 0:
29.9 10.
51.3 10.
1,46 1000
1.10 : 0.01
1.29 : 000
0.82 0.01
29.3 10.1
30.5 03
1.42 1001
1.25
RND-ENS
30.1
46.5
32.1
52.7
1.33
1.04
1.16
0.77
We also compare checkpoint ensembles with random-init ensembles of OOD detection. The results, which are pleasant surprise, show that while checkpoint ensembles are consistently inferior, they are only marginally so. The biggest differences occur on NMT L-FR and L-DE datasets, where the pathological 'copy-through' effect kicks in. However, even here the difference between the best CPT-ENS and RND-ENS measures is only about 6% ROC-AUC points. This shows that the approach considered in this work need to be too expensive for practical application, and useful ensembles can
30

---

Published as a conference paper at ICLR 2021
Table 20: Predictive performance in terms of BLEU, &WER and NI.. newstest14 and LibriSpeech.
ASR % WER
ASR NLL
Model
LTC
LTO
AMI
LTC
LTO
AMI
CPT-AVG
5.6 1:22
14.7 105
78.7 133
0.34 1:000
0.86 =0002
5.78
1:032
CPT-ENS
5.3
13.9 NA
54.0 +
0.27 NA
0.67 + NA
4.68
RND-ENS
4.2
11.3 50.4
0.20
0.48
4.05
be formed even from the last 10 checkpoints of a standard transformer model training run, enabling usage in extreme-scale industrial applications.
Table 21: OOD Detection % ROC-AUC in Seam--Sarch decoding regime for ASR and NMT.
PrEx R(BB
ExPt TU
ENS. PrÊx
ExPr KU
Task
Test Set
type
A(B)
H(
Zeww
M
(B)
7(2)
((8)
M
Mew
LTO
CPT.ENS RND-ENS
75.1
74.7
75.0
74.9
74.6
74.6
74.5
74.0
74.1
74.6
74.6
73.7
76.9
76.3
7644
77.0
76.6
77.0
77.0
76.1
74.8
76.7
76.9
75.3
CPI-ENS RND-ENS
ASR
AMI
96.5
97.9
96.4
97.9
94.9
94.9
94,8
97.4
93.0
94,8
94,8
97.0
CSRR
CPT-ENS
99.9
99.5
99.9
99.6
99.8
99.7
99.7
99.2
97.9
99.7
99.7
98.5
RND ENS
99.9
99.7
99.9
99.8
99.9
99.9
99.9
99.8
89.7
99.9
99.9
99.1
CPT-ENS
63.7
72.5
63.5
72.4
70.9
70.7
70.6
75.0
59.9
67.0
70.5
73.0
LTC
CPT-ENS
81.1
81.3
80.8
81.5
96.4
96.4
96.3
94.7
79.0
93.4
96.2
92.9
PRM
RND ENS
82.9
82.7
80.7
84.5
96.7
96.9
96.9
96.5
72.1
95.0
96.7
95.2
ENDE
CPT-ENS
29.5
27.8
28.9
27.7
60.4
62.9
64.6
73.2
33,3
47.2
65.0
74.2
RND-ENS ENS
27.1
21.7
23.0
22.9
65.2
71.2
74.8
79.6
23.4
50.1
73.8
80.7
CPT-ENS RND-ENS
39.4
35.9
38.5
35.7
72.3
73.3
74.0
82.3
45.9
61.5
73.9
82.0
L-DE
34.9
36.1
38.3
76.1
82.1
73.3
88.8
31


Reviews:
[{"review_id": "gGsr44vWWGH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "After rebuttal: I would like to thank the authors for paying attention to the comments and providing additional experiments and results. \nI updated my score. \n\n==================================\nThis paper investigates uncertainty estimation for autoregressive structured prediction models. The authors explored several metrics for computing uncertainty in structured neural network based models, such as expected pair-wise KL-divergence, mutual information, and reverse mutual information (RMI). The authors demonstrated the efficiency of the proposed RMI using machine translation and speech recognition models for error detection and out-of-distribution detection.\n\nOverall this paper is clearly written and investigates highly relevant topic. However, some clarifications are needed. First, I find the name of the paper a bit misleading, there are plenty of structured prediction models, and the authors investigated only neural based and autoregressive ones. \n\nSecond, I know the authors mentioned that they did not compare to prior work since it did not consider auto-regressive models, however, I feel that a comparison is still needed. There are a few papers (some of which the authors already mentioned), that suggest different ways to estimate uncertainties in AST and MT. I would expect to see a comparison to some of these methods. \n\nComments to the authors: \n\n1) In \"Practical Considerations\" there is a question mark, probably a missing citation?\n\n2) The improvements in Table 1 seems pretty marginal. Can the authors report the variance of the estimator too? \n\n3) \"The results also show that uncertainty-based rejection works better for ASR for NMT.\" → \"The results also show that uncertainty-based rejection works better for ASR than NMT.\"\n\n4) \"A better, but more expensive, approach to assess uncertainty estimates in NMT is whether they correlate well with human assessment of translation quality.\" → did the authors try to use subjective evaluations? \n\n5) \"Three OOD datasets are considered, each covering a different form of domain shift. First, LibriSpeech test-other (LTO)...\" → I find this definition of OOD a bit problematic. LTO is still part of librispeech, so I would not consider that as OOD.\n\n6) \"Notably, it was found that token-level Bayesian model averaging consistently yields both marginally better predictive performance and more robust estimates of uncertainty.\" → One reason for that is maybe since the authors are normalizing their probabilities at the token level and not at the sequence level? did the authors investigate such models that are normalizing at the seq level such as: [1].\n\nI'm willing to increase my score if the above questions will be answered \n\n[1] Collobert, Ronan, Christian Puhrsch, and Gabriel Synnaeve. \"Wav2letter: an end-to-end convnet-based speech recognition system.\" arXiv preprint arXiv:1609.03193 (2016).", "extra_scores": {}, "extra_text": {}}, {"review_id": "cOVkKIZL-SB", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "* **Summary**:\nThis work introduce rigorous information-theoretic measures for structured prediction tasks. It proposed metrics for both `\"total uncertainty\" (entropy) and \"knowledge uncertainty\" (MI, EPKL and RMI) on the sequence level, introduced efficient Monte-Carlo approaches to estimate them in practice. Finally, author conducted thorough experiment to evaluate the effect of ensemble strategy and choice of metric for error detection and OOD detection in ASR and NMT.\n\n* **Strength and Weakness**:\n * (Strength) Uncertainty quantification in structured prediction is an important but less explored topic. This work provided a much needed, information-theoretic framework to both conceptually quantify and empirically compute uncertainty measures for different sources of uncertainty.\n * (Strength) Thorough experiment on two tasks (ASR and NMT), where authors designed experiments to quantitatively measure uncertainty quality (Sequence-level Error Detection and OOD Detection) under different metrics and ensemble strategies.\n * (Weakness) The experiment conducted are mostly empirical where the ground truth is not necessarily known. Given the theoretical nature of this work, it might be good to conduct simulation study under known truth to examine the estimation quality of different metrics.\n\n* **Recommendation**: Acceptance. I believe this work provided a nice theoretical framing of different uncertainty measures for an important and less-explored area (structured prediction). It also conducted thorough empirical investigation showing the relative merit of different modeling and measure choices on two standard structure prediction task. The information contained in this paper should be of sufficient interest to ICLR community.\n\n* **Minor Comments**:\n   * Missing reference on bottom of page 4: \"rarely used in practice as it yields poor predictive performance ?.\"\n   * Middle of page 6: \"Rejection curves are summarised using the Prediction Rejection Ratio (PRR)\". Make sure to mention PRR is introduced in detail in Appendix D.\n    * Top of page 3 \"As will be shown later, RMI is particularly attractive ..\". It might be helpful to point to the section where the \"attractiveness\" of RMI is illustrated (Equation 12-13?).", "extra_scores": {}, "extra_text": {}}, {"review_id": "5XyXpQK2GyX", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes two different measures of knowledge (epistemic) uncertainty in structured prediction with an autoregressive model and discusses how to compute their approximations. The main contribution is the proposed reverse mutual information (RMI) as a measure of epistemic uncertainty in structured prediction. Experiments on benchmark datasets demonstrate the effectiveness of the proposed method in error detection and OOD detection.\n\n\n### Clarity\n#### Pros\n- This paper clearly states the main contributions, assumptions, experimental settings.\n\n#### Cons\n- The naming of reverse mutual information is confusing since mutual information itself is symmetric, unlike KL-divergence.\n- Table 1 shows the predictive performance with an ensemble of autoregressive models, however, it is unrelated to the main contribution of the paper. The proposed RMI is not used to improve the performance of the ensemble of autoregressive models.\n- Eq(4), $\\tilde{\\theta}$ is used without definition. And there is no $\\mathcal{D}$ on the right side of the equation. Why is $q(\\theta) = q(\\tilde{\\theta})$?\n- In Eq(12), how is $y$ sampled from $P(y\\mid x, \\mathcal{D})$? Do you mean to sample from $q(\\theta)$ instead of $p(\\theta\\mid \\mathcal{D})$?\n- Though the two ways, EP and PE, for combination of the ensemble of autoregressive models are both reasonable, only EP can be derived from assumptions in Eq (7). PE assumes the probability $P_{PE}(\\boldsymbol{y} \\mid \\boldsymbol{x}, \\mathcal{D})$ factorizes over the conditional probability ${P}(y_{l} \\mid \\boldsymbol{y}_{<l}, \\boldsymbol{x}, \\mathcal{D})$.\nThis should be made clear, otherwise can be confusing.\n\n- On page 4, above Eq (10), it is claimed that \n> (8) only considers the probabilities of individual tokens $y_{l}^{(s)}$ along a hypothesis $y^{(s)}$ while (9) considers the entire conditional distribution over each $y_{l}$.\n\n  It is not clear why this is the case. In my understanding, (8) computes the entropy of the joint distribution $P(\\boldsymbol{y} \\mid \\boldsymbol{x}, \\mathcal{D})$, \nwhich considers all the conditional distribution $\\mathrm{P}\\left(y_{l} \\mid \\boldsymbol{y}_{<l}, \\boldsymbol{x}\\right)$, no matter using EP or PE. Correct me if I am wrong.\n  \n- (Minor)\nTypos:\n  - Page 4, \nabove Eq (15),\npractical considerations\nas it yields poor predictive performance ? -> remove ?\n  - Page 4,\n  above Eq (10), while (8) yields A -> while (8) yields a\n  - Page 4,\n  below Eq (13), like (8)(9) -> like (8)(10)\n  - Page 5,\nabove Eq (16),\nthe models can combined -> the models can be combined\n\n\n### Originality\n#### Pros\n- The paper makes clear its main contributions, including introductions of different measures of uncertainty in structured prediction, the examination of two choices of the combination of ensemble models. The proposed reverse mutual information (RMI) improves the performance in OOD detection in structured prediction.\n\n#### Cons\n- The contribution of the paper is minor with the proposed RMI measure.\n- The proposed RMI for epistemic uncertainty estimation is not used to improve the performance of the ensemble of autoregressive models.\n\n\n### Significance\n#### Pros\n- Experimental results show improvement of the proposed method over baseline methods.\n\n#### Cons\n- The paper presents ablation studies of different methods for knowledge uncertainty estimation, but it is unclear whether the proposed method achieves state-of-the-art performance on different datasets for error detection and OOD detection. For example, compare to MC-dropout and other recently proposed methods without structured prediction with autoregressive models.\n- The contribution is not very significant, but mostly an extension and application of uncertainty measures to autoregressive models.\nFor example, the reverse mutual information (RMI), seems to be related to the Reverse KL-divergence proposed in [1]. More discussion is suggested on how the reverse version improves over the original version as is done in [1].\n- Besides OOD detection, it is also interesting to show whether the proposed uncertainty measure can improve other tasks, such as adversarial sample detection, and active learning. For example, [2] presents more thorough experiments in these different tasks.\n- The paper proposes two ways to combine the ensemble of autoregressive models, EP or PE. Experimental results are provided on these two ways of combination.\nHowever, it would be better if more theoretical analysis to compare these two could be given based on different assumptions. For example, analysis of the computational complexity of the two choices.\n- Similarly, it would be better if complexity analysis can be provided for computing approximations of the entropy using relative-entropy chain-rule in Eq (9),(11),(13), and using the traditional ways in Eq (8),(10),(12). Since it is claimed at \"no extra cost\" in the paper.\n- It is discussed in the paper that EPKL, RMI, MI all measures knowledge uncertainty, and RMI = EPKL - MI. And only MI 'cleanly' decomposes into total and data uncertainty. \nA natural question is that if we decompose total uncertainty using RMI, i.e. Total uncertainty = data uncertainty + RMI + res, what does the remaining term res represent?\nFor example, if RMI gives a better measure of knowledge uncertainty as shown empirically in the paper, then the remaining term in the total uncertainty may measure other types of uncertainty rather than data uncertainty or knowledge uncertainty? Or there are other ways to represent total uncertainty and/or data uncertainty correspondingly?\nCorrect me if I am wrong.\nAnd it would be better if more discussion can be provided in the paper.\n\n[1] Reverse kl-divergence training of prior networks: Improved uncertainty and adversarial robustness\n\n[2] SDE-Net: Equipping Deep Neural Networks with Uncertainty Estimates", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes information-theoretic quantification of epistemic uncertainty in autoregressive models. 

This is a difficult problem that receives much less attention than the unstructured case. The paper is well-written, contributes novel and tractable-to-estimate measures which are analysed formally and empirically with convincing experiments on ASR and NMT. 

The reviewers and myself are overall pleased by this submission. The discussion phase went well and most concerns have been resolved. 

Decision: Accept (Poster)