Evidential Turing Processes 

1 INTRODUCTION . The applicability of deep neural nets to safety-critical use cases such as autonomous driving or medical diagnostics is an active matter of fundamental research ( Schwalbe & Schels , 2020 ) . Key challenges in the development of total safety in deep learning are at least three-fold : i ) evaluation of model fit , ii ) risk assessment in difficult regions of the target domain ( e.g . class overlap ) based on which safety-preserving fallback functions can be deployed , and iii ) rejection of inputs that do not belong to the target domain , such as an image of a cat presented to a character recognition system . The attention of the machine learning community to each of these critical safety elements is in steady increase ( Naeini et al. , 2015 ; Guo et al. , 2017 ; Kuleshov et al. , 2018 ) . However , the developments often follow isolated directions and the proposed solutions are largely fragmented . Calibration of neural net probabilities focuses primarily on post-hoc adjustment algorithms trained on validation sets from in-domain data ( Guo et al. , 2017 ) , ignoring the out-of-domain detection and model fit evaluation aspects . On the other hand , recent advances in out-of-domain detection build on strong penalization of divergence from the probability mass observed in the target domain ( Sensoy et al. , 2018 ; Malinin & Gales , 2018 ) , distorting the quality of class probability scores . Such fragmentation of best practices not only hinders their accessibility by real-world applications but also complicates the scientific inquiry of the underlying reasons behind the sub-optimality of neural net uncertainties . We aim to identify the guiding principles for Bayesian modeling that could deliver the most complete set of uncertainty quantification capabilities in one single model . We first characterize Bayesian models with respect to the relationship of their global and local variables : ( a ) Parametric Bayesian Models comprise a likelihood function that maps inputs to outputs via probabilistic global parameters , ( b ) Evidential Bayesian Models apply an uninformative prior distribution on the parameters of the likelihood function and infer the prior hyperparameters by amortizing on an input observation . Investigating the decomposition of the predictive distribution variance , we find out that the ∗Work done while at Heidelberg University , Germany . uncertainty characteristics of these two approaches exhibit complementary strengths . We introduce a third approach that employs a prior on the likelihood parameters that both conditions on the input observations on the true model and amortizes on them during inference . The mapping from the input observations to the prior hyperparameters is governed by a random set of global parameters . We refer to the resulting approach as a ( c ) Complete Bayesian Model , since its predictive distribution variance combines the favorable properties of Parametric and Evidential Bayesian Models . Figure 1 gives an overview of the relationship between these three Bayesian modeling approaches . The advantages of the Complete Bayesian Models come with the challenge of choosing an inputdependent prior . We introduce a new stochastic process construction that addresses this problem by accumulating observations from a context set during training time into the parameters of a global hyperprior variable by an explicitly designed update rule . We design the input-specific prior on the likelihood parameters by conditioning it on both the hyperprior and the input observation . As conditioning via explicit parameter updates amounts to maintaining an external memory that can be queried by the prior distribution , we refer to the eventual stochastic process as a Turing Process with inspiration from the earlier work on Neural Turing Machines ( Graves et al. , 2014 ) and Neural Processes ( Garnelo et al. , 2018b ) . We arrive at our target Bayesian design that is equipped with the complete set of uncertainty quantification capabilities by incorporating the Turing Process design into a Complete Bayesian Model . We refer to the resulting approach as an Evidential Turing Process . We observe on five real-world classification tasks that the Evidential Turing Process is the only model that excels simultaneously at model fit , class overlap quantification , and out-of-domain detection . 2 PROBLEM STATEMENT : TOTAL CALIBRATION . Consider the following data generating process with K modes y|π ∼ Cat ( y|π ) , x|y ∼ K∑ k=1 Iy=kp ( x|y = k ) , ( 1 ) where Cat ( y|· ) is a categorical distribution , Iu is the indicator function for predicate u , and p ( x|y ) is the class-conditional density function on input observation x . For any prior class distribution π , the classification problem can be cast as identifying the class probabilities for observed patterns Pr [ y|x , π ] = Cat ( y|fπtrue ( x ) ) where fπtrue ( x ) is a K−dimensional vector with the k-th element fπ , ktrue ( x ) = πkp ( x|y = k ) / K∑ κ=1 πκp ( x|y = κ ) . ( 2 ) In many real-world cases fπtrue is not known and should be identified from a hypothesis space hπ ∈ Hπ via inference given a set of samples D = { ( xn , yn ) |n = 1 , . . . , N } obtained from the true distribution . The risk of the Bayes classifier arg maxy Cat ( y|hπ ( x ) ) for a given input x is R ( hπ ( x ) ) = K∑ κ=1 Iκ6=argmaxhπ ( x ) f π , κ true ( x ) . ( 3 ) For the whole data distribution , we have R ( hπ ) = Ex|π [ R ( hπ ( x ) ) ] . The optimal case is when hπ ( x ) = f π true ( x ) , which brings about the irreducible Bayes risk resulting from class overlap : R∗ ( f π true ( x ) ) = min { 1−max fπtrue ( x ) , max fπtrue ( x ) } . ( 4 ) Total Calibration . Given a set of test samples Dts coming from the true data generating process in Eq . 1 , we define Total Calibration as the capability of a discriminative predictor hπ ( x ) to quantify the following three types of uncertainty simultaneously : i ) Model misfit evaluated by the similarity of hπ ( x ) and fπtrue ( x ) measurable directly via KL ( Cat ( y|fπtrue ( x ) ) p ( x ) || Cat ( y|hπ ( x ) ) p ( x ) ) = Ep ( x ) [ log Cat ( y|fπtrue ( x ) ] + Ep ( x ) [ − log Cat ( y|hπ ( x ) ) ] where KL ( ·||· ) denotes the Kullback-Leibler divergence . Since Ep ( x ) [ log Cat ( y|fπtrue ( x ) ) ] is a constant with respect to hπ ( x ) , an unbiased estimator of the relative goodness of an inferred model is the negative test log-likelihood NLL ( hπ ( x ) ) = −1/|Dts| ∑ ( x , y ) ∈Dts log Cat ( y|hπ ( x ) ) . ii ) Class overlap evaluated by R∗ ( fπtrue ( x ) ) . As there is no way to measure this quantity from Dts , it is approximated indirectly via Expected Calibration Error ( ECE ) ECE [ hπ ] = M∑ m=1 |Bm| N ∣∣∣acc ( Bm ) − conf ( Bm ) ∣∣∣ where Bm = { ( n|hπ ( xn ) ∈ [ ( m − 1 ) /M , m/M ] } are M bins that partition the test set Dts into Dts = B1 ∪ · · · ∪BM and are characterized by their accuracy and confidence scores acc ( Bm ) = 1 |Bm| ∑ n∈Bm Iyn=argmaxhπ ( xn ) , conf ( Bm ) = 1 |Bm| ∑ n∈Bm hπ ( xn ) . ( 5 ) iii ) Domain mismatch defines the rarity of an input pattern x∗ for the target task , that is Pr [ p ( x = x∗ ) < ] for small . This quantity can not be measured since p ( x ) is unknown . It is approximated indirectly as the success of discriminating between samples x∗ coming from a different data distribution and those in Dts by calculating the Area Under ROC ( AUROC ) curve w.r.t . hπ ( x ) . 3 UNCERTAINTY QUANTIFICATION WITH PARAMETRIC AND EVIDENTIAL BAYESIAN MODELS . Parametric Bayesian Models . A commonplace design choice is to build the hypothesis space by equipping the hypothesis function with a global parameter θ that takes values from a feasible set Θ , that is Hπ = { hθπ ( x ) |θ ∈ Θ } . Parametric Bayesian Models ( PBMs ) employ a prior belief θ ∼ p ( θ ) , which is updated via Bayesian inference on a training set Dtr as p ( θ|Dtr ) = ∏ ( x , y ) ∈Dtr Cat ( y|h θ π ( x ) ) p ( θ ) /p ( Dtr ) . Consider the update on the posterior belief p ( θ|Dtr ∪ ( x∗ , y∗ ) ) = Cat ( y∗|hθπ ( x∗ ) ) p ( θ|Dtr ) /Z , ( 6 ) after a new observation ( x∗ , y∗ ) where the normalizer can be expressed as Z = 1 p ( Dtr ) ∫ Cat ( y∗|hθπ ( x∗ ) ) L ( θ ) p ( θ ) dθ . ( 7 ) with L ( θ ) = ∏ ( x , y ) ∈Dtr Cat ( y|h θ π ( x ) ) . This can be seen as an inner product between the functions Cat ( y∗|hθπ ( x∗ ) ) and L ( θ ) on an embedding space modulated by p ( θ ) . As the sample size grows , the high-density regions of Cat ( y∗|hθπ ( x∗ ) ) and L ( θ ) will superpose with higher probability , causing Z to increase , making p ( θ|Dtr ∪ ( x∗ , y∗ ) ) increasingly more peaked , and consequently we will have lim |Dtr|→+∞ V ar [ θ|Dtr ] = 0 . ( 8 ) This conjecture depends on the assumption that a single random variable θ modulates all samples . The variance of a prediction in favor of a class k decomposes according to the law of total variance V ar [ y∗ = k|x∗ , D ] = V arθ∼p ( θ|Dtr ) [ h θ π , k ( x∗ ) ] ︸ ︷︷ ︸ Reducible model uncertainty +Eθ∼p ( θ|Dtr ) [ ( 1− hθπ , k ( x∗ ) ) hθπ , k ( x∗ ) ] ︸ ︷︷ ︸ Data uncertainty . ( 9 ) Due to the Eq . 8 , the first term on the r.h.s . vanishes in the large sample regime and the second one coincides with the asymptotic solution of the maximum likelihood estimator θMLE . In cases when Hπ contains fπtrue ( x ) , an ideal inference scheme has the potential to recover it and we get lim N→+∞ Eθ∼p ( θ|Dtr ) [ ( 1− hθπ , k ( x∗ ) ) hθπ , k ( x∗ ) ] = ( 1− hθMLEπ , k ( x∗ ) ) h θMLE π , k ( x∗ ) = ( 1− fπtrue ( x∗ ) ) fπtrue ( x∗ ) . The simple proposition below sheds light on the meaning of this result . It points to the fact that the Bayes risk of a classifier ( Eq . 4 ) and the second component of its predictive variance ( Eq 9 ) are proportional . This gives a mechanistic explanation for why the second term on the r.h.s . of Eq . 9 quantifies class overlap . Despite the commonplace allocation of the wording data uncertainty for this term , we are not aware of prior work that derives it formally from basic learning-theoretic concepts . Proposition . The following inequality holds for any π , π′ ∈ [ 0.5 , 1 ] pair min ( 1− π , π ) ≥ min ( 1− π′ , π′ ) ⇒ ( 1− π ) π ≥ ( 1− π′ ) π′ . Proof . Define π = 0.5+ and π′ = 0.5+ ′ for , ′ > 0 . As min ( 1−π , π ) ≥ min ( 1−π′ , π′ ) ⇒ ≤ ′ , we get ( 1−π ) π = ( 1−0.5− ) ( 0.5+ ) = 0.25−0.25 − 2 ≥ 0.25−0.25 ′− ′2 = ( 1−π′ ) π′ The conclusion is that ( 1− fπtrue ( x ) ) fπtrue ( x ) ∝ R∗ ( fπtrue ( x ) ) . Hence , the second term on the r.h.s . of Eq . 9 is caused by the class overlap and it can be used to approximate the Bayes risk . Put together , the first term of the decomposition reflects the missing knowledge on the optimal value of θ that can be reduced by increasing the training set size , while the second reflects the uncertainty stemming from the properties of the true data distribution . Hence we refer to the first term as the reducible model uncertainty and the second the data uncertainty . Evidential Bayesian Models . In all the analysis above , we assumed a fixed and unknown π that modulates the whole process and can not be identified by the learning scheme . When we characterize the uncertainty on the class distribution by a prior belief π ∼ p ( π ) , we attain the joint p ( y , π|x ) = Pr [ y|x , π ] p ( π|x ) . The variance of the Bayes optimal classifier that averages over this uncertainty Pr [ y|x ] = ∫ Pr [ y|x , π ] p ( π|x ) dπ decomposes as V ar [ y = k|x ] = V arπ∼p ( π|x ) [ fπ , ktrue ( x ) ] ︸ ︷︷ ︸ Irreducible model uncertainty +Eπ∼p ( π|x ) [ ( 1− fπ , ktrue ( x ) ) f π , k true ( x ) ] ︸ ︷︷ ︸ Data uncertainty . The main source of uncertainty p ( π|x ) is not a posterior this time but an empirical prior on a single observation x , hence its variance will not shrink as |Dtr| → +∞ . Therefore , the first term on the r.h.s . reflects the irreducible model uncertainty caused by the missing knowledge about π . The second term indicates data uncertainty as in PBMs , but accounts for the local uncertainty at x caused by π. Evidential Bayesian Models ( EBMs ) ( Sensoy et al. , 2018 ; Malinin & Gales , 2018 ) suggest p ( y , π|x ) = Pr [ y|x , π ] p ( π|x ) ≈ Pr [ y|π ] qψ ( π|x ) where qψ ( π|x ) is a density function parameterized by ψ and the dependency of the class-conditional on the input is dropped . Note that the resultant model employs uncertainty on individual data points via π , but it does not have any global random variables . The standard evidential model training is performed by maximizing the expected likelihood subject to a regularizer that penalizes the divergence of qψ ( π|x ) from the prior belief arg min ψ − ∫ logPr [ y|π ] qψ ( π|x ) dπ + βKL ( qψ ( π|x ) ||p ( π ) ) . Although presented in the original work as as-hoc design choices , such a training objective can alternatively be viewed as β-regularized ( Higgins et al. , 2017 ) variational inference of Pr [ y|π ] p ( π|x ) with the approximate posterior qψ ( π|x ) ( Chen et al. , 2019 ) .

Summary: This paper introduces the category of complete Bayesian models, with the goal of providing both in-domain uncertainty quantification (that is, calibration) and and out-of-domain detection. Concretely, the authors develop the evidential Turing process as an implementation of complete Bayesian model. The approach is experimentally validated by experiments on FMNIST, CIFAR10 and SVHN, showing that the evidential Turing process is competitive on all criteria.