Evidential Turing Processes 

1 INTRODUCTION . The applicability of deep neural nets to safety-critical use cases such as autonomous driving or medical diagnostics is an active matter of fundamental research ( Schwalbe & Schels , 2020 ) . Key challenges in the development of total safety in deep learning are at least three-fold : i ) evaluation of model fit , ii ) risk assessment in difficult regions of the target domain ( e.g . class overlap ) based on which safety-preserving fallback functions can be deployed , and iii ) rejection of inputs that do not belong to the target domain , such as an image of a cat presented to a character recognition system . The attention of the machine learning community to each of these critical safety elements is in steady increase ( Naeini et al. , 2015 ; Guo et al. , 2017 ; Kuleshov et al. , 2018 ) . However , the developments often follow isolated directions and the proposed solutions are largely fragmented . Calibration of neural net probabilities focuses primarily on post-hoc adjustment algorithms trained on validation sets from in-domain data ( Guo et al. , 2017 ) , ignoring the out-of-domain detection and model fit evaluation aspects . On the other hand , recent advances in out-of-domain detection build on strong penalization of divergence from the probability mass observed in the target domain ( Sensoy et al. , 2018 ; Malinin & Gales , 2018 ) , distorting the quality of class probability scores . Such fragmentation of best practices not only hinders their accessibility by real-world applications but also complicates the scientific inquiry of the underlying reasons behind the sub-optimality of neural net uncertainties . We aim to identify the guiding principles for Bayesian modeling that could deliver the most complete set of uncertainty quantification capabilities in one single model . We first characterize Bayesian models with respect to the relationship of their global and local variables : ( a ) Parametric Bayesian Models comprise a likelihood function that maps inputs to outputs via probabilistic global parameters , ( b ) Evidential Bayesian Models apply an uninformative prior distribution on the parameters of the likelihood function and infer the prior hyperparameters by amortizing on an input observation . Investigating the decomposition of the predictive distribution variance , we find out that the ∗Work done while at Heidelberg University , Germany . uncertainty characteristics of these two approaches exhibit complementary strengths . We introduce a third approach that employs a prior on the likelihood parameters that both conditions on the input observations on the true model and amortizes on them during inference . The mapping from the input observations to the prior hyperparameters is governed by a random set of global parameters . We refer to the resulting approach as a ( c ) Complete Bayesian Model , since its predictive distribution variance combines the favorable properties of Parametric and Evidential Bayesian Models . Figure 1 gives an overview of the relationship between these three Bayesian modeling approaches . The advantages of the Complete Bayesian Models come with the challenge of choosing an inputdependent prior . We introduce a new stochastic process construction that addresses this problem by accumulating observations from a context set during training time into the parameters of a global hyperprior variable by an explicitly designed update rule . We design the input-specific prior on the likelihood parameters by conditioning it on both the hyperprior and the input observation . As conditioning via explicit parameter updates amounts to maintaining an external memory that can be queried by the prior distribution , we refer to the eventual stochastic process as a Turing Process with inspiration from the earlier work on Neural Turing Machines ( Graves et al. , 2014 ) and Neural Processes ( Garnelo et al. , 2018b ) . We arrive at our target Bayesian design that is equipped with the complete set of uncertainty quantification capabilities by incorporating the Turing Process design into a Complete Bayesian Model . We refer to the resulting approach as an Evidential Turing Process . We observe on five real-world classification tasks that the Evidential Turing Process is the only model that excels simultaneously at model fit , class overlap quantification , and out-of-domain detection . 2 PROBLEM STATEMENT : TOTAL CALIBRATION . Consider the following data generating process with K modes y|π ∼ Cat ( y|π ) , x|y ∼ K∑ k=1 Iy=kp ( x|y = k ) , ( 1 ) where Cat ( y|· ) is a categorical distribution , Iu is the indicator function for predicate u , and p ( x|y ) is the class-conditional density function on input observation x . For any prior class distribution π , the classification problem can be cast as identifying the class probabilities for observed patterns Pr [ y|x , π ] = Cat ( y|fπtrue ( x ) ) where fπtrue ( x ) is a K−dimensional vector with the k-th element fπ , ktrue ( x ) = πkp ( x|y = k ) / K∑ κ=1 πκp ( x|y = κ ) . ( 2 ) In many real-world cases fπtrue is not known and should be identified from a hypothesis space hπ ∈ Hπ via inference given a set of samples D = { ( xn , yn ) |n = 1 , . . . , N } obtained from the true distribution . The risk of the Bayes classifier arg maxy Cat ( y|hπ ( x ) ) for a given input x is R ( hπ ( x ) ) = K∑ κ=1 Iκ6=argmaxhπ ( x ) f π , κ true ( x ) . ( 3 ) For the whole data distribution , we have R ( hπ ) = Ex|π [ R ( hπ ( x ) ) ] . The optimal case is when hπ ( x ) = f π true ( x ) , which brings about the irreducible Bayes risk resulting from class overlap : R∗ ( f π true ( x ) ) = min { 1−max fπtrue ( x ) , max fπtrue ( x ) } . ( 4 ) Total Calibration . Given a set of test samples Dts coming from the true data generating process in Eq . 1 , we define Total Calibration as the capability of a discriminative predictor hπ ( x ) to quantify the following three types of uncertainty simultaneously : i ) Model misfit evaluated by the similarity of hπ ( x ) and fπtrue ( x ) measurable directly via KL ( Cat ( y|fπtrue ( x ) ) p ( x ) || Cat ( y|hπ ( x ) ) p ( x ) ) = Ep ( x ) [ log Cat ( y|fπtrue ( x ) ] + Ep ( x ) [ − log Cat ( y|hπ ( x ) ) ] where KL ( ·||· ) denotes the Kullback-Leibler divergence . Since Ep ( x ) [ log Cat ( y|fπtrue ( x ) ) ] is a constant with respect to hπ ( x ) , an unbiased estimator of the relative goodness of an inferred model is the negative test log-likelihood NLL ( hπ ( x ) ) = −1/|Dts| ∑ ( x , y ) ∈Dts log Cat ( y|hπ ( x ) ) . ii ) Class overlap evaluated by R∗ ( fπtrue ( x ) ) . As there is no way to measure this quantity from Dts , it is approximated indirectly via Expected Calibration Error ( ECE ) ECE [ hπ ] = M∑ m=1 |Bm| N ∣∣∣acc ( Bm ) − conf ( Bm ) ∣∣∣ where Bm = { ( n|hπ ( xn ) ∈ [ ( m − 1 ) /M , m/M ] } are M bins that partition the test set Dts into Dts = B1 ∪ · · · ∪BM and are characterized by their accuracy and confidence scores acc ( Bm ) = 1 |Bm| ∑ n∈Bm Iyn=argmaxhπ ( xn ) , conf ( Bm ) = 1 |Bm| ∑ n∈Bm hπ ( xn ) . ( 5 ) iii ) Domain mismatch defines the rarity of an input pattern x∗ for the target task , that is Pr [ p ( x = x∗ ) < ] for small . This quantity can not be measured since p ( x ) is unknown . It is approximated indirectly as the success of discriminating between samples x∗ coming from a different data distribution and those in Dts by calculating the Area Under ROC ( AUROC ) curve w.r.t . hπ ( x ) . 3 UNCERTAINTY QUANTIFICATION WITH PARAMETRIC AND EVIDENTIAL BAYESIAN MODELS . Parametric Bayesian Models . A commonplace design choice is to build the hypothesis space by equipping the hypothesis function with a global parameter θ that takes values from a feasible set Θ , that is Hπ = { hθπ ( x ) |θ ∈ Θ } . Parametric Bayesian Models ( PBMs ) employ a prior belief θ ∼ p ( θ ) , which is updated via Bayesian inference on a training set Dtr as p ( θ|Dtr ) = ∏ ( x , y ) ∈Dtr Cat ( y|h θ π ( x ) ) p ( θ ) /p ( Dtr ) . Consider the update on the posterior belief p ( θ|Dtr ∪ ( x∗ , y∗ ) ) = Cat ( y∗|hθπ ( x∗ ) ) p ( θ|Dtr ) /Z , ( 6 ) after a new observation ( x∗ , y∗ ) where the normalizer can be expressed as Z = 1 p ( Dtr ) ∫ Cat ( y∗|hθπ ( x∗ ) ) L ( θ ) p ( θ ) dθ . ( 7 ) with L ( θ ) = ∏ ( x , y ) ∈Dtr Cat ( y|h θ π ( x ) ) . This can be seen as an inner product between the functions Cat ( y∗|hθπ ( x∗ ) ) and L ( θ ) on an embedding space modulated by p ( θ ) . As the sample size grows , the high-density regions of Cat ( y∗|hθπ ( x∗ ) ) and L ( θ ) will superpose with higher probability , causing Z to increase , making p ( θ|Dtr ∪ ( x∗ , y∗ ) ) increasingly more peaked , and consequently we will have lim |Dtr|→+∞ V ar [ θ|Dtr ] = 0 . ( 8 ) This conjecture depends on the assumption that a single random variable θ modulates all samples . The variance of a prediction in favor of a class k decomposes according to the law of total variance V ar [ y∗ = k|x∗ , D ] = V arθ∼p ( θ|Dtr ) [ h θ π , k ( x∗ ) ] ︸ ︷︷ ︸ Reducible model uncertainty +Eθ∼p ( θ|Dtr ) [ ( 1− hθπ , k ( x∗ ) ) hθπ , k ( x∗ ) ] ︸ ︷︷ ︸ Data uncertainty . ( 9 ) Due to the Eq . 8 , the first term on the r.h.s . vanishes in the large sample regime and the second one coincides with the asymptotic solution of the maximum likelihood estimator θMLE . In cases when Hπ contains fπtrue ( x ) , an ideal inference scheme has the potential to recover it and we get lim N→+∞ Eθ∼p ( θ|Dtr ) [ ( 1− hθπ , k ( x∗ ) ) hθπ , k ( x∗ ) ] = ( 1− hθMLEπ , k ( x∗ ) ) h θMLE π , k ( x∗ ) = ( 1− fπtrue ( x∗ ) ) fπtrue ( x∗ ) . The simple proposition below sheds light on the meaning of this result . It points to the fact that the Bayes risk of a classifier ( Eq . 4 ) and the second component of its predictive variance ( Eq 9 ) are proportional . This gives a mechanistic explanation for why the second term on the r.h.s . of Eq . 9 quantifies class overlap . Despite the commonplace allocation of the wording data uncertainty for this term , we are not aware of prior work that derives it formally from basic learning-theoretic concepts . Proposition . The following inequality holds for any π , π′ ∈ [ 0.5 , 1 ] pair min ( 1− π , π ) ≥ min ( 1− π′ , π′ ) ⇒ ( 1− π ) π ≥ ( 1− π′ ) π′ . Proof . Define π = 0.5+ and π′ = 0.5+ ′ for , ′ > 0 . As min ( 1−π , π ) ≥ min ( 1−π′ , π′ ) ⇒ ≤ ′ , we get ( 1−π ) π = ( 1−0.5− ) ( 0.5+ ) = 0.25−0.25 − 2 ≥ 0.25−0.25 ′− ′2 = ( 1−π′ ) π′ The conclusion is that ( 1− fπtrue ( x ) ) fπtrue ( x ) ∝ R∗ ( fπtrue ( x ) ) . Hence , the second term on the r.h.s . of Eq . 9 is caused by the class overlap and it can be used to approximate the Bayes risk . Put together , the first term of the decomposition reflects the missing knowledge on the optimal value of θ that can be reduced by increasing the training set size , while the second reflects the uncertainty stemming from the properties of the true data distribution . Hence we refer to the first term as the reducible model uncertainty and the second the data uncertainty . Evidential Bayesian Models . In all the analysis above , we assumed a fixed and unknown π that modulates the whole process and can not be identified by the learning scheme . When we characterize the uncertainty on the class distribution by a prior belief π ∼ p ( π ) , we attain the joint p ( y , π|x ) = Pr [ y|x , π ] p ( π|x ) . The variance of the Bayes optimal classifier that averages over this uncertainty Pr [ y|x ] = ∫ Pr [ y|x , π ] p ( π|x ) dπ decomposes as V ar [ y = k|x ] = V arπ∼p ( π|x ) [ fπ , ktrue ( x ) ] ︸ ︷︷ ︸ Irreducible model uncertainty +Eπ∼p ( π|x ) [ ( 1− fπ , ktrue ( x ) ) f π , k true ( x ) ] ︸ ︷︷ ︸ Data uncertainty . The main source of uncertainty p ( π|x ) is not a posterior this time but an empirical prior on a single observation x , hence its variance will not shrink as |Dtr| → +∞ . Therefore , the first term on the r.h.s . reflects the irreducible model uncertainty caused by the missing knowledge about π . The second term indicates data uncertainty as in PBMs , but accounts for the local uncertainty at x caused by π. Evidential Bayesian Models ( EBMs ) ( Sensoy et al. , 2018 ; Malinin & Gales , 2018 ) suggest p ( y , π|x ) = Pr [ y|x , π ] p ( π|x ) ≈ Pr [ y|π ] qψ ( π|x ) where qψ ( π|x ) is a density function parameterized by ψ and the dependency of the class-conditional on the input is dropped . Note that the resultant model employs uncertainty on individual data points via π , but it does not have any global random variables . The standard evidential model training is performed by maximizing the expected likelihood subject to a regularizer that penalizes the divergence of qψ ( π|x ) from the prior belief arg min ψ − ∫ logPr [ y|π ] qψ ( π|x ) dπ + βKL ( qψ ( π|x ) ||p ( π ) ) . Although presented in the original work as as-hoc design choices , such a training objective can alternatively be viewed as β-regularized ( Higgins et al. , 2017 ) variational inference of Pr [ y|π ] p ( π|x ) with the approximate posterior qψ ( π|x ) ( Chen et al. , 2019 ) .

Summary: The paper presents a complete Bayesian model that combines strengths of parametric and evidential Bayesian models.  An instance of this model is an Evidential Turing Process (ETP) -- a model in which the prior on the global variable that governs the likelihood function is input-dependent, and this dependence is established in a way similar to the Neural Turing Machines. Unlike Neural Processes (NPs), it doesn't need a context set during test time as the context is extracted from the training set and stored in a memory variable. Experiments show that  ETPs are somewhat better than BNNs and evidential NPs.