Amortized Conditional Normalized Maximum Likelihood

1 INTRODUCTION . Current machine learning methods provide unprecedented accuracy across a range of domains , from computer vision to natural language processing . However , in many high-stakes applications , such as medical diagnosis or autonomous driving , rare mistakes can be extremely costly , and thus effective deployment of learned models requires not only high expected accuracy , but also a way to measure the certainty in a model ’ s predictions in order to assess risk and allow the model to abstain from making decisions when there is low confidence in the prediction . While deep networks offer excellent prediction accuracy , they generally do not provide the means to accurately quantify their uncertainty . This is especially true on out-of-distribution inputs , where deep networks tend to make overconfident incorrect predictions ( Ovadia et al. , 2019 ) . In this paper , we tackle the problem of obtaining reliable uncertainty estimates under distribution shift . Most prior work approaches the problem of uncertainty estimation from the standpoint of Bayesian inference . By treating parameters as random variables with some prior distribution , Bayesian inference can compute posterior distributions that capture a notion of epistemic uncertainty and allow us to quantitatively reason about uncertainty in model predictions . However , computing accurate posterior distributions becomes intractable as we use very complex models like deep neural nets , and current approaches require highly approximate inference methods that fall short of the promise of full Bayesian modeling in practice . Bayesian methods also have a deep connection with the minimum description length ( MDL ) principle , a formalization of Occam ’ s razor that recasts learning as performing efficient lossless data compression and has been widely used as a motivation for model selection techniques . Codes corresponding to maximum-a-posteriori estimators and Bayes marginal likelihoods have been commonly used within the MDL framework . However , other coding schemes have been proposed in MDL centered around achieving different notions of minimax optimality . Interpreting coding schemes as predictive distributions , such methods can directly inspire prediction strategies that give conservative predictions and do not suffer from excessive overconfidence due to their minimax formulation . One such predictive distribution is the conditional normalized maximum likelihood ( CNML ) ( Grünwald , 2007 ; Rissanen and Roos , 2007 ; Roos et al. , 2008 ) model , also known as sequential NML or predictive NML ( Fogel and Feder , 2018b ) . To make a prediction on a new input , CNML considers every possible label and tries to find the model that best explains that label for the query point together with the training set . It then uses that corresponding model to assign probabilities for each input and normalizes to obtain a valid probability distribution . Intuitively , instead of relying on a learned model to extrapolate from the training set to the new ( potentially out-of-distribution ) input , CNML can obtain more reasonable predictive distributions by asking “ given the training data , which labels would make sense for this input ? ” While CNML provides compelling minimax regret guarantees , practical instantiations have been exceptionally difficult , because computing predictions for a test point requires retraining the model on the test point concatenated with the entire training set . With large models like deep neural networks , this can potentially require hours of training for every prediction . In this paper , we proposed amortized CNML ( ACNML ) , a tractable and practical algorithm for approximating CNML utilizing approximate Bayesian inference . ACNML avoids the need to optimize over large datasets during inference by using an approximate posterior in place of the training set . We demonstrate that our proposed approach is substantially more feasible and computationally efficient than prior techniques for using CNML predictions with deep neural networks and compares favorably to a number of prior techniques for uncertainty estimation on out-of-distribution inputs . 2 MINIMUM DESCRIPTION LENGTH : BACKGROUND AND PRELIMINARIES . ACNML is motivated from the minimum description length ( MDL ) principle , which can be used to derive a connection between optimal codes and prediction . We begin with a review of the MDL principle and discuss the challenges in implementing minimax codes that motivate our method . For more comprehensive treatments of MDL , we refer the readers to ( Grünwald , 2007 ; Rissanen , 1989 ) . Minimum description length . The MDL principle states that any regularities in a dataset can be exploited to compress it , and hence learning is reformulated as losslessly transmitting the data with the fewest number of bits ( Rissanen , 1989 ; Grünwald , 2007 ) . Simplicity is thus formalized as the length of the resulting description . MDL was originally formulated in a generative setting where the goal is to code arbitrary data , and we will present a brief overview in this setting . We can translate the results to a supervised learning setting , which corresponds to transmitting the labels after assuming either a fixed coding scheme for the inputs or that the inputs are known beforehand . While MDL is typically described in terms of code lengths , in general , we can associate codes with probability distributions , with the code length of an object corresponding to the negative log-likelihood under that probability distribution ( Cover and Thomas , 2006 ) . Normalized Maximum Likelihood . Let θ̂ ( x1 : n ) denote the maximum likelihood estimator for a sequence of data x1 : n over all θ ∈ Θ . For any x1 : n ∈ Xn and distribution q over Xn , we can define a regret relative to the model class Θ as R ( q , Θ , x1 : n ) def = log pθ̂ ( x1 : n ) ( x1 : n ) − log q ( x1 : n ) . ( 1 ) This regret corresponds to the excess number of bits q uses to encode x1 : n compared to the best distribution in Θ , denoted θ̂ ( x1 : n ) . We can then define the normalized maximum likelihood distribution ( NML ) with respect to Θ as pNML ( x1 : n ) = pθ̂ ( x1 : n ) ( x1 : n ) ∑ x̃1 : n∈Xn pθ̂ ( x̃1 : n ) ( x̃1 : n ) ( 2 ) when the denominator is finite . The NML distribution can be shown to achieve minimax regret ( Shtarkov , 1987 ; Rissanen , 1996 ) pNML = argmin q max x1 : n∈Xn R ( q , Θ , x1 : n ) . ( 3 ) This corresponds , in a sense , to an optimal coding scheme for sequences of known fixed length . Conditional NML . Instead of making predictions across entire sequences at once , we can adapt NML to the setting where we make predictions about the next data point based on the previously seen data , resulting in conditional NML ( CNML ) ( Rissanen and Roos , 2007 ; Grünwald , 2007 ; Fogel and Feder , 2018a ) . While several variations on CNML exist , we consider the following : pCNML ( xn|x1 : n−1 ) ∝ pθ̂ ( x1 : n ) ( xn ) . ( 4 ) For any fixed sequence x1 : n−1 , pCNML solves the minimax regret problem pCNML = argmin q max xn log pθ̂ ( x1 : n ) ( xn ) − log q ( xn ) , ( 5 ) where the inner maximization is only over the last data point xn . We can extend this approach to the supervised classification setting , where our models represent conditional distributions pθ ( y|x ) . The CNML distribution , given a sequence of already seen datapoints ( x1 : n−1 , y1 : n−1 ) and the next input xn , then takes the form pCNML ( yn|xn ; x1 : n−1 , y1 : n−1 ) ∝ pθ̂ ( y1 : n|x1 : n ) ( yn|xn ) , ( 6 ) and solves the minimax problem pCNML = argmin q max yn log pθ̂ ( y1 : n|x1 : n ) ( yn|xn ) − log q ( yn ) . ( 7 ) We see that this conditional distribution is amenable to our usual inductive learning procedure , where ( x1 : n−1 , y1 : n−1 ) is our training set , and we want to output a predictive distribution over labels yn for a new test input xn . CNML provides conservative predictions . For each query point , CNML considers each potential label and finds the model that would be most consistent with that label and with the training set . If that model assigns high probability to the label , then minimizing the worst-case regret forces CNML to assign relatively high probability to it . In particular , compared to simply letting a model trained only on the training set extrapolate blindly , we expect CNML to give more conservative predictions on out-of-distribution inputs , since it explicitly considers what would have happened if the new data point had been included in the training dataset with each particular label . We use a 2D logistic regression example to illustrate CNML ’ s conservative predictions , showing a heatmap of CNML probabilities in Figure 1 . CNML provides uniform predictions on most of the input space away from the training samples . In Figure 2 , we illustrate how CNML arrives at these predictions , showing the predictions for the parameters θ̂0 and θ̂1 , corresponding to labeling the test point ( shown in pink in Figure 2 , left ) with either the label 0 or 1 . However , CNML may be too conservative when the model class Θ is very expressive . Naïvely applying CNML with large model classes can result in the per-label models fitting their labels for the query point arbitrarily well , such that CNML gives unhelpful uniform predictions even on inputs we would hope to reasonably extrapolate on . We see this in the 2D logistic regression example in Figure 1 . Thus , the model class Θ would need to be restricted in some form , for example by only considering only parameters within a certain distance from the training set solution as a hard constraint . Another approach for controlling the expressivity of the model class is to generalize CNML to use regularized estimators instead of maximum likelihood , resulting in normalized maximum a posteriori ( NMAP ) ( Kakade et al. , 2006 ) codes . Instead of using maximum likelihood parameters , NMAP selects θ̂s to be the parameter that maximizes both data likelihood and a regularization term , or prior , over parameters , and we can define slightly altered notions of regret using these MAP estimators in all the previous equations to get a conditional normalized maximum a posteriori distribution instead . See Appendix D for completeness . Going back to the logistic regression example , we plot heatmaps of CNMAP predictions in Figure 3 , adding different amounts of L2 regularization to the logistic regression weights . As we add more regularization , the model class becomes effectively less expressive , and the CNMAP predictions become less conservative . Computational Costs of CNML . A major practical issue with actually utilizing CNML or CNMAP with neural networks is the prohibitive computational costs of computing the maximum likelihood estimators for each new input and label combination . To evaluate the distribution on a new test point , one must solve a nonconvex optimization problem for each possible label , with each problem involving the entire training dataset along with the new test point . This direct evaluation of CNML therefore becomes computationally infeasible with large datasets and high-capacity models , and further requires that the model carry around the entire training set even when it is deployed . In settings where critical decisions must be made in real time , even running a single epoch of additional training would be infeasible . For this reason , NML-based methods have not gained much traction as a practical tool for improving the predictive performance of high-capacity models .

Summary: The paper presents an approach based on conditional normalized maximum likelihood (CNML) for uncertainty estimation, calibration, and out-of-distribution robustness with deep networks. CNML is intractable to compute in general and therefore the authors propose a tractable approximation which uses approximate Bayesian inference techniques. Experimentally, the authors show that their new approach is competitive and sometimes better than existing approaches for uncertainty estimation and calibration on out-of-distribution test data points.