Single Layer Predictive Normalized Maximum Likelihood for Out-of-Distribution Detection

1 Introduction . An important concern that limits the adoption of deep neural networks ( DNN ) in critical safety systems is how to assess our confidence in their predictions , i.e , quantifying their generalization capability ( Kaufman et al. , 2019 ; Willers et al. , 2020 ) . Take , for instance , a machine learning model for medical diagnosis ( Bibas et al. , 2021 ) . It may produce ( wrong ) diagnoses in the presence of test inputs that are different from the training set rather than flagging them for human intervention ( Singh et al. , 2021 ) . Detecting such unexpected inputs had been formulated as the out-of-distribution ( OOD ) detection task ( Hendrycks and Gimpel , 2017 ) , as flagging test inputs that lie outside the training classes , i.e. , are not in-distribution ( IND ) . Previous learning methods that designed to offer such generalization measures , include VCdimension ( Zhong et al. , 2017 ; Vapnik and Chervonenkis , 2015 ) and norm based bounds ( Neyshabur et al. , 2018 ; Bartlett et al. , 2017 ) . As a whole , these methods characterized the generalization ability based on the properties of the parameters . However , they do not consider the test sample that is presented to the model ( Jiang et al. , 2020 ) , which makes them useless for OOD detection . Other approaches build heuristics over the empirical risk minimization ( ERM ) learner , by post-processing 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . the model output ( Sastry and Oore , 2020 ) or modifying the training process ( Papadopoulos et al. , 2021 ; Vyas et al. , 2018 ) . Regardless of the approach , these methods choose the learner that minimizes the loss over the training set . This may lead to a large generalization error because the ERM estimate may be wrong on unexpected inputs ; especially with large models such as DNN ( Belkin et al. , 2019 ) . To produce a useful generalization measure , we exploit the individual setting framework ( Merhav and Feder , 1998 ) . In the individual setting , there is no assumption about how the training and the test data are generated , nor about their probabilistic relationship . Moreover , the relationship between the labels and data can be deterministic and may therefore be determined by an adversary . The generalization error in this setting is often referred to as the regret ( Merhav and Feder , 1998 ) . This regret is defined by the log-loss difference between a learner and the genie : a learner that knows the specific test label , yet is constrained to use an explanation from a set of possible models . The individual setting is the most general framework and so the result holds for a wide range of scenarios . Specifically , the result holds to OOD detection where the distribution of the OOD inputs is unknown . The pNML learner ( Fogel and Feder , 2018 ) was proposed as the min-max solution of the regret , where the minimum is over the learner choice and the maximum is for any possible test label value . Intuitively , the pNML assigns a probability for a potential outcome as follows : Add the test sample to the training set with an arbitrary label , find the ERM solution of this new set , and take the probability it gives to the assumed label . Follow this procedure for every label and normalize to get a valid probability assignment . The pNML was developed before for linear regression ( Bibas et al. , 2019b ) and was evaluated empirically for DNN ( Fu and Levine , 2021 ; Bibas et al. , 2019a ) . We derive an analytical solution of the pNML learner and its generalization error ( the regret ) for a single layer NN . We analyze the derived regret and show it obtains low values when the test input either ( i ) lies in a subspace spanned by the eigenvectors associated with the large eigenvalues of the training data empirical correlation matrix or ( ii ) is located far from the decision boundary . Crucially , although our analysis focuses on a single layer NN , our results are applicable to the last layer of DNNs without changing the network architecture or the training process : We treat the pretrained DNN as a feature extractor with the last layer as a single layer NN classifier . We can therefore show the usage of the pNML regret as a confidence score for the OOD detection task . To summarize , we make the following contributions . 1 . Analytical derivation of the pNML regret . We derive an analytical expression of the pNML regret , which is associated with the generalization error , for a single layer NN . 2 . Analyzing the pNML regret . We explore the pNML regret characteristics as a function of the test sample data , training data , and the corresponding ERM prediction . We provide a visualization on low dimensional data and demonstrate the situations in which the pNML regret is low and the prediction can be trusted . 3 . DNN adaptation . We propose an adaptation of the derived pNML regret to any pretrained DNN that uses the softmax function with neither additional parameters nor extra data . Applying our pNML regret to a pretrained DNN does not require additional data , it is efficient and can be easily implemented . The derived regret is theoretically justified for OOD detection since it is the individual setting solution for which we do not require any knowledge on the test input distribution . Our evaluation includes 74 IND-OOD detection benchmarks using DenseNet-BC-100 , ResNet-34 , and WideResNet-40 trained with CIFAR-100 , CIFAR-10 , SVHN , and ImageNet-30 . Our approach outperforms leading methods in nearly all 74 OOD detection benchmarks up to a remarkable +15.2 % 2 Related work . OOD detection . Many recent work use additional OOD set in training ( Malinin and Gales , 2018 ; Hendrycks et al. , 2019a ; Nandy et al. , 2020 ; Mohseni et al. , 2020 ) . A well-known method for OOD detection , named ODIN ( Liang et al. , 2018 ) , manipulates the input image based on the gradient of the loss function . This input manipulation increases the margin between the maximum probability of images with known classes and images with unknown objects . However , the manipulation strength is defined by OOD data , which might be unavailable or different from the OOD samples in inference . Different approaches propose to manipulate the loss function in the training phase . Lee et al . ( 2018 ) proposed to use the Mahalanobis distance between the test input representations and the class conditional distribution in the intermediate layers to train a logistic regression classifier to determine if the input sample is OOD . The Energy method ( Liu et al. , 2020 ) adds a cost function to the training phase to shape the energy surface explicitly for OOD detection . Papadopoulos et al . ( 2021 ) suggested the outlier exposure with confidence control ( OECC ) method in which a regularization term is added to the loss function such that the model produces a uniform distribution for OOD samples . All mentioned methods require manipulating the DNN architecture or adding additional data . The Baseline method ( Hendrycks and Gimpel , 2017 ) is one of the earliest approaches designed to identify whether a test input is IND and does not require changing the pretrained model . This method uses the maximum probability of the DNN output as the OOD score . The Gram method ( Sastry and Oore , 2020 ) detects OODs based on feature representations obtained in intermediate layers . Given a test sample , the Gram matrices of the test sample features are compared with those of the training samples known to belong to the estimated class of the test sample . However , this method requires IND validation set and does not have a theoretical motivation . Other work suggested using Bayesian techniques to estimate the prediction confidence ( Gal and Ghahramani , 2016 ; Lakshminarayanan et al. , 2017 ; van Amersfoort et al. , 2020 ) . However , the Bayesian techniques add extensive compute overhead to the prediction . The pNML learner . The pNML learner is the min-max solution of the supervised batch learning in the individual setting ( Fogel and Feder , 2018 ) . For sequential prediction it is termed the conditional normalized maximum likelihood ( Rissanen and Roos , 2007 ; Roos and Rissanen , 2008 ) . Several methods deal with obtaining the pNML learner for different hypothesis sets . Bibas et al . ( 2019b ) and Bibas and Feder ( 2021 ) showed the pNML solution for linear regression . Rosas et al . ( 2020 ) proposed an NML based decision strategy for supervised classification problems and showed it attains heuristic PAC learning . Fu and Levine ( 2021 ) used the pNML for model optimization based on learning a density function by discretizing the space and fitting a distinct model for each value . For the DNN hypothesis set , Bibas et al . ( 2019a ) estimated the pNML distribution with DNN by fine-tuning the last layers of the network for every test input and label combination . This approach is computationally expensive since training is needed for every test input . Zhou and Levine ( 2020 ) suggested a way to accelerate the pNML computation in DNN by using approximate Bayesian inference techniques to produce a tractable approximation to the pNML . Pesso et al . ( 2021 ) used the pNML with adversarial target attack as a defense mechanism against adversarial perturbation . 3 Notation and preliminaries . In the supervised machine learning scenario , a training set consisting of N pairs of examples is given DN = { ( xn , yn ) } Nn=1 , xn ∈ X , yn ∈ Y , ( 1 ) where xn is the n-th data instance and yn is its corresponding label . The goal of a learner is to predict the unknown test label y given a new test data x by assigning probability distribution q ( ·|x ) to the unknown label . The performance is evaluated using the log-loss function ` ( q ; x , y ) = − log q ( y|x ) . ( 2 ) For the problem to be well-posed , we must make further assumptions on the class of possible models or hypothesis set that is used to find the relation between x and y. Denote Θ as a general index set , this class is a set of conditional probability distributions PΘ = { pθ ( y|x ) , θ ∈ Θ } . ( 3 ) The ERM is the learner from the hypothesis set that attains the minimal log-loss on the training set . The individual setting . An additional assumption required to solve the problem is related to how the data and the labels are generated . In this work we consider the individual setting ( Merhav and Feder , 1998 ; Fogel and Feder , 2018 ; Bibas et al. , 2019b , a ) , where the data and labels , both in the training and test , are specific individual quantities : We do not assume any probabilistic relationship between them , the labels may even be assigned in an adversarial manner . The genie . In the individual setting the goal is to compete with a reference learner , a genie , that has to following properties : ( i ) knows the test label value , ( ii ) is restricted to use a model from the given hypotheses set PΘ , and ( iii ) does not know which of the samples is the test . This reference learner then chooses a model that attains the minimum loss over the training set and the test sample θ̂ ( DN ; x , y ) = arg min θ∈Θ [ ` ( pθ ; x , y ) + N∑ n=1 ` ( pθ ; xn , yn ) ] . ( 4 ) The regret is the log-loss difference between a learner q and this genie : R ( q ; DN ; x , y ) = − log q ( y|x ) − [ − log pθ̂ ( DN ; x , y ) ( y|x ) ] . ( 5 ) Theorem 1 ( Fogel and Feder ( 2018 ) ) . The universal learner , denoted as the pNML , minimizes the regret for the worst case test label Γ = R∗ ( DN , x ) = min q max y∈Y R ( q ; DN ; x , y ) . ( 6 ) The pNML probability assignment and regret are qpNML ( y|x ) = pθ̂ ( DN ; x , y ) ( y|x ) ∑ y′∈Y pθ̂ ( DN ; x , y′ ) ( y ′|x ) , Γ = log ∑ y′∈Y pθ̂ ( DN ; x , y′ ) ( y ′|x ) . ( 7 ) Proof . The regret is equal for all choices of y . If we consider a different probability assignment , it should assign a smaller probability for at least one of the outcomes . If the true label is one of those outcomes it will lead to a higher regret . For more information see Fogel and Feder ( 2018 ) . The pNML regret is associated with the model complexity ( Zhang , 2012 ) . This complexity measure formalizes the intuition that a model that fits almost every data pattern very well would be much more complex than a model that provides a relatively good fit to a small set of data . Thus , the pNML incorporates a trade-off between goodness of fit and model complexity as measured by the regret . Online update of a neural network . Let XN and YN be the data and label matrices of N training points respectively XN = [ x1 x2 . . . xN ] > ∈ RN×M , YN = [ y1 y2 . . . yN ] > ∈ RN×C , ( 8 ) such that the number of input features and model outputs are M and C respectively . Denote X+N as the Moore–Penrose inverse of the data matrix X+N = { ( X > NXN ) −1X > N Rank ( X > NXN ) = M X > N ( XNX > N ) −1 otherwise , ( 9 ) f ( · ) and f−1 ( · ) as the activation and inverse activation functions , and θ ∈ RM×C as the learnable parameters . The ERM , that minimizes the training set mean squared error ( MSE ) , is θ̂N = X + Nf −1 ( YN ) = arg min θ ||YN − f ( XNθ ) ||22 . ( 10 ) Recently , Zhuang et al . ( 2020 ) suggested a recursive formulation of the DNN weights . Using their scheme only one training sample is processed at a time : Denote the projection of a sample x onto the orthogonal subspace of the training set correlation matrix as x⊥ = ( I −X+NXN ) x , ( 11 ) the update rule when receiving a new training sample with data x with label y is θ̂ ( DN ; x , y ) = θ̂N + g ( f−1 ( y ) − x > θ̂N ) , g , { 1 ‖x⊥‖2 x⊥ x⊥ 6= 0 1 1+x > X+NX + > N x X+NX + > N x , x⊥ = 0 . ( 12 ) In their paper , Zhuang et al . ( 2020 ) applied this formulation for a DNN , layer by layer .

Summary: This paper generalizes the *predictive normalized maximum likelihood (pNML)* learner to single-layer neural network (linear regression with softmax) and show its application to detect out of distribution test samples for neural networks. The pNML learner can output a regret value (defined with respect to the learner that knows the actual test label) for each testing sample and this regret value is used for classifying the testing samples as out-of-distribution. The proposed method achieves strong empirical performance on image recognition tasks and can be further combined with existing OOD methods.   I find the proposed method to be novel and empirically strong. Therefore, I am leaning toward accepting. I think the authors have done a good job setting up the intuition and explaining the pNML learner. However, I wish the authors had done more analysis for the proposed method when combined with neural network.