GRODIN: Improved Large-Scale Out-of-Domain detection via Back-propagation

1 INTRODUCTION . Many problems are solved with machine learning techniques today . Prominent examples of supervised learning power could be found in many domains : image classification , speech recognition , medical surveys , self-driving cars , text translation ( 6 ; 19 ; 28 ; 24 ; 1 ) , etc .. In some applications , it is very important to be confident on the prediction of a trained model : self-driving cars , medicine , stock market trading , etc .. Usually , the prediction mechanism remains uncovered and it requires additional efforts to ensure the model quality when working with real-world data . We search for a new way to automatically understand that prediction on this particular sample is not reliable with data uncertainty estimation or knowledge uncertainty estimation ( 2 ) . One of the most important tasks in this field is out-of-domain detection . It allows us to separate the data suitable for inference from the examples that are not intended for model usage . Due to the importance of the task out-of-domain methods evolved from ‘ naive ‘ approaches to sophisticated algorithms . These algorithms either analyse the prediction model results to understand if we could rely on them , or build a new dedicated model for OOD . The first type of the methods we call “ lightweight ” , as they usually could be used for on-the-fly verification of prediction . The other class could deliver better detection quality , but requires substantive computational resources and separate training . We refer to this one as “ heavy ” . The “ lightweight ” models are usually built on top of certain prediction tasks , such as classification , and are based on the properties of loss functions used in the training process . Softmax probability is an example of such property , it is directly used for many practical implementations ( 10 ; 7 ) . The prediction quality of the softmax probability thresholding could be extended with input preprocessing and probability calibration as shown in ODIN ( 20 ) algorithm . This method provides the state-of-theart results on most available benchmarks ( tiny-ImageNet ( 26 ) , CIFAR-10 ( 16 ) , CIFAR-100 ( 16 ) , MNIST ( 3 ) , SVHN ( 25 ) , etc. ) . However , there is room for improvement . There is a bunch of more challenging problems constructed on top of ImageNet ( 26 ) image classification task : ImageNet-A ( 12 ) , ImageNet-O ( 12 ) , ImageNet-R ( 8 ) , ImageNet-C ( 9 ) . The alternative “ heavy ” methods were developed : outlier exposure ( 11 ) ; mahalanobis distance-based confidence score ( 18 ) ; class of strong and theoretically justified out-of-distribution detectors called ensembles ( 2 ; 23 ; 22 ) . All these methods usually provide better results , but are hard to use in practice : some of them require changing the training loss function and therefore neural network retraining ( 11 ) ; or require solving auxiliary logistic regression problem and heavy computation of mahalanobis distance in ( 18 ) ; sometimes researcher need additional training and inference of several neural networks simultaneously for ensemble methods ( 22 ) . Recent studies in deep learning suggest that the gradient of the loss function taken for a particular example carries a lot of information other than model prediction . Influence functions ( 14 ) use information about first and second derivatives with respect to network weights to estimate what is the impact of a single precedent on test set quality . Neural tangent kernel ( 13 ) gives us another point of view on the training process and generalization from the position of gradient analysis . To the best of our knowledge , there is only one work in out-of-distribution detection domain that utilize the gradient of the model with respect to weights ( 17 ) . This approach uses information from the gradient of different layers as a building blocks for their separate DNN model . The approach requires a lot of out-of-distribution data to train the model and could not be assigned to “ lightweight ” models . In this paper , we focus on developing a new simple and computationally lightweight approach for detecting out-of-distribution examples based on gradient analysis . We investigate the derivative of cross-entropy loss function and decompose it into two parts , showing the difference between them . The proposed method is being tested on several challenging benchmarks and compared with various methods . Also , we analyse the impact of highly popular input preprocessing and give intuition why this technique helps OOD detectors . 2 METHOD . Model mistakes come from different sources : irreducible errors , low model capacity , wrong modeling assumptions , sample distribution shift , and so on . For each type of errors , we define a formal problem setup and develop methods to deal with it . Those setups are not mutually exclusive and their methods could complement each other . This paper focuses on a particular problem : estimation of the probability that a particular sample comes from a different population . To formally define this problem we use notation from Tab . 1 . Given this notation , there exist a statistical population Γ . Elements of this population are pairs ( x , y ) , where x ∈ Rn is a sample , y ∈ Y is a label . We assume that there exists a deterministic correspondence between population precedent and corresponding label . Statistically representative training set X is used to train prediction model by minimization of empirical loss function L. With enough data and mild assumptions on loss function , the resulting model with parameters θ̂ should replicate correspondence between x and y . As a result , our model should give a correct prediction to any x from the general population . Thus , given assumptions , there is a natural way to define xi as the out-of-domain sample : xi is out-of-domain↔ ∀ ( x , y ) ∈ Γ : xi 6= x ( 1 ) Our task is to estimate , if the particular sample xi is out-of-domain , given only the estimated model parameters θ̂ . The core idea of our method is quite simple . We train prediction model by minimisation of some additive loss function L = 1m ∑ ( x , y ) ∈X ` ( θ , x , y ) . When minimisation is performed by stochastic gradient descent or similar methods , the expectation of the model parameters gradient norm reduces on training set elements with the number of optimization steps . The expectation aims to zero for flexible enough decision functions . In tangent kernel space ( NTK for neural models ) the optimization process looks like seeking for optimization parameter θ that pulls representation of training set points ∇ ` ( θ , x , y ) to zero vector . We can use this property and determine the model behavior for an example by the distance of its representation in the trained tangent kernel space ( defined by θ̂ ) from zero . Using this information we can predict many practical properties of the particular prediction : expected accuracy , model uncertainty , out-of-domain points detection , etc .. In this paper , we focus on the latter case and build a method for out-of-domain detection on top of this principle . The modern definition of the domain depends on the application and we will show this difference in the experimental section . From a theoretical perspective , we equate domain and the population . This way we allow a researcher to define their notion of the domain with the selection of sampling technique . The ideally trained model brings all points of the population to a stationary point in model parameters space . In this paper we make an assumption that kernel of this transformation is equal to the population and the observed transformation differs from ideal one with Gaussian error : ∇θ ` ( θ , x , y ) |θ̂ = ∇θ ` ( θ , x , y ) |θ∗ + δ , δ ∼ N ( 0 , E ) . Now we can define a log probability of population Γ membership : g ( x , θ̂ ) = logP ( x|Γ ) ∼ ∥∥∇θ ` ( θ , x , y ) |θ̂∥∥22 The main problem for practical application of the method is to define a loss function for examples , that come to the model during its exploitation and have no label . In this case , it is natural to use the class assigned by the model ĉ as a label and check the uncertainty of the model in the decision . We focus , without loss of generality , on image classification problem over k classes with neural models . Negative log likelihood over softmax probability modeling is one of the most commonly used loss functions for this problem : ` ( θ , x , y ) = − log e hy ( x , θ ) ∑k c=1 e hc ( x , θ ) where hc are outputs of the neural model for each class c. Using this model and the label assumption we can formulate the result log probability of population membership g ( x , θ̂ ) for arbitrary point x : g ( x , θ̂ ) = ∥∥∥∥∥∇θ log ehĉ ( x , θ ) ∑k c=1 e hc ( x , θ ) ∣∣∣∣∣ θ̂ ∥∥∥∥∥ 2 2 Now we can choose significance level α and filter out points that belong to the population with probability less than this significance level : P ( x|Γ ) < α . In practice we don ’ t use significance level directly , but define a threshold δ ( α ) for g ( x , θ̂ ) . If we have information on out-of-domain examples we choose the threshold that delivers the selected significance level on cross-validation . We use this setup as a baseline variant of our method in the experimental section . In the literature there are at least two ways how to enhance the result of the baseline model : temperature scaling ( 5 ) , and input perturbation ( 20 ) . In our framework these enhancements are easily explained and compliment the baseline model . To show this we need to unfold components of the gradient vector : ∇θ ` ( x , θ̂ ) = −∇θ log maxc′ e h c′ ( x , θ ) ∑k c=1 e hc ( x , θ ) ∣∣∣ θ̂ = S ( x , θ̂ ) G ( x , θ̂ ) , where S ( x , θ̂ ) = 1 1+ ∑ c 6=ĉ e ( hc ( x , θ̂ ) −hĉ ( x , θ̂ ) ) G ( x , θ̂ ) = − ∑ c 6=ĉ e ( hc ( x , θ̂ ) −hĉ ( x , θ̂ ) ) ∂ ( hĉ ( x , θ ) −hc ( x , θ ) ) ∂θ ∣∣∣ θ̂ In the last expression there are two multiplicative parts ( S and G ) . S is a scalar part , that reflects the probability of the predicted class . The G is cumulative gradient vector for all classes output nodes of the network weighted by their preference over predicted class ehc ( x , θ̂ ) /ehĉ ( x , θ̂ ) . The first part is equal to the softmax score from ODIN up to the notations used . The second part is the information from NTK space . As one can see both S and G heavily depend on class weights hc , and it is important to calibrate these values for the task the model is used for . One of the most popular ways of model calibration is temperature parameter injection ( 5 ) . In this method we scale class weights hc with the temperature factor 1T . The optimal temperature is then optimized using some calibration quality metric like Expected Calibration Error ( 5 ) for the target dataset . The solution of this optimization problem depends on the data provided , see the experimental section for details . The other way of out-of-domain detection model improvement is altering the source signal , e.g . the image in case of the image classification task . In our framework we assume that during training process the optimization pulls domain space to a stationary point in model parameters space . As only training examples are observed , we pull the entire domain using the train set as mount points . In this process the rest of the domain slacks and this cause well-known difference in gradients between training set points and validation points . To reduce this slack we can move point towards training points . We follow the alteration procedure introduced in the ODIN paper : xp = xo − ε sign ( −∇x log maxc′ e hc′ ( x , θ ) /T∑k c=1 e hc ( x , θ ) /T ∣∣∣∣∣ xo ) During this procedure , source image is altered to reduce the gradient and compensate the slack between training points and the rest of the domain . To sum it up , the proposed method is a composition of ε-input perturbation , temperature calibration by scaling outputs by 1/T factor and decomposition of ∇θ ` ( x , θ̂ ) into two multiplicative parts : S ( x , θ̂ ) and G ( x , θ̂ ) . These parts can be used as a single estimator : g ( x , θ̂ ) = ‖S ( x , θ̂ ) G ( x , θ̂ ) ‖22 , or alone : g ( x , θ̂ ) = ‖G ( x , θ̂ ) ‖22 , g ( x , θ̂ ) = S ( x , θ̂ ) . The last variant gives us exactly ODIN .

Summary: This paper proposes to use the norm of gradients as the score to detect OOD data. The experiments use CIFAR-10 and ImageNet as the distribution datasets and use the unseen classes or perturbed images for the OOD data. The result shows the norm of gradients improves the OOD detection.