GRODIN: Improved Large-Scale Out-of-Domain detection via Back-propagation

1 INTRODUCTION . Many problems are solved with machine learning techniques today . Prominent examples of supervised learning power could be found in many domains : image classification , speech recognition , medical surveys , self-driving cars , text translation ( 6 ; 19 ; 28 ; 24 ; 1 ) , etc .. In some applications , it is very important to be confident on the prediction of a trained model : self-driving cars , medicine , stock market trading , etc .. Usually , the prediction mechanism remains uncovered and it requires additional efforts to ensure the model quality when working with real-world data . We search for a new way to automatically understand that prediction on this particular sample is not reliable with data uncertainty estimation or knowledge uncertainty estimation ( 2 ) . One of the most important tasks in this field is out-of-domain detection . It allows us to separate the data suitable for inference from the examples that are not intended for model usage . Due to the importance of the task out-of-domain methods evolved from ‘ naive ‘ approaches to sophisticated algorithms . These algorithms either analyse the prediction model results to understand if we could rely on them , or build a new dedicated model for OOD . The first type of the methods we call “ lightweight ” , as they usually could be used for on-the-fly verification of prediction . The other class could deliver better detection quality , but requires substantive computational resources and separate training . We refer to this one as “ heavy ” . The “ lightweight ” models are usually built on top of certain prediction tasks , such as classification , and are based on the properties of loss functions used in the training process . Softmax probability is an example of such property , it is directly used for many practical implementations ( 10 ; 7 ) . The prediction quality of the softmax probability thresholding could be extended with input preprocessing and probability calibration as shown in ODIN ( 20 ) algorithm . This method provides the state-of-theart results on most available benchmarks ( tiny-ImageNet ( 26 ) , CIFAR-10 ( 16 ) , CIFAR-100 ( 16 ) , MNIST ( 3 ) , SVHN ( 25 ) , etc. ) . However , there is room for improvement . There is a bunch of more challenging problems constructed on top of ImageNet ( 26 ) image classification task : ImageNet-A ( 12 ) , ImageNet-O ( 12 ) , ImageNet-R ( 8 ) , ImageNet-C ( 9 ) . The alternative “ heavy ” methods were developed : outlier exposure ( 11 ) ; mahalanobis distance-based confidence score ( 18 ) ; class of strong and theoretically justified out-of-distribution detectors called ensembles ( 2 ; 23 ; 22 ) . All these methods usually provide better results , but are hard to use in practice : some of them require changing the training loss function and therefore neural network retraining ( 11 ) ; or require solving auxiliary logistic regression problem and heavy computation of mahalanobis distance in ( 18 ) ; sometimes researcher need additional training and inference of several neural networks simultaneously for ensemble methods ( 22 ) . Recent studies in deep learning suggest that the gradient of the loss function taken for a particular example carries a lot of information other than model prediction . Influence functions ( 14 ) use information about first and second derivatives with respect to network weights to estimate what is the impact of a single precedent on test set quality . Neural tangent kernel ( 13 ) gives us another point of view on the training process and generalization from the position of gradient analysis . To the best of our knowledge , there is only one work in out-of-distribution detection domain that utilize the gradient of the model with respect to weights ( 17 ) . This approach uses information from the gradient of different layers as a building blocks for their separate DNN model . The approach requires a lot of out-of-distribution data to train the model and could not be assigned to “ lightweight ” models . In this paper , we focus on developing a new simple and computationally lightweight approach for detecting out-of-distribution examples based on gradient analysis . We investigate the derivative of cross-entropy loss function and decompose it into two parts , showing the difference between them . The proposed method is being tested on several challenging benchmarks and compared with various methods . Also , we analyse the impact of highly popular input preprocessing and give intuition why this technique helps OOD detectors . 2 METHOD . Model mistakes come from different sources : irreducible errors , low model capacity , wrong modeling assumptions , sample distribution shift , and so on . For each type of errors , we define a formal problem setup and develop methods to deal with it . Those setups are not mutually exclusive and their methods could complement each other . This paper focuses on a particular problem : estimation of the probability that a particular sample comes from a different population . To formally define this problem we use notation from Tab . 1 . Given this notation , there exist a statistical population Γ . Elements of this population are pairs ( x , y ) , where x ∈ Rn is a sample , y ∈ Y is a label . We assume that there exists a deterministic correspondence between population precedent and corresponding label . Statistically representative training set X is used to train prediction model by minimization of empirical loss function L. With enough data and mild assumptions on loss function , the resulting model with parameters θ̂ should replicate correspondence between x and y . As a result , our model should give a correct prediction to any x from the general population . Thus , given assumptions , there is a natural way to define xi as the out-of-domain sample : xi is out-of-domain↔ ∀ ( x , y ) ∈ Γ : xi 6= x ( 1 ) Our task is to estimate , if the particular sample xi is out-of-domain , given only the estimated model parameters θ̂ . The core idea of our method is quite simple . We train prediction model by minimisation of some additive loss function L = 1m ∑ ( x , y ) ∈X ` ( θ , x , y ) . When minimisation is performed by stochastic gradient descent or similar methods , the expectation of the model parameters gradient norm reduces on training set elements with the number of optimization steps . The expectation aims to zero for flexible enough decision functions . In tangent kernel space ( NTK for neural models ) the optimization process looks like seeking for optimization parameter θ that pulls representation of training set points ∇ ` ( θ , x , y ) to zero vector . We can use this property and determine the model behavior for an example by the distance of its representation in the trained tangent kernel space ( defined by θ̂ ) from zero . Using this information we can predict many practical properties of the particular prediction : expected accuracy , model uncertainty , out-of-domain points detection , etc .. In this paper , we focus on the latter case and build a method for out-of-domain detection on top of this principle . The modern definition of the domain depends on the application and we will show this difference in the experimental section . From a theoretical perspective , we equate domain and the population . This way we allow a researcher to define their notion of the domain with the selection of sampling technique . The ideally trained model brings all points of the population to a stationary point in model parameters space . In this paper we make an assumption that kernel of this transformation is equal to the population and the observed transformation differs from ideal one with Gaussian error : ∇θ ` ( θ , x , y ) |θ̂ = ∇θ ` ( θ , x , y ) |θ∗ + δ , δ ∼ N ( 0 , E ) . Now we can define a log probability of population Γ membership : g ( x , θ̂ ) = logP ( x|Γ ) ∼ ∥∥∇θ ` ( θ , x , y ) |θ̂∥∥22 The main problem for practical application of the method is to define a loss function for examples , that come to the model during its exploitation and have no label . In this case , it is natural to use the class assigned by the model ĉ as a label and check the uncertainty of the model in the decision . We focus , without loss of generality , on image classification problem over k classes with neural models . Negative log likelihood over softmax probability modeling is one of the most commonly used loss functions for this problem : ` ( θ , x , y ) = − log e hy ( x , θ ) ∑k c=1 e hc ( x , θ ) where hc are outputs of the neural model for each class c. Using this model and the label assumption we can formulate the result log probability of population membership g ( x , θ̂ ) for arbitrary point x : g ( x , θ̂ ) = ∥∥∥∥∥∇θ log ehĉ ( x , θ ) ∑k c=1 e hc ( x , θ ) ∣∣∣∣∣ θ̂ ∥∥∥∥∥ 2 2 Now we can choose significance level α and filter out points that belong to the population with probability less than this significance level : P ( x|Γ ) < α . In practice we don ’ t use significance level directly , but define a threshold δ ( α ) for g ( x , θ̂ ) . If we have information on out-of-domain examples we choose the threshold that delivers the selected significance level on cross-validation . We use this setup as a baseline variant of our method in the experimental section . In the literature there are at least two ways how to enhance the result of the baseline model : temperature scaling ( 5 ) , and input perturbation ( 20 ) . In our framework these enhancements are easily explained and compliment the baseline model . To show this we need to unfold components of the gradient vector : ∇θ ` ( x , θ̂ ) = −∇θ log maxc′ e h c′ ( x , θ ) ∑k c=1 e hc ( x , θ ) ∣∣∣ θ̂ = S ( x , θ̂ ) G ( x , θ̂ ) , where S ( x , θ̂ ) = 1 1+ ∑ c 6=ĉ e ( hc ( x , θ̂ ) −hĉ ( x , θ̂ ) ) G ( x , θ̂ ) = − ∑ c 6=ĉ e ( hc ( x , θ̂ ) −hĉ ( x , θ̂ ) ) ∂ ( hĉ ( x , θ ) −hc ( x , θ ) ) ∂θ ∣∣∣ θ̂ In the last expression there are two multiplicative parts ( S and G ) . S is a scalar part , that reflects the probability of the predicted class . The G is cumulative gradient vector for all classes output nodes of the network weighted by their preference over predicted class ehc ( x , θ̂ ) /ehĉ ( x , θ̂ ) . The first part is equal to the softmax score from ODIN up to the notations used . The second part is the information from NTK space . As one can see both S and G heavily depend on class weights hc , and it is important to calibrate these values for the task the model is used for . One of the most popular ways of model calibration is temperature parameter injection ( 5 ) . In this method we scale class weights hc with the temperature factor 1T . The optimal temperature is then optimized using some calibration quality metric like Expected Calibration Error ( 5 ) for the target dataset . The solution of this optimization problem depends on the data provided , see the experimental section for details . The other way of out-of-domain detection model improvement is altering the source signal , e.g . the image in case of the image classification task . In our framework we assume that during training process the optimization pulls domain space to a stationary point in model parameters space . As only training examples are observed , we pull the entire domain using the train set as mount points . In this process the rest of the domain slacks and this cause well-known difference in gradients between training set points and validation points . To reduce this slack we can move point towards training points . We follow the alteration procedure introduced in the ODIN paper : xp = xo − ε sign ( −∇x log maxc′ e hc′ ( x , θ ) /T∑k c=1 e hc ( x , θ ) /T ∣∣∣∣∣ xo ) During this procedure , source image is altered to reduce the gradient and compensate the slack between training points and the rest of the domain . To sum it up , the proposed method is a composition of ε-input perturbation , temperature calibration by scaling outputs by 1/T factor and decomposition of ∇θ ` ( x , θ̂ ) into two multiplicative parts : S ( x , θ̂ ) and G ( x , θ̂ ) . These parts can be used as a single estimator : g ( x , θ̂ ) = ‖S ( x , θ̂ ) G ( x , θ̂ ) ‖22 , or alone : g ( x , θ̂ ) = ‖G ( x , θ̂ ) ‖22 , g ( x , θ̂ ) = S ( x , θ̂ ) . The last variant gives us exactly ODIN .

Summary: This paper proposes an anomaly detection score based on the derivative of the log-likelihood: $ \mid\mid \nabla_{\theta} \log p_{\theta}(\hat{c} | \mathbf{x}) \mid \mid_{2}^{2} $ where $\hat{c}$ is the predicted class (as we want the method to be applicable to test points).  The intuition is that if $\mathbf{x}$ is from training set, then the model should already be fairly well fit to the point, meaning that the derivative should be near zero.  This metric along with some variants derived from sub-terms are compared on image data sets against baselines such as ODIN, maximum softmax probability, maximum logit, and ensembles.