Improving Confident-Classifiers For Out-of-distribution Detection

1 INTRODUCTION . Discriminatively trained deep neural networks have achieved state of the art results in many classification tasks such as speech recognition , image classification , and object detection . This has resulted in deployment of these models in real life applications where safety is paramount ( e.g. , autonomous driving ) . However , recent progress has shown that deep neural network ( DNN ) classifiers make overconfident predictions even when the input does not belong to any of the known classes ( Nguyen et al . ( 2015 ) ) . This follows from the design of DNN classifiers that are optimized over in-distribution data without the knowledge of OOD data . The resulting decision boundaries are typically “ unbounded/open ” as shown in Figure 1a resulting in over-generalization ( Spigler ( 2019 ) , Scheirer et al . ( 2012 ) ) . There have been many approaches proposed to address this problem under the umbrella of OOD detection1 . Lee et al . ( 2018a ) propose to explicitly train a classifier using the OOD samples generated by a GAN ( Goodfellow et al . ( 2014a ) ) . They empirically try to show that , for effective OOD detection , the generated OOD samples should follow and be close to the low-density boundaries of in-distribution , and the proposed GAN training indeed tries to do that . A multi-class softmax DNN classifier is trained with in-distribution samples to minimize the standard cross-entropy loss ( minimizing the output entropy ) and the generated OOD samples are trained to minimize a KL loss that forces the classifier ’ s predictive distribution to follow a uniform one ( maximizing the output entropy ) . The resulting classifier is called a “ confident-classifier ” . One can then classify a sample as being in or out-of distribution based on the maximum prediction probability or the entropy of the output . Sricharan & Srivastava ( 2018 ) also follow a similar approach with slight modifications . Contribution . One of the key assumptions in Lee et al . ( 2018a ) and Sricharan & Srivastava ( 2018 ) is that the effect of maximizing the entropy for OOD samples close to the low-density boundaries of in-distribution might also propagate to samples that are far away from in-distribution . This training is expected to result in “ bounded/closed ” regions in input space with lower entropy over the in-distribution , and the rest of the region ( corresponding to OOD ) , with higher entropy . The ideal 1Related work section is in the appendix due to space constraints . decision boundary in such a scenario would be as shown in Figure 1b . We find that even though such a solution exists , the proposed training algorithm is unlikely to reach it . We justify this both theoretically and experimentally for a ReLU network ( network with ReLU activation units ) that was indeed used in Lee et al . ( 2018a ) . Assuming training with OOD samples close to the in-distribution boundary , we find that having an explicit reject class for OOD samples results in a solution close to the one depicted in Figure 1b . Therefore we propose to use such a classifier instead . We give intuitive arguments to justify the proposal . This forms the first key contribution of our paper . Moreover , with toy experiments ( refer to section D in appendix ) on low-dimensional synthetic data , we analyze if GAN can indeed produce samples that can follow the low-density boundaries of indistribution . We find that , even though GAN produces samples close to the low-density boundaries of in-distribution , it is unable to cover the whole boundary , thus resulting in a sub-optimal OOD detector when trained on such samples . We therefore propose a novel algorithm to generate “ boundary ” OOD samples using a manifold learning network , ( e.g. , variational auto-encoder ( VAE ) ) and show that the generated samples are diverse and cover the in-distribution boundaries better than the method proposed in Lee et al . ( 2018a ) . The resulting classifier trained with those samples improves the OOD detection results . This forms the second key contribution of our paper . 2 BACKGROUND . Lee et al . ( 2018a ) propose a joint training of GAN and a classifier based on the following objective : min G max D min θ EPin ( x̂ , ŷ ) [ − logPθ ( y = ŷ|x̂ ) ] ︸ ︷︷ ︸ ( a ) +β EPG ( x ) [ KL ( U ( y ) ||Pθ ( y|x ) ) ] ︸ ︷︷ ︸ ( b ) + EPin ( x ) [ logD ( x ) ] + EPG ( x ) [ log ( 1−D ( x ) ) ] ︸ ︷︷ ︸ ( c ) ( 1 ) where ( b ) + ( c ) is the modified GAN loss and ( a ) + ( b ) is the classifier loss ( θ is the classifier ’ s parameter ) called the confidence loss . The difference from the regular GAN objective is the additional KL loss in ( 1 ) , which when combined with the original loss , forces the generator to generate samples in the low-density boundaries of the in-distribution ( Pin ( x ) ) space . β is a hyper-parameter that controls how close the OOD samples are to the in-distribution boundary . For the classifier , the KL loss pushes the OOD samples generated by GAN to produce a uniform distribution at the output , and therefore have higher entropy . This enables one to detect OOD samples based on the entropy or the confidence at the output of the classifier . 3 WHY MINIMIZING CONFIDENCE LOSS IS INSUFFICIENT FOR OOD DETECTION . Let f : Rd → RK be the neural network function that maps input in Rd toK output classes ( input to the softmax layer ) . Let fk : Rd → R be the function that maps the input to output for a specific class k ∈ { 1 , 2 , 3 ... K } . For a neural network with affine activations ( e.g. , ReLU and Leaky ReLU ) , each fk is a continuous piece-wise affine function over a finite set of polytopes , { Q1 , Q2 , · · · , QM } such that Rd = ⋃M l=1Ql , as described in Croce & Hein ( 2018 ) . This means that each fk is affine within each Ql ( l ∈ { 1 , 2 , 3 ... M } ) . If the input space is Rd , some of these polytopes stretch to infinity ( grow without bounds ) . Let Q∞l ≡ Ql denote these “ infinity polytopes ” . The choice of the neural network structure and the weights define fk ’ s . Figure 2a illustrates these polytopes and fk ’ s for a simple 3-class ReLU classifier , where the input space is R. In this example , there are 4 polytopes in which Q∞1 and Q ∞ 4 stretch to infinity . Hein et al . ( 2019 ) mathematically show that a ReLU classifier ( with softmax output ) produces arbitrarily high confidence predictions ( approaching 1 ) far away from the training data in almost all directions on an unbounded input space . This happens over Q∞l ’ s . Their results are summarized as follows . For any x ∈ Rd , there exists a βl > 0 such that for all αl ≥ βl , αlx ∈ Q∞l . Let f lk ( x ) = 〈vlk , x〉 + alk be the piece-wise affine function for class k over Ql . Let k∗ = arg maxk〈vlk , βlx〉2 . Then , as αl →∞ , the confidence for input αlx for class k∗ becomes arbitrarily high if k∗ is unique . i.e , lim αl→∞ efk∗ ( αlx ) ∑K l=1 e fl ( αlx ) = 1 But if there are multiple k∗ ’ s , arbitrarily large confidence values can not be obtained far away from the in-distribution in the direction of x . For instance , as shown in Figure 2b3 , for Q∞4 , k ∗ = { 2 , 3 } and therefore arbitrarily high confidence predictions can not be achieved as αl →∞ . Having multiple k∗ ’ s for every Q∞l is highly unlikely , given that we are dealing with floating point numbers and also that it is not explicitly enforced during training . Therefore , arbitrarily high confidence values far away from the in-distribution are likely inevitable . The above analysis is for the case where the input domain is unbounded ( Rd ) . For bounded domains ( for example , [ 0 , 1 ] d for images ) , as pointed out in Hein et al . ( 2019 ) , since we can not let αl →∞ , the above analysis can not be directly applied to get arbitrary high confidence values . However the above technique in principle can be applied to increase the prediction confidence for samples far away from the in-distribution . Hein et al . ( 2019 ) conduct experiments to support the claim . The theoretical analysis of which can be done as follows . Let Zd represent the bounded input domain . Similar to the unbounded case , let Q∞l denote “ infinity polytopes ” that stretch till the 2Note , k∗ = argmaxk〈vlk , αlx〉 = argmaxk〈vlk , βlx〉 , ∀αl ≥ βl . Also note , we define k * only for infinity polytopes . 3Note , for x ∈ R , k∗ = argmaxk [ slope offk ( αlx ) ] ( = argmaxk [ negative slope offk ( αlx ) ] ) as αl →∞ ( αl → −∞ ) . bounds of the input domain Zd . For any x ∈ Zd , there exists a βl > 0 such that for all αl ≥ βl , αlx ∈ Q∞l . Let f lk ( x ) = 〈v l k , x〉+alk be the piece-wise affine function for class k overQl . Let k∗ = arg maxk〈vlk , βlx〉 . Then , as αl increases , the confidence for input αlx for class k∗ keeps increasing until the bounds of the domain is reached if k∗ is unique . If fk∗ ( βlx ) > > fk ( βlx ) ∀k 6= k∗ ( ignoring the effect of bias term for simplicity ) , the confidence for input αlx for the class k∗ is very high . Therefore , even for the case of bounded input space , one can obtain confidence predictions for OOD samples high enough for it to be considered as in-distribution samples . Corollary . The higher the confidence of the output , the lower is the entropy . Hence a direct corollary of Hein et al . ( 2019 ) ’ s result is that the entropy of the classifier output for data far away from the in-distribution data in all directions would almost always be arbitrarily low ( approaching 0 ) like the in-distribution samples . This makes it almost impossible to detect OOD samples based on the confidence or the entropy of the classifier outputs . For the case of bounded input domain , as one can increase the prediction confidence for OOD samples far from the in-distribution , the entropy of classifier output also decreases making those OOD samples to be classified as in-distribution samples . Therefore , the approaches in Lee et al . ( 2018a ) and Sricharan & Srivastava ( 2018 ) would not be applicable . 4 ADDING AN EXPLICIT “ REJECT ” CLASS . When OOD samples are generated close to the in-distribution and follow its low-density boundaries as proposed in Lee et al . ( 2018a ) and Sricharan & Srivastava ( 2018 ) , we recommend adding an explicit reject class for OOD samples instead of minimizing the loss in Eq.1 ( b ) . Let the resulting classifier be called the reject-classifier . By adding an explicit reject class , our goal is to obtain a decision boundary close to the ideal decision boundary shown in Figure.1b , where the decision boundary of a K + 1 classifier divides the input space into regions such that the in-distribution region is classified as one of the first K classes and the rest of the region as the K + 1th class , i.e. , the reject class . The intuition on how such a decision boundary can be obtained is as follows . The arbitrarily high confidence predictions happen in polytopes that stretch to infinity ( or stretch till the bounds of input space in case of bounded input space ) . Each of the “ infinity polytopes ” has its own class ( or classes ) , k∗ ( or k∗ ’ s ) where high confidence predictions occur . If adding an explicit “ reject ” class results in k∗ = reject-class for all the “ infinity polytopes ” ( i.e there is only one k∗ ) , the arbitrarily high confidence predictions would only happen at the reject class for OOD samples far-off from training data . Therefore , these samples will be detected as OOD . We argue that in reject-classifier training , since we explicitly maximize the prediction confidence of K + 1th-class for boundary OOD samples , we expect the same effect to persist for OOD samples far from the in-distribution as well ( i.e. , k∗ = K + 1 ) resulting in close to ideal decision boundaries depicted in Figure.1b . This claim is supported by our experiments on a toy dataset ( Figure . 5 ) and the superior performance of the reject-classifier over the confident-classifier on MNIST and Fashion MNIST datasets ( Table . 1 ) . Note that how close the resulting decision boundary of the reject-classifier is to the ideal one depends on how well the OOD samples follow the in-distribution boundary . We find that the method proposed in Lee et al . ( 2018a ) to generate boundary OOD samples is not diverse enough as evidenced by experiments shown in the appendix ( Section . D ) . Therefore we propose a novel approach for boundary OOD sample generation which is described in the next section that results in better boundary OOD samples that cover the in-distribution boundary quite effectively . This evident from our experiments described in the appendix ( Section . D ) . Lee et al . ( 2018a ) indeed experiment with adding an explicit reject class instead of using a confidentclassifier , but the results are found to be worse . But this is because instead of using the boundary OOD samples they use another natural image dataset called “ seen OOD ” similar to Hendrycks et al . ( 2019 ) to train the classifier . However for images , it is difficult to represent the entire OOD space with a small number of samples such methods may not perform that well . Moreover as pointed out in both Lee et al . ( 2018a ) and Hendrycks et al . ( 2019 ) , as these “ seen OOD ” samples aren ’ t diverse , when used to train a reject classifier they can overfit to these training OOD samples . However we use boundary OOD samples that can guide the decision boundary of the classifier to be bounded around the in-distribution regions as depicted in Figure . 1b Note that both the reject-classifier and the confident-classifier use boundary OOD samples for training . The confident-classifier tries to equalize fk ’ s for boundary OOD samples ( i.e. , maximize the en- tropy of output predictions ) and expect this to persist over OOD samples far from the in-distribution as well ( i.e. , have multiple k∗ ’ s ) . The reject-classifier on the other hand maximizes the prediction confidence of K + 1th for boundary OOD samples and expects it to persists over OOD samples far from the in-distribution ( i.e. , have a single k∗ at k = K + 1 ) . confidence fk . In the unbounded case , for a confident-classifier , while it is proven that one can almost always find arbitrarily high confidence regions far from the in-distribution , for a reject classifier we can still expect those OOD samples to be classified as belonging to the K + 1th class . In the bounded case too as shown previously , one can obtain decreasingly low entropy regions far from the in-distribution for the confident classifier whereas for the reject-classifier it is similar to the unbounded case . As evident from the experimental results in Figure . 5 we can indeed find OOD samples with low entropy for the confident-classifiers without stretching to infinity , whereas for the reject-classifier , all those OOD samples far away from the in-distribution are correctly classified as OOD . The results in Table . 1 further reinforces the superiority of the reject-classifier over the confident classifier .

Summary: The paper proposes an algorithm to generate boundary OOD positive/negative samples to train a classifier for OOD samples. The algorithm is based on the theoretical analysis on why confidence value could be high in unbounded polytopes. CVAE is used as a generative model to get new samples. Experiments are conducted on MNIST and Fashon-MNIST datasets, which are used as OOD and in-distribution, and vice versa. Other datasets are also used for OODs. Comparison are made with Confident-Classifier, ODIN, and Mahalanobis distance-based approach, and the proposed method outperforms the others.