Unknown-Aware Deep Neural Network

1 Introduction . Motivation . In recent years , Convolutional Neural Networks ( CNN ) have been used with great success for a rich variety of classification problems , particularly when dealing with high dimensional , complex data such as images or time series ( Goodfellow et al. , 2016 ) . A CNN classifier ( Krizhevsky et al. , 2012 ) typically classifies test objects as one of the target classes supplied in the training set . In this , state-of-the-art classifiers make the implicit assumption that all testing objects belong to one of the target classes . However , this assumption is rarely true in real-world deployments of CNN classifiers . Consider for example , an autonomous car or healthcare system : it is extremely likely that the system will be exposed to objects that were not in its training set . We call such objects “ unknowns ” . Clearly , blindly assigning these unknowns into one of the target classes degrades the prediction accuracy . Worst yet , it can lead to serious safety concerns . For example , in a collaboration with a top hospital in the US ( name removed due to anonymity ) , we have been developing a seizure detector that classifies patients into different types of seizures based on EEG signals collected during the clinical observation of 4,000 patients . The detector was trained based on 6 types of seizures observed in the training data . However , when deployed , the CNN classifier may encounter patients who have types of seizures that do not exist in the training data because they are rare or even unknown by the medical community . Misclassifying these patients into the existing types of seizures brings serious risks and potential harm due to the potential for mistreatment of these patients . Ideally , in this case , the unknowns would be recognized and rejected by the classifier . In this work , we focus on this important problem , describing a deep neural network that not only accurately classifies test objects into known target classes , but also correctly rejects unknowns . State-of-the-Art . In a typical CNN , the output of the last fully connected layer is fed into a softmax layer to generate a class probability in [ 0 , 1 ] for each target class . An object will then be assigned to the class with the maximal probability . Intuitively , unknowns would be detected by leveraging this confidence , as was done in Bendale & Boult ( 2016 ) ; Hendrycks & Gimpel ( 2017 ) ; Liang et al . ( 2018 ) . Since unknowns should not exhibit as many features of a target class versus known objects , the CNN should report a lower confidence . In prior work ( Bendale & Boult , 2016 ; Hendrycks & Gimpel , 2017 ; Liang et al. , 2018 ) , the maximal probability or the largest value in the input vector to the softmax layer ( maximal weighted sum ) is used as a confidence to detect unknowns . In particular , an object will be rejected as an unknown if its confidence is smaller than a predetermined cutoff threshold ct . However , as shown in our experiments ( Sec . 5 ) , these state-of-the-art methods are not particularly effective at rejecting unknowns . This is because CNNs achieve high classification accuracy by providing a strong ability to generalize , allowing it to overcome the gap between the training and testing data ( Goodfellow et al. , 2016 ) . Unfortunately , this strength here is also a weakness , because it increases the chance of erroneously assigning an unknown to some target class even if it is quite different from the training objects in any target class . More specifically , the maximal probability ( or maximal weighted sum ) in a CNN is computed by the weighted sum operation on the multiple features produced by the convolutional layers . Because of this sum operation , an unknown can be classified to a target class with high confidence even if it matches some key features of a target class only by chance . Therefore , the requirements of accurately classifying the knowns and correctly rejecting the unknowns conflict with each other . Proposed Approach and Contributions . In this work we propose an Unknown-aware Deep Neural Network ( UDN for short ) to overcome this problem . The key intuition of UDN is to modify the CNN to use a product operation which models the product relationship among the features produced by the convolutional layers . This way , similar to the product rule in probability theory ( Stroock , 2010 ) , if just one feature indicative of a target class is not matched , the probability of assigning an object to this class is greatly reduced . Since an unknown is unlikely to match most of the features of a target class , the chance of assigning an unknown to a target class with high confidence is reduced . Therefore , the confidence produced by UDN should more effectively detect unknowns than the typical maximal probability/maximal weighted sum produced by classical CNNs . In UDN , the product operations are learned as a set of product relationship ( PR ) subnets leveraging the hierarchical nature of the binary tree structure . The strong bias of the classification decisions made via the product operations and the generalization ability introduced by the ensemble nature of multiple PR subsets together balance the contradictory requirements of accurately classifying known objects and correctly rejecting unknowns . In addition , we propose an information-theoretic regularization strategy that actively incorporates the objective of unknown rejection into the learning process of UDN . This further improves the accuracy of UDN at rejecting unknowns by enlarging the confidence gap between unknown and known objects . We then show that the final loss function of UDN is fully differentiable . Therefore , UDN can be learned by following the common practice of back-propagation in deep neural networks . We demonstrate the effectiveness of UDN using a rich variety of benchmark datasets including MNIST , CIFAR-10 , CIFAR-100 , and SVHN . UDN outperforms the state-of-the-art up to 20 points in the accuracy of unknown rejection – while preserving the accuracy of the underlying CNN at classifying objects from the target classes . 2 Related Work . Out-of-Distribution Detection . In Bendale & Boult ( 2016 ) , CNNs were adapted to discover unknowns by adding one additional unknown class to the softmax layer . This method , called OpenMax , measures the distance between the maximal weighted sum vector produced for a test object and the mean maximal weighted sum vectors of the target classes that this test object is most likely assigned to . This distance is translated to a probability by the OpenMax layer . A testing object will be assigned to the unknown class if this probability is larger than a pre-defined cutoff threshold . Similarly , Malinin & Gales ( 2018 ) detects out-of-distribution objects by parameterizing a prior distribution over predictive distributions , so called Prior Networks ( PNs ) , while Hendrycks & Gimpel ( 2017 ) directly uses softmax probabilities to detect out-of-distribution objects . ODIN ( Liang et al. , 2018 ) detects unknowns using a two-pass inference strategy . That is , each test image goes through the inference stage twice . During the second inference round , each input is perturbed based on the gradient of its loss acquired in the first inference . The goal is to make the maximal probability produced by softmax more effective at separating unknowns . One problem with ODIN is that it introduces two extra hyper-parameters to control the level of perturbation , which are hard to tune . As shown in experiments , our UDN method significantly outperforms OpenMax and ODIN , because UDN uses the maximal path probability as confidence measure , which as the product of multiple probabilities w.r.t . a set of nodes , is more effective in rejecting unknowns than using maximal weighted sum or maximal probability as a confidence measure . MC-Dropout ( Gal & Ghahramani , 2016 ) uses Bayesian model to reason about model uncertainty by casting dropout training in deep neural networks as as approximate Bayesian inference in deep Gaussian processes . It then rejects a testing object as unknown if the uncertainty about this object is large . As shown in experiments , MC-Dropout in many cases outperforms OpenMax and ODIN in rejecting unknowns , although is still worse than our UDN . Further , it scarifies the accuracy of classifying the known classes . In Liu et al . ( 2018 ) and Vyas et al . ( 2018 ) , methods were proposed to detect the objects that do not belong to known classes in a “ clean ” training dataset . However , these methods rely on a “ contaminated ” training set that contains a fraction of unknowns , i.e. , that were not in the original clean training data . In other words , they solve the problem of rejecting “ known unknowns ” , while our UDN instead focuses on the problem of rejecting “ unknown unknowns ” . Our UDN does not require any labeled unknowns in training . Deep Outlier Detection . Methods in Perera & Patel ( 2018 ) ; Erfani et al . ( 2016 ) ; Ruff et al . ( 2018 ) ; Nguyen & Vien ( 2018 ) ; Schlegl et al . ( 2017 ) ; Zhai et al . ( 2016 ) ; Chen et al . ( 2017 ) ; Zong et al . ( 2018 ) ; Hendrycks et al . ( 2019 ) use representation learning of deep neural networks to enhance the classical unsupervised outlier detection methods such as one-class classification ( Schölkopf et al. , 2001 ; Manevitz & Yousef , 2002 ; Tax & Duin , 2004 ) and neighbor-based methods ( Breunig et al. , 2000 ; Knorr & Ng , 1998 ; Ramaswamy et al. , 2000 ; Bay & Schwabacher , 2003 ) . However , unlike our work which focuses on enhancing the CNNs to reject the unknown objects during the inference process , these outlier detection methods do not take the classification objective into consideration . Instead , these methods detect outliers from a given dataset as the objects that significantly deviate from the majority of this dataset ( Aggarwal , 2017 ) , without using any labeled outliers or normal objects . Deep Neural Decision Forest . Similar to the deep neural decision forest ( DNDF ) ( Kontschieder et al. , 2015 ) , nodes in UDN at the final FC layers are connected to the split nodes of multiple trees . However , UDN is different from DNDF in several important ways . First , conceptually UDN corresponds to a variation of CNN model by replacing the softmax layer with tree structures such that the CNN can model the product relationship among the features learned by CNN , while DNDF instead enhances a random forest classifier with the feature learning ability of a CNN . Second , UDN incorporates the objective of unknown rejection into the learning process , while DNDF only considers the classification of known classes . Third , DNDF estimates both the decision node parametrizations θ and the leaf predictions π by using a two-step optimization strategy , where θ and π are updated alternatively to minimize the log-loss , while UDN is fully differentiable and therefore can be optimized in one step . Forth , in DNDF different trees share the FC layers ( except the final FC layer ) of the CNNs , while UDN divides all FC layers into m independent components , each of which is connected to one individual tree . This ensures independence among the trees and improves the classification accuracy because of the excellent generalization capability .

Summary: This paper is about a novel method to detect unknown samples which are of a different class than the trained ones. The idea is to use an output subnet which use a fully connected layer and a binary tree which encode the product relationship instead of the sum currently used in state of the art method (particularly the softmax with low confidence). The binary tree is made of split nodes which are responsible to produce a probability distribution from the root to each leaf. The max path i.e. the path with the largest probabilities determines the class of the input and can be used to measure how confident the classifier is about the classification decision. Combine multiple subnets and the tool obtained is able to do complex predictions and maintain a good generalization performance. The method also uses an information theory based regularization which decrease the probability of having subnets with uniform probability distribution i.e. a large entropy. Experiments on CIFAR-10 and MNIST against CIFAR-100, SVHN show that the method has an improved rejection accuracy while maintaining a good classification accuracy on the test set.