Cautious Deep Learning

1 INTRODUCTION . We consider multiclass classification with a feature space X and labels Y = { 1 , . . . , k } . Given the training data ( X1 , Y1 ) , . . . , ( Xn , Yn ) , the usual goal is to find a prediction function F̂ : X 7−→ Y with low classification error P ( Y 6= F̂ ( X ) ) where ( X , Y ) is a new observation of an input-output pair . This type of prediction produces a definite prediction even for cases that are hard to classify . In this paper we use conformal predictionVovk et al . ( 2005 ) where we estimate a set-valued function C : X 7−→ 2Y with the guarantee that P ( Y ∈ C ( X ) ) ≥ 1 − α for all distributions P . This is a distribution-free confidence guarantee . Here , 1− α is a user-specified confidence level . We note that the “ classify with a reject option ” Herbei & Wegkamp ( 2006 ) also allows set-valued predictions but does not give a confidence guarantee . The function C can sometimes output the null set . That is , C ( x ) = ∅ for some values of x . This allows us to distinguish two types of uncertainty . When C ( x ) is a large set , there are many possible labels consistent with x . But when x does not resemble the training data , we will get C ( x ) = ∅ alerting us that we have not seen examples like this so far . There are many ways to construct conformal prediction sets . Our construction is based on finding an estimate p̂ ( x|y ) of p ( x|y ) . We then find an appropriate scalar t̂y and we set C ( x ) = { y : p̂ ( x|y ) > t̂y } . The scalars are chosen so that P ( Y ∈ C ( X ) ) ≥ 1 − α . We shall see that this construction works well when there is a large number of classes as is often the case in deep learning classification problems . This guarantees that x ’ s with low probability — that is regions where we have not seen training data — get classified as ∅ . An important property of this approach is that p̂ ( x|y ) can be estimated independently for each class . Therefore , x is predicted to a given class in a standalone fashion which enables adding or removing classes without the need to retrain the whole classifier . In addition , we empirically demonstrate that the method we propose is applicable to large-scale high-dimensional data by applying it to the ImageNet ILSVRC dataset and the CelebA and IMDB-Wiki facial datasets using features obtained from state of the art convolutional neural networks . Paper Outline . In section 2 we discuss the difference between p ( y|x ) and p ( x|y ) . In section 3 we provide an example to enlighten our motivation . In section 4 we present the general framework of conformal prediction and survey relevant works in the field . In section 5 we formally present our method . In section 6 we demonstrate the performance of the proposed classifier on the ImageNet challenge dataset using state of the convolutional neural networks . In section 7 we consider the problem of gender classification from facial pictures and show that even when current classifiers fails to generalize from CelebA dataset to IMDB-Wiki dataset , the proposed classifier still provides sensible results . Section 8 contains our discussion and concluding remarks . The Appendix in the supplementary material contains some technical details . Related Work . There is an enormous literature on set-valued prediction . Here we only mention some of the most relevant references . The idea of conformal prediction originates from Vovk et al . ( 2005 ) . There is a large followup literature due to Vovk and his colleagues which we highly recommend for the interested readers . Statistical theory for conformal methods was developed in Lei ( 2014 ) ; Lei et al . ( 2013 ) ; Lei & Wasserman ( 2014 ) , and the multiclass case was studied in Sadinle et al . ( 2017 ) where the goal was to develop small prediction sets based on estimating p ( y|x ) . The authors of that paper tried to avoid outputting null sets . In this paper , we use this as a feature , similarly to Vovk et al . ( 2003 ) . Finally , we mention a related but different technique called classification with the “ reject option ” Herbei & Wegkamp ( 2006 ) . This approach permits one to sometimes refrain from providing a classification but it does not aim to give confidence guarantees . 2 p ( y|x ) VERSUS p ( x|y ) Most classifiers — including most conformal classifiers — are built by estimating p ( y|x ) . Typically one sets the predicted label of a new x to be f̂ ( x ) = arg maxy∈Y { p̂ ( y|x ) } . Since p ( y|x ) = p ( x|y ) p ( y ) /p ( x ) the prediction involves the balance between p ( y ) and p ( x|y ) . Of course , in the special case p ( y ) = 1/k for all y , we have arg maxy∈Y { p ( y|x ) } = arg maxy∈Y { p ( x|y ) } . However , for set-valued classification , p ( y|x ) can be negatively affected by p ( y ) and p ( x|y ) . Indeed , in this case there are significant advantages to using p ( x|y ) to construct the classifier . Taking p ( y ) into account ties the prediction of an observation x with the likelihood of observing that class . Since there is no restriction on the number of classes , ultimately an observation should be predicted to a class regardless of the class popularity . Normalizing by p ( x ) makes the classifier oblivious to the probability of actually observing x . When p ( x ) is extremely low ( an outlier ) , p ( y|x ) still selects the most likely label out of all tail events . In practice this may result with most of the space classified with high probability to a handful of classes almost arbitrarily despite the fact that the classifier has been presented with virtually no information in those areas of the space . This approach might be necessary if a single class has to be selected ∀x ∈ X . However , if this is not the case , then a reasonable prediction for an x with small p ( x ) is the null set . There are also conformal methods utilizing p ( y|x ) to predict a set of classes ( Sadinle et al. , 2017 ; Vovk et al. , 2003 ) . This methods does not overcome the inherent weakness within p ( y|x ) . As will be explained later on , the essence of this methods is to classify x to C ( x ) = { y | P ( y | x ) ≥ t } for some threshold t. Due to the nature of p ( y|x ) the points which are most likely to be predicted as the null set are when P ( y = j|x ) = 1k , for all classes j ∈ Y . But this is exactly the points in space for which any set valued prediction should predict all class as possible . As we shall see , conformal predictors based on p ( x|y ) can overcome all these issues . 3 MOTIVATING EXAMPLE - IRIS DATASET . The Iris flower data set is a benchmark dataset often used to demonstrate classification methods . It contains four features that were measured from three different Iris species . In this example , for visualization purposes , we only use two features : the sepal and petal lengths in cm . Figure 1 shows the decision boundaries for this problem comparing the results of ( a ) K-nearest neighbors ( KNN ) , ( b ) support vector machines with the RBF kernel ( SVM ) and ( c ) our conformal prediction method using an estimate p̂ ( x|y ) . Both the KNN and the SVM methods provide sensible boundaries between the class where there are observations . In areas with low density p ( x ) the decision boundaries are significantly different . The SVM classifies almost all of the space to a single class . The KNN creates an infinite strip bounded between two ( almost affine ) half spaces . In a hubristic manner , both methods provide very different predictions with probability near one without sound justification . The third plot shows the conformal set C ( x ) = { y : p̂ ( x|y ) > t̂y } where the t̂y is chosen as described in Section 5 . The result is a cautious prediction . If a new X falls into a region with little training data then we output ∅ . In such cases our proposed method modestly avoids providing any claim . 4 CONFORMAL PREDICTION . Let ( X1 , Y1 ) , . . . , ( Xn , Yn ) be n independent and identically distributed ( iid ) pairs of observations from a distribution P . In set-valued supervised prediction , the goal is to find a set-valued function C ( x ) such that P ( Y ∈ C ( X ) ) ≥ 1− α , ( 1 ) where ( X , Y ) denotes a new pair of observations . Conformal prediction — a method created by Vovk and collaborators Vovk et al . ( 2005 ) — provides a general approach to construct prediction sets based on the observed data without any distributional assumptions . The main idea is to construct a conformal score , which is a real-valued , permutationinvariant function ψ ( z , D ) where z = ( x , y ) and D denotes the training data . Next we form an augmented dataset D′ = { ( X1 , Y1 ) , . . . , ( Xn , Yn ) , ( Xn+1 , Yn+1 ) } where ( Xn+1 , Yn+1 ) is set equal to arbitrary values ( x , y ) . We then define Ri = ψ ( ( Xi , Yi ) , D′ ) for i = 1 , . . . , n + 1 . We test the hypothesis H0 : Y = y that the new label Y is equal to y using the p-value π ( x , y ) = 1/ ( n+ 1 ) ∑n+1 i=1 I ( Ri ≥ Rn+1 ) . Then we set C ( x ) = { y : π ( x , y ) ≥ α } . Vovk et al . ( 2005 ) proves that P ( Y ∈ C ( X ) ) ≥ 1 − α for all distributions P . There is a great flexibility in the choice of conformity score and 4.1 discusses important examples . As described above , it is computationally expensive to construct C ( x ) since we must re-compute the entire set of conformal scores for each choice of ( x , y ) . This is especially a problem in deep learning applications where training is usually expensive . One possibility for overcoming the computational burden is based on data splitting where p̂ ( x|y ) is estimated from part of the data and the conformal scores are estimated from the remaining data ; see Vovk ( 2015 ) ; Lei & Wasserman ( 2014 ) . Another approach is to construct the scores from the original data without augmentation . In this case , we no longer have the finite sample guarantee P ( Y ∈ C ( X ) ) ≥ 1−α for all distributions P , but we do get the asymptotic guarantee P ( Y ∈ C ( X ) ) ≥ 1− α− oP ( 1 ) as long as some conditions are satisfied1 . See Sadinle et al . ( 2017 ) for further discussion on this point .

Summary: The paper proposes deep learning extension of the classic paradigm of 'conformal prediction'. Conformal prediction is similar to multi-label classification, but with a statistical sound way of thresholding each (class-specific) classifier: if our confidence in the assignment of an x to a class y is smaller than \alpha, then we say 'do not know / cannot classify'). This is interesting when we expect out of distribution samples (e.g., adversarial ones).