Fourier networks for uncertainty estimates and out-of-distribution detection

A simple method for obtaining uncertainty estimates for Neural Network classifiers ( e.g . for out-of-distribution detection ) is to use an ensemble of independently trained networks and average the softmax outputs . While this method works , its results are still very far from human performance on standard data sets . We investigate how this method works and observe three fundamental limitations : “ Unreasonable ” extrapolation , “ unreasonable ” agreement between the networks in an ensemble , and the filtering out of features that distinguish the training distribution from some out–of–distribution inputs , but do not contribute to the classification . To mitigate these problems we suggest “ large ” initializations in the first layers and changing the activation function to sin ( x ) in the last hidden layer . We show that this combines the out-of-distribution behavior from nearest neighbor methods with the generalization capabilities of neural networks , and achieves greatly improved out-of- distribution detection on standard data sets ( MNIST/fashionMNIST/notMNIST , SVHN/CIFAR10 ) . 1 INTRODUCTION . When a neural network classifies inputs , we often need to have some measure of the uncertainty involved in the prediction . In particular , in safety critical applications we would like to know when the classifier received an input that is different from the inputs it has been trained on - in such cases it could be dangerous to just use the “ best guess ” , instead we may want to choose some safe action or involve a human . One solution for this “ out-of-distribution detection ” problem could be to add a label “ unknown ” and augment the training set with inputs that do not belong to any of the labels of the classifier and which get the new label “ unknown ” . However , the danger is that the classifier learns the particular type of “ known unknowns ” it was trained on and does not generalize to the “ unknown unknowns ” . To avoid this , we treat out-of-distribution detection as a “ one class classification ” problem , i.e . we want to learn the input distribution without help of out of distribution inputs . The simplest approach is to use the softmax output of a neural net as a confidence measure . It is well known that the softmax output tends to be “ overconfident ” , so we can not interpret it directly as a probability for the chosen class . However , it still tends to be more confident for samples from the correct distribution than for outliers , so setting a threshold for the softmax output can to some extent distinguish between “ in distribution ” and “ out-of-distribution ” samples . This is the “ baseline method ” for outlier detection suggested in ( Hendrycks & Gimpel , 2017 ) . In ( Lakshminarayanan et al. , 2017 ) this baseline is improved by using an ensemble of classifiers ( and adversarial training ) : Often each random initialization of a neural network gives an overconfident classifier , but different classifiers disagree - averaging the softmax output over an ensemble of classifiers then gives an improved signal . Other , more computationally demanding approaches are Bayesian Neural Networks ( ( Neal , 1996 ) , ( Barber & Bishop , 1998 ) , ( Blundell et al. , 2015 ) ) , nearest neighbor methods ( e.g . ( Mandelbaum & Weinshall , 2017 ) , ( Jiang et al. , 2018 ) , ( Papernot & McDaniel , 2018 ) , ( Frosst et al. , 2019 ) ) . For some further ( less closely ) related works , see appendix A . We investigate limitations of the “ ensemble averaged softmax output ” method , as this seems to hit the sweet spot of being easy and fast , but still giving good results . To overcome the observed limitations and further improve this method , we propose two changes : • Changing the activation function of the last hidden layer to the sin ( x ) function . ( We will see in the next section how this can guard against overgeneralization . ) • Use larger than usual initialization , to increase the chances of obtaining more diverse networks for an ensemble . In the next three sections we go through three observed problems with ensembles of ReLU networks and explain how these changes mitigate them , and in the following sections we evaluate the suggested changes on standard data sets ( MNIST , CIFAR10 , etc. ) . We evaluate this uncertainty mainly with view to out-of-distribution detection : If we have meaningful uncertainties , the classifier should in particular “ know when it does not know the answer ” . This means it should display uncertainty for inputs that do not “ belong to the same type of input ” as the training examples . By selecting different thresholds for the uncertainty , we can plot the ROC curve by recording for each threshold the fraction of samples below the threshold for “ in distribution ” ( x ) and for “ out-of-distribution ” ( y ) inputs . In particular , figure 1 gives this evaluation for our method against the standard ensemble methods on two standard data sets . x-axis : MNIST samples recognized as not belonging to MNIST , y-axis : notMNIST samples recognized as not belonging to MNIST . 2 PROBLEM : “ UNREASONABLE ” EXTRAPOLATION . We first look at the simplest possible example of classification : The training set consists only of one point x0 ∈ Rd which has label “ 1 ” ( we assume there is another label , for which we do not have a sample ) . Training standard ReLU networks with one hidden layer on just one point x0 = 2 ∈ R we get something like figure 1 : Gradient descent finds a simple functions describing the data , which will ( with very high probability ) be an increasing function of x . So for larger values of x the network will become arbitrarily certain that the label of x should also be “ 1 ” . ( See appendix C for more details . ) It has been observed in ( Hein et al. , 2018 ) that this is true in general : Given any ReLU network trained on some data for a classification problem in Rd , it will describe a piecewise linear function . If we look at the values of this function along a ray to infinity , this will be a linear function when we are far enough from the origin , and in general this means the network will become arbitrarily certain of its classification for all points on the ray far enough from the origin . This is clearly not desirable , we would rather get a low confidence when we are far from the training data . ( Here “ far from the training data ” is a convenient aspect that can easily be checked mathematically . Since the data are often bounded , it is actually more relevant in practice that this behavior starts as we leave the range of the training data . As an illustration , see figures 3 and 4 . ) Going back to our simplest possible special case , what should our confidence be that other points belong to the same label ? Of course , there is no unique “ correct ” answer , but a reasonable model could be that our confidence decays like a Gauss function around this point ~x0 , i.e . is proportional to e−|~x−~x0| 2/2σ2 ( 1 ) for some σ that defines the scale on which our confidence decays . One way to force such a decay would be to use directly Gauss functions ( equation 1 ) as activation functions , which leads to RBF networks which we could consider as a form of nearest neighbor methods . While they recognize reliably points that are not close to one of the ~x0 in ( equation 1 ) , they do not generalize as well as ReLU networks ( see appendix G ) , so we will use a more indirect approach . Using that the Fourier transform of such a Gauss function is again of this shape , we can write equation 1 also as an expectation : e−|~x−~x0| 2/2σ2 = E~w∼N ( 0 , Σ ) [ cos ( ~w · ( ~x− ~x0 ) ) ] , where Σ is the diagonal covariance matrix σ−2 · I . When we approximate the expectation by a finite sum over N samples ~wi from N ( 0 , Σ ) we get an expression of the form f ( x ) = N∑ i=1 ui · cos ( ~wi · ~x+ bi ) where ui = 1/N and the bi are chosen such that the cos ( ~wi · ~x0 + bi ) = 1 . This function can be interpreted as the output of a neural net with one hidden layer of N neurons with activation function cos ( x ) ( we will later shift the phase bi and use sin ( x ) ) . ( The output is one dimensional since we only care about the logits for the one label that appears in our training set . ) This is the main idea in the proof of the following result ( see appendix C for details and proof ) : Proposition 1 : Assume the weight vectors ~wi are sampled from a normal distribution N ( 0 , Σ ) and then fixed , the numbers bi and ui are sampled independently from some distribution ( not identically zero ) with finite second moments ( e.g . a normal distribution ) and then trained on one data point ~x0 with label 1 , then Gradient Descent ( for the usual cross-entropy loss ) will make the network ’ s output converge to a function that is approximately proportional to e−~x T Σ~x/2 . This approximation becomes exact when the number of neurons approaches infinity , or when we take the expected value . We also have for general training sets ( see appendix C for a proof ) : Proposition 2 : Under the same idealized conditions ( infinite number of neurons , weights ~wi frozen ) , but arbitrary finite training set , and assuming the training converged , the function goes to 0 for |~x| → ∞ . To get an exact formula , we froze the weights ~wi . In practice we would train them as well , but at least experimentally we do not see a difference when the ~wi are large : They do not get changed by a large amount , and the result still looks the same . An example is given in the two plots of the right side of figure 2 . When we train on a training set of a few points , we see experimentally that ( under the same idealized conditions : Large number of hidden neurons , frozen ~wi ) we get an approximation to a weighted sum of Gaussian functions at these points ( see appendix E ) . So it may seem we constructed a complicated approximation to a nearest neighbor classifier . We now switch the activation function to sin ( x ) , this makes no difference for the function if we compensate by shifting the bi by π/2 , but the region of “ small initialization ” |~wi| , bi ≈ 0 is now similar to a linear network : The input to the hidden neurons i will be ~wi · ~x ≈ 0 , and sin ( ~wi · ~x ) ≈ ~wi · ~x . As training progresses , |x| usually increases . For ReLU activation , this increase is not bounded ( if we are on the side ~wi · ~x > 0 ) , but for sin the increase will usually stop at the first maximum or minimum of sin , i.e . around ≈ ±π/2 ( or also at a different minimum/maximum if we start from higher initialization , see appendix P for experimental examples ) . This means that on the training set , features ( i.e . the output of the sin ( x ) neurons in the last hidden layer ) get a large value by tuning the input to around x ≈ ±π/2 , which is difficult to achieve “ accidentally ” for input that is not in the training set . By contrast , for ReLU ( or tanh ) activation , the inputs only have to achieve any high value . We will call networks in which the last hidden layer uses the sin ( x ) activation function “ Fourier networks ” . So the Fourier network is somewhere between a ReLU network ( small initialization ) and a nearest neighbor classifier ( large initialization ) . In particular for intermediate initializations it generalizes better than a nearest neighbor classifier , but still detects more outliers than a ReLU network , which leads to an out-of-distribution detection which is better than either of them , as we will see later ( see in particular appendix G for a comparison to nearest neighbor classifiers ) .

Summary: The paper presents a method to detect out-of-distribution or anomalous data points. They argue that Fourier networks have lower confidence and thus better estimates of uncertainty in areas far away from training data. They also argue for using “large” initializations in the first layers and sin(x) as the activation function for the final hidden layer.