The Compact Support Neural Network
Authors: Adrian Barbu
Hongyu Mou
Venue:  (2021)
Keywords: RBF network
OOD detection
overconfident neural networks

Abstract:  Neural networks are popular and useful in many fields, but they have the problem of giving high confidence responses for examples that are away from the training data. This results in the neural networks being very confident while making gross mistakes, thus limiting their reliability for safety critical applications such as autonomous driving, space exploration, etc.
In this paper, we present a more generic neuron formulation that contains the standard dot-product based neuron and the RBF neuron as two extreme cases of a shape parameter. Using ReLU as the activation function we obtain a novel neuron that  compact support, which means its output is zero outside a bounded domain. We also show how to avoid difficulties in training a neural network with such neurons, by starting with a trained standard neural network and gradually increasing the shape parameter to the desired value. 
Through experiments on standard benchmark datasets, we show the promise of the proposed approach, in that it can have good prediction on in-distribution samples, and it can consistently detect and have low confidence on out of distribution samples.

Under review as a conference paper at ICLR 2021
THE COMPACT SUPPORT NEURAL NETWORK
A Anonymous authors
Paper under double-blind review
ABSTRACT
Neural networks are popular and useful in many fields, but they have the problem of giving high confidence responses for examples that are away from the training data. This makes the neural networks very confident their prediction while making gross mistakes, thus limiting their reliability for safety critical applications such as autonomous driving, space exploration, etc. In this paper, we present a neuron generalization that has the standard dot- -oo-product based neuron and the RBF neuron as two extreme cases of a shape parameter. Using ReLU as the activation function we obtain novel neuron that compact support, which means its output is zero outside a bounded domain. We show how to avoid difficulties in training a neural network with such neurons, by starting with trained standard neural network and gradually increasing the shape parameter to the desired value. Through experiments on standard benchmark datasets, we show the promise of the proposed approach, in that it can have good prediction on in-distribution samples, while being able to consistently detect and have low confidence on out of distribution samples.
INTRODUCTION
Neural networks have been proven to be extremely useful in all sorts of applications, including object detection, speech and handwriting recognition, medical imaging, etc. They have become the state of the art in these applications, and in some cases they even surpass human performance. However, neural networks have been observed to have a major disadvantage: they don't 't know when they don't 't know, i.e. don't know when the input is far away from the type of data they have been trained on. Instead of saying "I don't 't know", they give some output with high confidence (Goodfellow et al., 2015 Nguyen et al.] 2015). An explanation of why this is happening for ReLUU based networks has been given in Hein al. (2019). This issue is very important for safety-critics as space exploration, autonomous driving, medical diagnosis, etc. In these cases is important that the system know when the input data is outside its nominal range, to alert the human (e.g. driver for autonomous driving or radiologist for medical diagnostic) to take charge in such cases.
applications such
In this paper suspect that the root of this problem is actually the neuron design, and propose different type of neuron to address what we think are its issues. The standard neuron can be written as (xx a (ww" x + b), which a direction w. followed by a nonlinearity (-)). In this design, the neuron has a large response for vectors x E E2" that are half-space. This can be an advantage when training the NN since it creates high connectivity in the weight space and makes the neurons sensitive to far-away signals. However, is a disadvantage when using the trained NN, since can lead to the neurons unpredictably firing with high responses to far-away signals, which can result (with some probability) in high confidence responses of the whole network for examples that are far away from the training data.
be regarded
projection (dot product) - w x + b onto
To address these problems, we use type of radial basis function neuron (Broomhead & Lowe 1988 f(x = g[|[ - f=ll),, which we modify to have high response only for examples that are close to μu. and to have zero response at distance at least R from μ.. Therefore the neuron has compact support, and the same applies to a layer formed entirely of such neurons. Using one such compact guarantee that the space where the NN has non-zero response is bounded, obtaining a more reliable neural network.
support layer before the output layer
In this formulation, the parameter vector μ is directly comparable the neuron inputs x, thus μ has a simple and direct interpretation as a "template". A layer consisting of such neurons forms can be interpreted sparse coordinate system the manifold containing the inputs of that layer.

---

Under review as a conference paper at ICLR 2021
Because of the compact support, the loss function of such compact support NN has many flat areas and it can be difficult to training it directly by backpropagation. However, we will show how to train such NN, by starting with trained regular NN and gradually bending the neuron decision boundaries to make them have smaller and smaller support.
The contributions of this paper are the following:
We introduce type of neuron formulation that generalizes the standard neuron and the RBF neuron as two extreme cases of a shape parameter. Moreover one can smoothly transition from a regular neuron to RBF neuron by gradually changing this parameter. We introduce the RBF correspondent to a ReLU neuron and observe that it has compact support, i.c. its output is zero outside a bounded domain.
The above construction allows us to smoothly bend the decision boundary of a standard ReLU based neuron, obtaining compact support neuron. We use this idea to train compact support neural network (CSNN) starting from a pre-trained regular neural network. We show through experiments on standard datasets that the proposed CSNN can achieve comparable test errors with regular CNNs, and the same time it can detect and have low confidence on out-of-distribution data.
1.1 RELATED WORK
common way to address the problem of high confidence predictions for out of distribution (OOD) examples is through ensembles (Lakshminarayanan et al.) 2017), where multiple neural networks trained with different random initializations and their outputs are averaged in some way. The reason why ensemble methods have low confidence OOD samples is that the high-confidence domain of each NN is random outside the training data, and the common high-confidence domain is therefore shrunk by the averaging process. This reasoning works well when the representation space (the space of the NN before the output layer) is high dimensional, but it fails when this space is low dimensional (see van Amersfoort al. (2020) for example).
Another popular approach is adversarial training (Madry et al. 2018). where the training set is augmented with adversarial examples generated by maximizing the loss starting from slightly perturbed examples. This method is modified in adversarial confidence enhanced training (ACET) (Hein et al. 2019) where the adversarial samples are added through a hybrid loss function. However, we believe that training with out of distribution samples could be a computationally expensive not hopeless endeavor, since the instance space extremely vast when it is high dimensional. Consequently, a finite number of training examples can only cover insignificant part of it and no matter how many -fi-oii-iiriuttionn examples are used, there always will be other parts of the instance space that have not been explored. Other methods include the estimation of the uncertainty using dropout (Gal & Ghahramani] (016), softmax calibration (Guo et al. 2017), 7), and the detection of out-of dit-of--disributiin inputs (Hendryeks & Gimpell (007). CutMix Yun al. 2019) is a method to generate training samples with larger variability, which help improve generalization and OOD detection. All these methods are complementary to our approach and could be used together with our classifiers to improve accuracy and OOD detection.
In Ren et al. (2019) trained two auto-regressive models, one for the foreground in-distribution data and one for the background, and the likelihood ratio is used to decide for each observation whether it OOD or not. This is a generative model, while our model is discriminative.
A number of works assume that the distance in the representation space (the space of outputs of the last layer before the final classification layer) is meaningful. They will be reviewed next.
Recently, Jiang et al. (2018) proposed trust score that measures the agreement between given classifier and modified version of a k-nearest neighbor classifier. While this approach does consider the distance of the test samples to the training set, it only does so to a certain extent since the A--N does not have concept of "too far", and is also computationally expensive.
A simple method based on the Mahalanobis distance is presented in Lee et al. (0188). It assumes that the observations are normally distributed in the representation space, with a shared covariance matrix for all classes. While we also assume that the distance in the representation space is meaningful, we make a much weaker assumption that the observations for each class are clustered in a number of clusters, not necessarily Gaussian. In our representation, each class is usually covered by more than one compact support neuron, and each neuron could be involved in multiple classes. Furthermore, the method in Lee et al. (2018) simply replaces the last layer of the NN with their Mahalanobis measure
2

---

Under review as a conference paper at ICLR 2021
0=
==0.5
0=0.8
o=1
-10
-3
10
Figure The construction
(3) smoothly interpolates between a standard neuron (a 0) and an
RBF-type of neuron (a 1). Shown are the neuron decision boundaries for various values of a.
and makes attempt further train the new model, while train our layers together with the whole network.
The Generalized ODIN Hsu et al. (2020) decomposes the output prediction into the ratio of a class- specific function (x) and a common denominator g(z), both defined over instances x from the representation space. Good results are obtained using based on the Euclidean distance the cosine similarity. Again, this approach assumes that the observations are grouped in a single cluster for each class, which explains it uses very deep models (with 34- 100 layers) that are more capable to obtain representations where this assumption is satisfied. Our method does not make the single cluster per class assumption, and can use deep or shallow models.
The Deterministic Uncertainty Quantification (DUQ) (van Amersfoort et al. 2020) method uses an RBF network and special gradient penalty decrease the prediction confidence away from the training examples. The authors also propose a centroid updating scheme to handle the difficulties in training an RBF network. In contrast, our paper proposes a generalized neuron model that has the RBF neurons and the standard neurons two extreme cases, and trains all models starting from standard NN where the local minima are more well behaved.
2
THE COMPACT SUPPORT NEURAL NETWORK
The compact support neural network consists of number of layers, where the last layer before the output layer contains only compact support neurons, which will be described next. The other layers could be regular neural network or convolutional neural network layers, or compact support layers. The final output layer is regular linear layer without a bias term, so that it can output vector of all zeros when appropriate.
2.1 THE COMPACT SUPPORT NEURON
We start with the radial basis function (RBF) neuron (Broomhead & Lowe 1988).
fw (x) g[|xx - 3
The RBF neuron has g(u) exp(-9u) the activation function, but in this paper we will use g(u) = max R² - 0) because related to the ReLUU.
flexible representation. We can introduce an extra parameter a = 1 and rewrite eq. as
fx) g(x"xxxxxxx x + w'w 2w 2w g(a([[x|) 2 + ||w (2) 2w 2wTx) x).
(2)
Using the parameter a, obtain representation that smoothly changes between an RBF neuron when a and a standard projection neuron when a = 0. However, starting with an REF neuron with g(u) exp( (B)), obtain the projection neuron for 0 fw (x) exp(2w T)) which has an exponential activation function.
The compact support neuron. We want to obtain a standard ReLU based neuron fw (x) a (w"
with (u) max(u, 0) for 0. For this purpose will use g(u) = oR u). and modify the above construction to obtain the compact support neuron:
fw (x) (R² xxx wTw 2w x) = cla(RR (R² |x|| ||w]
b) + 2w + b), (3)
3

---

Under review as a conference paper at ICLR 2021
a R
0,R
7x7 conv
3x3 conv
ave. pool
CSN
V
FC
1
64
512
Norm.
CSN
FC
CSNN
ResNet
CSNN
Figure 2: Left: Diagram of the compact support neural network (CSNN), with the CSN layer described in Eq. (6). Right: example of the CSNN with normalized input from ResNet. Only the full arrows have backpropagation.
where we also introduced a bias term for the standard neuron. We usually make b = 0 for simplicity, The parameter R defines the radius of the support of the neuron when a
One can easily check that the support of (x) from eq. B (i.e. the domain where it takes nonzero values) sphere radius
R = b(1/a-111110 - 1) + ||/| -)
(4)
centered at w. w/a. Therefore the neuron from eq. (3 has compact support for any a > 0 and the larger the value of a, the smaller the support of the neuron will be. In Figure Dis shown the support for several values of a & [0, 1] of the neuron (3) with w = (0, 2) 0 and
1.
Convolutional version. If one desires to make a compact support convolutional neuron, let w be its k k matrix of weights. Then the convolutional version can be obtained by taking into consideration that each & X patch of an image I is a candidate x in eq. Therefore one can easily check that the convolutional compact support neuron should be:
(w(() = oorr b 1 +
|/1l)) + w
(5)
where 1 is a ls x matrix of ones, I is done elementwise and * is the convolution.
2.2 THE COMPACT SUPPORT NEURAL NETWORK
If we have a layer containing only compact support neurons (CSN), combining the weights into matrix WT (W1,.., WK) and the biases into a vector b (bi,... bx), we can write the CSN layer as:
fw fw(x) = aa - x'x T(WW 2Wx b).
where fw (x) = enables the use of standard neural network machinery (e.g. PyTorch) train CSN. In practice will have no bias term (i.e. b = 0), except in low dimensional experiments.
fK (x)) is the vector of neuron outputs of that layer. This formulation
The simplest compact support neural network (CSNN) has layers: hidden layer containing compact support neurons (3) or their convolutional counterparts (5), and an output layer which is standard fully connected layer without bias. It illustrated in Figure left.
Normalization. For best results, all variables of the input data x should be on the same scale. For better control, it also preferable that |X||| be approximately I on the training examples. These goals can be achieved by standardizing the variables to have zero mean and standard deviation 1 1/vd on the training examples (where d is the dimension of x). This way | ~ 1 when the dimension d is large (under assumptions normality and independence of the variables of x). Our experiments on three datasets indicate that indeed ||x| 1 on real data when the inputs are normalized described
above, as exemplified by the histograms of [ from Figure
MNIST
CIFAR-10
CIFAR-100
Figure 3: Histogram of the norms |/ of the normalized input features v, to the CSN layer for the three datasets trained in our experiments.
4

---

Under review as a conference paper at ICLR 2021
Training, Like the RBF network, training neural network with such neurons with 1 is difficult because the loss function has many local optima. To make matters even worse, the compact support neurons have small support when a is close 1, and consequently the loss function has flat regions between the local minima.
This is why take another approach to training. Using equations (6 or (5)
train CSNN by first training a regular NN (a = 0) and then gradually increasing the shape parameter a from 0 towards 1 while continuing update the NN parameters. Observe that whenever a 0 the NN has compact support, but the support gets smaller as a gets closer to 1. The training procedure is described in detail in Algorithm I
Algorithm 1 Compact Support Neural Network (CSNN) Training
Input: Training set T' = {(x; 21) IPP x R]
Output: Trained CSNN.
1: Train regular CNN f(x) L2 g(x + b) where W, L are the last two layer weight matrices and g(x) is the rest of the CNN.
2: Freeze g(x), compute u, g(x,), i L, 1.,., n, their mean μ and standard deviation g.
3: Obtain normalized versions v, u, as V. = (u; - 10)--/)/a /o = 1,,., n.
4: for e= to Nepochs do
5: Set a = e/Nepochs
6: Use the examples (v., v.) to update (W, L, b) based on epoch of
f(v) (a vTv - Tr(WWTT) b] 2Wv + b)
7: (optional) Remove any neurons w, of W = -(w1, Wx) that are dead, i.e. satisfy:
(aaR2 - l|v: ||w,]l?) by] + 2w v; + b,) = 0, 1,...n
8: end for
In the synthetic experiment in Figure we succeeded to bring the train and test errors close to 0 for using a carefully crafted schedule for increasing the a. However, in the real data applications. the training, test and validation errors might first decrease a little bit but ultimately increase as a approaches For example one could see the test errors vs a for the synthetic dataset in Figure and for the real datasets in Figure For this reason, in practice we stop the training at an < where the training and validation errors still take acceptable values, e.g. a validation error less than the validation еrror for = O. However, we noticed that the larger the value of the tighter the support around the training data and the better the generalization.
It is worth noting that in contrast to the weights of standard neuron, the weights of the compact support neuron cxist in the same space as the neuron inputs and they can be regarded as templates. Thus they have more meaning, and one could easily visualize the type of responses that make them maximal, using standard neuron visualization techniques such as Zeiler & Fergus (014)). Furthermore, can also obtain samples from the compact support neurons, e.g. for generative GAN models.
3 EXPERIMENTS
In this section we first present an experiment on 2D data to showcase what can be achieved with the proposed compact support neural network, and then experiments on real datasets show the power of the CSNN to model real data and how it can detect out-of cut-of-distribution samples.
3.1 2D EXAMPLE
We present first experiment with the moons 2D dataset, where the data organized on two intertwining half-cirele like shapes, containing the positives and one the negatives. The data is scaled so that all observations are in the interval [0, 1 (shown as a white rectangle in Figure 4 As out of distribution data (OOD) started with 100 x 100 10000 samples a grid spanning [-0.5, 1. and we removed all samples at distance at most 0.1 from the moons data, obtaining 8763 samples.
We used a two layer CSNN, with the first layer having 128 CSNN neurons, and the second layer being standard NN layer without bias, as illustrated in Figure 2 left. The second layer is used integrate
5

---

Under review as a conference paper at ICLR 2021
-2-
2
150
1.0
150
150
125
125
125
830
030
820
000
000
000
-075
-1%
-126
-0.5
0
0.6
0.94
Qr
Figure 4: The confidence map (0.5 for white and for black) of the trained CSNN on the moons dataset for different values of a E [0, 1]. Top: zoom out on the interval -5, 6]2. Bottom: zoom in view of the interval [-0.5, 1.5]
activation patterns a = 0 activation patterns a = 0.825 confidence map & 0.825
Figure Example of activation pattern domains for a = and a = 0.825 and the resulting confidence map (0.5 for white and 1 for black) for 0.825 for 32 neuron 2-layer CSNN.
the evidence from the CSNN neurons into the class prediction. We used 200 training examples and trained the CSNN using Algorithm 1 We trained 2000 epochs with R2 decreasing linearly from 0.04 to 0.01, and increasing from 0 to as a; min(1, 1, max(0, (10.1 1.5)) 1,..., 2000. This way a increases slower as gets closer to Using this special training we avoided the training and test errors blowing up when gets close 1. As specified in line of Algorithm [ the NN nodes that had zero response on all training examples were eliminated. These neurons cannot be trained anymore and only give uncontrolled responses unseen data. This way from the 128 neurons, only 73 were left at the end of training.
The training/test errors and the AUROC and NZ confidence measures for the OOD data descrihed above vs. a are shown in Figure Observe that the training and test errors for 0 are quite large, because the standard NN with 128 neurons cannot fit the data well enough, and they decrease as the neuron support decreases and the model is better capable to fit the data.
2nd enic Bal entr AUROC
The confidence map for the obtained classifier is shown in Figure We can see that the confidence is 0.5 (white) almost everywhere except close to the training data, where it is close insight that the method works as expected, shrinking the support the neurons to a small domain around the training data. We also that the support already reasonably small for a 0.6 and it gets tighter and tighter gets closer to 1.
to 1 (black). This gives
Figure 6: CSNN train and test er- rors, AUROC and percent nonzero outputs (NZ) vs. for the moons data.

---

Under review as a conference paper at ICLR 2021
It is known Croce & Hein (018)) Hein al. (2019) that the output of a ReLU-based neural network is piecewise linear and the domains of linearity are given by the activation pattern of the neurons. The activation pattern of the neurons consists of the domains where the set of neurons that are active (i.e. their output is positive) does not change. These activation pattern domains are polytopes, as shown in in Figure 5 left, for two-layer NN with 32 neurons. The activation domains for CSNN are intersections of circles, as illustrated in Figure [] middle, with the domain where all neurons are inactive shown in white. The corresponding confidence map is shown in Figure [ right.
In real data applications we don't need to go all the way to a 1 since even for smaller a the support is still bounded and if the instance space is high dimensional (c.g. 512 to 1024 in the real data experiments below), the volume of the support of the CNN will be very small compared to the instance space, making it unlikely to have high confidence on distiiiiiiiitttion data.
1.0
125
0.5
025
0.00
The role of pruning dead neurons. Due to the random initialization of the neurons, there might exist neurons that have zero response on all the training observations. These neurons are dead in the sense that they are not updated in the back-propagation since their response is always zero. We have observed that in some cases these dead neurons will produce some small high confidence regions far away from the training examples (see Fig. 77 This problem can be eliminated by removing these neurons during training, which is done by line of Algorithm! !
-0.50
Figure 7: Confidence map
without pruning, a = 0.985.
a,R
5x5 conv 32
max pool
5x5 conv
max pool
++
64
BN
CSN
FC
y
Figure 8: The CSNN-F with LeNet backbone, where all layers are trained by backpropagation.
3.2 REAL DATA EXPERIMENTS
We conduct experiments by training three different datasets: MNIST (LeCun & Cortes) 2010).
CIFAR-10 and CIFAR-100 (Krizhevsky et al. 2009). We evaluate the confidence on in-sample and out-of-sample data, by testing them on their respective test sets (in-sample) and on other datasets as shown in Table I including the test sets of EMNIST (Cohen et al.), FashionMNIST (Xiao et al.
2017) and SVHN (Netzer et al. 201T). and the validation set of ImageNet (Deng et al.) 2009). For
MNIST we also tested on a grayscale version of 100 obtained by converting the 10,000 test images to gray-scale and resizing them to 28 x 28.
CNN architecture. For MNIST we use a layer LeNet CNN as backbone, with two 5 convolution layers with 32 and 64 filters respectively, followed by ReLU and 2 x max pooling, and two fully connected layers with 256 and 10 neurons. For the other two datasets, we used as backbone
ResNet-18 18 architecture (He et al.) (016) with 4 residual blocks with 64, 128, 256 and 512 filters respectively. After the backbone CNN has been trained, the FC layers were removed and only the convolutional layers were kept, as illustrated in Figure 2 right and Figure
For the CSNN we will experiment with two architectures. illustrated in Figure and Figure The first is a small one (called CSNN) that takes as input the output of the last convolutional layer of the backbone, normalized as described in Section 2.2 The normalization of the CSNN input can also be achieved using a batch normalization layer without any learnable affine parameters. The second one a full network (called CSNN-F), illustrated in Figure g where the backbone (LeNet or ResNet) is part of the backpropagation and a batch normalization layer (BN) without any learnable parameters has been introduced between the backbone and the CSN layer.
Training details. For all datasets we used data augmentation with padding (3 pixels for MNIST, 4 pixels for the rest) and random cropping to train the backbones. For CIFAR- 100 we also used random rotation up 15 degrees. We used no data augmentation when training the CSNN and CSNN-F.
The training/test data was passed though the backbone without the FC layers, and the output was normalized. A CSNN without bias term trained for 510 epochs with R 0.1, of which 10
7

---

Under review as a conference paper at ICLR 2021
EES SITOF
To HITT
Extr anpT
AUROC CIFAR10
AUROO OIFARIOS
AJROG CIARRO
527 CIFARID
CIFARTOO
KZ CIFARRO
33
33
3003
MNIST
CIFAR-10
CIFAR-100
Figure 9: Train and test errors, Area under ROC Curve (AUROC) and percent nonzero outputs (NZ) vs a for CSNN classifiers trained on three real datasets. These results are obtained from one training run.
epochs at = O. For the CSNN training we used the Adam optimizer with learning rate 0,001 and weight decay 0.0001. We also tried SGD and obtained similar results. The CSNN-F was trained with SGD with a learning rate of 0.001 and weight decay 0.0005. Its layers were initialized with the trained backbone and the trained CSNN. Then a was kept fixed for two epochs and increased by 0.005 every epoch for 4 more epochs.
Training the CSNN from a to a = for 510 epochs takes less than an hour on MSI GS-60 Core 17 laptop with 16Gb RAM and Nvidia GTX 970M GPU. Each epoch of the CSNN-F took less than a minute with the LeNet backbone and about 3 minutes with the ResNet-18 backbone.
OOD detection. The out of distribution (OOD) detection is performed similarly to the way it is done in a standard CNN. For any observation, the maximum value of CSNN raw outputs is used as the OOD score for predicting whether the observation is OOD not. If the observation is in in-distribution, its score will usually be large, and if it is OOD, it will be usually close to zero or even zero. The ROC curve based on these scores for the test set of the in-distribution data (as class 0) and one OOD) dataset (as class 1) will give us the AUROC. If the two distributions are not separable (have concept overlap), some of the OOD scores will be large, but for the OOD observations that are away from the area of overlap they will be small or even zero.
In Figure are shown the train/test errors vs for the CSNN on the three datasets. Also shown are the Area under the ROC curve (AUROC) for OOD detection on CIFAR- 10 or CIFAR- 100 and the percentage of OOD samples with nonzero outputs (NZ). Observe that all curves on the real data are very smooth, even though they are obtained from one run, not averaged. We see that the training and test errors stay flat for a while then they start increasing from a certain a that depends on the dataset. At the same time, the AUROC stays flat and slightly increases, and there is range of values of a where the test error is low and the AUROC is large. Looking in more detail at the CIFAR- 10 dataset (middle plot in Figure D. we see that for 0.7 the NZ-CIFAR 100 is about 0.6, which means that about 40% of the CIFAR- 100 observations have all 0 CSNN outputs, therefore an OOD score of O. Combined with the scores of the other observations and the fact that at most 10% of the CIFAR- 10 test observations have an OOD score of O (because the test error is less than 0.1) it results in a slight increase in AUROC
In practice, a should be chosen as large as possible where an acceptable validation error is still obtained, to have the smallest support possible. For example one could choose the largest a such that the validation error at a is less then or equal to the validation error at a = O. However, for better comparison with the other methods, for each dataset we chose the CSNN classifier corresponding to the largest a where the test eтror takes a value comparable to the other methods compared, and reported the AUROC values in Table [ The CSNN-F was obtained by merging the corresponding CSNN head with the ResNet or LeNet backbone and training them together for 6 epochs.
Methods compared. We compare our results with the Adversarial Confidence Enhanced Training (ACET) (Hein et al., 2019), Deterministic Uncertainty Quantification (DUQ) (van Amersfoort ctt al. 2020), a standard CNN, and an ensemble of five or 10 CNNs trained with different random initializations. The ACET results are taken directly from Hein et al. 2211), and the DUQ, CNN and ensemble results were obtained using the DUQ authors' code. For DUQ we trained multiple models with various combinations of the length scales {0.05, 0.1, 0.2. 0.3, 0.5, 1.0} and gradient penalty E {0, 0.05, 0.1, 0.2, O.3, 0.5, 1.0} and selected the combination with the best test error-AU AUROC trade-off.
8

---

Under review as a conference paper at ICLR 2021
CNN
ACET
DUQ
Ens
10 Ens
CSNN
CSNN-F
Train on MNIST
0.53% (.05)
0.66
0.57 (.04) 0.50 (.02)
0.51 (.03)
0.52 (.01)
0.50 (0.02)
EMNIST
0.983 (.001)
0.912
0.988 (.001) 0.985 (.001)
0.985 (_001)
0.992 (.001)
0.990 (.002)
FashionMNIST
0.989 (.001)
0.998
0.998 (.001) 0.992 (.001)
0.992 (.001)
0.998 (.001)
0.997 (_001)
grayCIFAR-10 10
0.995 (.001)
1,000
0.978
(:005) 0.992 (.003)
0.992 (.002)
1.000 (.0001)
1.000 (.0001)
Average
0.989 (.005)
0.970
0..988
(.009)
(.004)
0.996 (.003)
0.996 (.004)
Train on CIFAR- 10
5.99% (.09)
8.44
6.88 (.40) 4.83
4.59 (.08)
7.28 (.06)
6,18 (.09)
CIFAR- 100
0.860 (.001)
0.852
0.827 (.016) 0.891 (.001)
0.897 (.001)
0.865 (.001)
0.882 (.003)
SVHN
0.899 (.012)
0.981
0.912 (.031) 0.917 (.006)
0.924 (.002)
0.908 (.001)
0.900 (.013)
ImageNet
0.834 (.002)
0.859
0.816 (.021) 0.863 (.001)
0.869 (.001)
0.848 (.001)
0.854 (.004)
Average
0.865 (.029)
0.897
0.852 (:050) 0.890 (.023)
0.897 (.023)
0.874 (.026)
0.879 (.021)
Train on CIFARTOO
18% (.28)
32.24
31.14 (.23) 22,43 (.20)
21.86 (.11)
30.89 (0. 10)
24.46 (.12)
CIFAR-I0 10
0.750 (.002)
0.720
0.722 (.008) 0.781 (.001) 0.786 (.001)
0,783 (.001)
0.762 (.002)
SVHN
0.781 (.035)
0.912
0.774 (.006) 0.832 (.013) 0.834 (.009) 0.872 (.001)
0,860 (.006)
ImageNet
0.766 (.002)
0.752
0.742 (.006) 0.798 (.001) 0.803 (.001) 0.755 (.001)
0.793 (.001)
[Average
0.766 (.023) 0.793 0.746 (324) 0.804 (.022) 0.808 ((22T (.50-
0.805 (T022)
Table I: OOD detection comparison in terms of Area under the ROC curve (AUROC) for models
trained and tested on several datasets. For each model the test error in % is shown in the "Train on"
row. The ACET results are taken from Hein et al. (2019). All other results are averaged over 10 runs and the standard deviation is shown in parentheses.
The results are shown in Table 1 All results except the ACET results are averaged over 10 runs and the standard deviation is shown in parentheses. From Table I we observe that our methods obtain the best results MNIST and the 10-ensemble obtains the best results on the other two datasets. The test errors of the CSNN-F approach are smaller than the CSNN, and the AUROCs are comparable. Compared to ACET both CSNN and CSNN-F obtain smaller test errors on all three dataset and better average AUROC on two out of three datasets. Compared to DUQ, the CSNN and CSNN-F obtain comparable test errors and better average AUROC on all three datasets. Compare to the 5-ensemble. the CSNN-F obtains comparable errors on two datasets and comparable or better AUROC on two datasets. Comparing the training time, both our methods are about times faster than training 5-ensemble, 8 times faster than 10- ensemble and about 3 times faster than DUQ.
Results.
4 CONCLUSION
In this paper, we presented a generic neuron formulation that encompasses the standard projection based neuron and the RBF neuron as two extreme cases of a shape parameter a E [0, 1]. By using ReLU the activation function we obtained novel type of neuron that has compact support. We showed how to avoid the difficulties in training the compact support NN by training a standard neural network first (a 0) and gradually shrinking the support by increasing a. We showed the advantages of the proposed compact support neural network in that it can still have good prediction on data coming from the same distribution, but it can detect out of distribution samples consistently well. This feature is important in safety critical applications such as autonomous driving, space exploration and medical imaging. Our results have been obtained without any adversarial training or ensembling, and adversarial training or ensembling could be used in our framework to obtain further improvements.
In the real data applications we used compact support layer as the last layer before the output layer. This ensures that the compact support is involved in the most relevant representation space of the CNN. However, because the CNN still has many projection-based layers to obtain this representation space, it means that the corresponding representation in the original image space does not have compact support and high confidence erroneous predictions are still possible. In the future we plan to study architectures with multiple compact support layers that have even smaller support in the image space.
REFERENCES
David S Broomhead and David Lowe. Radial basis functions, multi-variable functional interpolation and adaptive networks. Technical report, Royal Signals and Radar Establishment Malver (United Kingdom), 1988.
Gregory Cohen, Saeed Afshar, Jonathan Tapson, and André van Schaik. Emnist: an extension of mnist to handwritten letters (2017). arXiv preprint arXiv: 1702.0537.

---

Under review as a conference paper at ICLR 2021
Francesco Croce and Matthias Hein. A randomized gradient-free free attack on relu networks. In German Conference on Pattern Recognition, pp. 225-22152222 Springer, 2018.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. CVPR, pp. 248-255. leee, 2009.
Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In ICML, pp. 1050-1059, 2016.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. ICLR, 2015.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In ICML, pp. 1321-1330. JMLR. org, 2017.
Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pp. 770-778, 2016.
Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. Why relu networks yield high- confidence predictions far away from the training data and how to mitigate the problem. In CVPR, pp. 41-50, 2019.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In ICLR, 2017.
Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. Generalized odin: Detecting out-of- distribution image without learning from out-of-distribution data. In CVPR, pp. 10951-10960, 2020.
Heinrich Jiang, Been Kim, Melody Guan, and Maya Gupta. To trust or not to trust classifier. In Neur/PS, pp. 5541-5552, 2018.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In NewrlPS, pp. 6412-6413, 2017.
Yann LeCun and Corinna Cortes. MNIST handwritten digit database. 2010. URL hhtp:://yan
lecun wwtp//////////eece/e
Kimin Lec, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In NeurlPS, pp. 7167-7177,201
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2018.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew NgNg Reading digits in natural images with unsupervised feature learning. In NeurIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011, 2011. URL htt.tttt://fflddd stanford edu http:/houseenmmmbrr//nisssss
housenumbers. pdd
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In CVPR, pp. 427-436, 2015.
Jie Ren, Peter J Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark Depristo, Joshua Dillon, and Balaji Lakshminarayanan. Likelihood ratios for out-of out-of-distribution detection. In NeurIPS, pp. 14707-14718, 2019.
Joost van Amersfoort, Lewis Smith, Yee Whye Teh, and Yarin Gal. Uncertainty estimation using a single deep deterministic neural network. In ICML, 2020.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mmist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.077477 2017.
10

---

Under review as a conference paper at ICLR 2021
Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In ICCV, pp. 6023-6032, 2019.
Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In ECCV, pp. 818-833. Springer, 2014.


Reviews:
[{"review_id": "q-9cVwJFZVC", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, authors propose compact support neurons to prevent high confidence responses from examples that are away from the training data. The design and training of such a neuron seem novel. \n\nThe reviewer has the following comments:\n\n1. In (3), what is R2? The missing definition of R2 adds difficulty to fully interpret subsequent equations. \n\n2. Authors start with the RBF neuron, and then replace it with a compact support neuron. However, the relationship between these two seems not very well-explained.\n\n3. From experiments, the proposed method shows effective for the out-of-distribution (OOD) sample detection task. Authors may like to use cifar-10 and cifar-100 as the example to further explain the out-of-distribution detection. For example, how the conceptual overlap across these two datasets contribute to the results? How does a typical CNN perform for OOD detection?\n\n4. While appreciating proposed OOD detection capability, the reviewer is not fully convinced that a compact support network is in general beneficial to knowledge representation. Of course, such a debate might appear slightly out-of-scope here. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "XZZyoUGU9e9", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "### Summary\n\nThe authors propose a new neural network unit and training algorithm in order to improve OOD detection. Units can smoothly be changed between standard (dot-product) and RBF type through a shape hyperparameter. During training, this hyperparameter is slowly moved in the direction of the RBF shape. Empirical comparisons on three OOD problems are presented, showing that the proposed approach is competitive.\n\n\n### Score\nThe authors propose a novel algorithmic idea for OOD detection and show that it is competitive. However, the results show that their approach is highly sensitive to the choice of the shape hyperparameter alpha. In practice, robust ways of choosing and annealing the shape hyperparameter alpha will be needed for the method to be useful. As detailed below, I think this aspect is not addressed in satisfactory detail. Hopefully the weaknesses and comments will be addressed by the authors during rebuttal, currently, I view the paper as marginally below acceptance.\n\n### Pros\n+ New methods to reliable detect OOD samples are potentially impactful, since they can be applied across different domains\n+ The authors present a novel algorithmic idea to the best of my knowledge\n+ CSNNs overall show good performance when compared against alternative approaches\n\n### Cons\n- The results show that the authors method is highly sensitive to the choice of the shape hyperparameter alpha. For their empirical comparisons the authors chose alpha as follows: “Based on these insights, for each dataset we chose the classifier corresponding to the largest α where the test error takes a value comparable to the other methods compared, and reported the AUROC and NZ values in Table 1”. This selection strategy can hardly be transferred to practice — it would mean training another OOD method first, in order to select a suitable alpha. For the proposed algorithm to be useful in practice, it is central to devise and demonstrate robust schemes of finding the shape hyperparameter that do not rely on running another method. Else, methods that can be more easily used and show comparable performance, such as deep ensembles, may be preferable despite higher training times\n- Similar to the UCQ paper, it would be worthwhile to report performance of larger ensemble methods, e.g., ensemble of 10 networks\n- No variability in performance metrics is provided in Table 1, runs should be repeated and reported with different seeds and measures of variability (e.g. the standard error of mean) should be reported\n\n### Additional comments\n- My understanding from reading the paper is that neurons with zero outputs are pruned. In figure 9, the amount of neurons with non-zero outputs (NZ curve) seems to quickly go down to 0. Wouldn’t this imply that all neurons in the output layer are pruned, i.e., how can such a network still be used?\n- The curves for train and test error of MNIST in Figure 9 seem to be identical, is there an explanation for this?\n- Reference section should be revised carefully, e.g.: Goodfellow et al. (2014) was published in ICLR 2015, Hendrycks et al. (2017) misses venue\n\n\n### Update\n\nThe authors have not sufficiently addressed my comments regarding the choice of alpha in practice. While a possible strategy was proposed, it has not been evaluated empirically, see comments below for more details. I am thus maintaining my score.", "extra_scores": {}, "extra_text": {}}, {"review_id": "2KNYc_M5nW", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The main contribution is that the author porposed a formulation which connects the RBF neuron and RELU neuron in neural network to overcome the difficulties in training a DNN model with RBF layer(s). \n\npros:\n- This paper is well written and easy to understand. The illustration summarizes well its effect of OOD predictions.\n- The formulation of eqn.3 is interesting and naturally connect RBF and RELU neurons with a parameter $\\alpha$. \n\ncons:\n- The essential idea of this compact support network is based on the assumption that $L_2$ distance of the representation space is meaningful. Is that correct? How about other metric? How about to pre-calculate the distance on dataset in the representation space, and reject those samples which are away from the class center?\n- The performance gain of proposed method comparing to other baseline methods which can also stable the training of RBF network is marginal (espeically in Table.1).\n- The backbone and the training datasets of RBF network are quite small, could the proposed method stable the training of large network with large dataset (such as training deeper ResNet on Imagenet)?", "extra_scores": {}, "extra_text": {}}, {"review_id": "-2rBJXXeoEV", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper presents an approach that supports better performance when out of distribution cases occur. It does so by letting neurons be of only compact support and thus if the input is out of distribution (OOD) it is expected to be outside that support and therefore the output will be zero. This is used to detect OOD examples. A parameter alpha is used in the algorithm that determines the size of the support. When it is small then the network acts very similarly to a regular network and when it increases the support is limited. Therefore, to make training stable, they start with a small alpha and then increase it throughout the training. \nThen in inference, a value for alpha should be selected that balances the classification accuracy and the OOD detection. For small alpha the classification accuracy is relatively good, yet, the OOD detection is virtually zero. When alpha is very large the OOD detection rate is very high but the classification accuracy deteriorates significantly. \n\nThe strategy proposed in this work is quite interesting and might be useful. Yet, I have some concerns:\n\nTo make the comparison fair for all the methods, the same test error should be used. As can be seen from the graphs, the OOD decreases significantly when the Test error decreases. \n\nAlso, a discussion on the selection of alpha is missing. How it should be selected in a real application? \n\nAnother issue is related to some recent works \n\nHow this work compares to \n-Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In International Conference on Neural Information Processing Systems, 2018\n\n\n-https://proceedings.neurips.cc/paper/2019/hash/1e79596878b2320cac26dd792a6c51c9-Abstract.html\n\n- https://openaccess.thecvf.com/content_CVPR_2020/html/Hsu_Generalized_ODIN_Detecting_Out-of-Distribution_Image_Without_Learning_From_Out-of-Distribution_Data_CVPR_2020_paper.html\n\n\n-https://openaccess.thecvf.com/content_ICCV_2019/html/Yun_CutMix_Regularization_Strategy_to_Train_Strong_Classifiers_With_Localizable_Features_ICCV_2019_paper.html\n\n\nReview Update:\nI am raising my score but I think the following changes are still required in the paper: \n1. Add a section about the choice of alpha (current addition is lacking) discussing the robustness of alpha selection and how easy it is to select it on real problems.\n2. Compare to the methods mentioned.\n3. Display the proposed approach on deep networks. Does the proposed approach apply there? Given that current SOTA uses very deep networks (with skips) it is important to be able to use the proposed method with such networks.\n\n\n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper presents an approach that supports better performance when out of distribution cases occur, by letting neurons be of only compact support and thus if the input is out of distribution (OOD). 

Pros:
- The proposed strategy is interesting and may be useful.

Cons:
- The choice of the parameter alpha, whose value is crucial to the success in experiments, is left murky. The approach suggested by the authors was not validated experimentally. 
- There is insufficient comparison to recent works.
Decision: Reject