  Neural networks are popular and useful in many fields, but they have the
problem of giving high confidence responses for examples that are away from the
training data. This makes the neural networks very confident in their
prediction while making gross mistakes, thus limiting their reliability for
safety-critical applications such as autonomous driving, space exploration,
etc. In this paper, we present a neuron generalization that has the standard
dot-product-based neuron and the RBF neuron as two extreme cases of a shape
parameter. Using ReLU as the activation function we obtain a novel neuron that
has compact support, which means its output is zero outside a bounded domain.
We show how to avoid difficulties in training a neural network with such
neurons, by starting with a trained standard neural network and gradually
increasing the shape parameter to the desired value. Through experiments on
standard benchmark datasets, we show the promise of the proposed approach, in
that it can have good prediction accuracy on in-distribution samples while
being able to consistently detect and have low confidence on
out-of-distribution samples.
