  State-of-the-art neural networks with early exit mechanisms often need
considerable amount of training and fine-tuning to achieve good performance
with low computational cost. We propose a novel early exit technique based on
the class means of samples. Unlike most existing schemes, our method does not
require gradient-based training of internal classifiers. This makes our method
particularly useful for neural network training in low-power devices, as in
wireless edge networks. In particular, given a fixed training time budget, our
scheme achieves higher accuracy as compared to existing early exit mechanisms.
Moreover, if there are no limitations on the training time budget, our method
can be combined with an existing early exit scheme to boost its performance,
achieving a better trade-off between computational cost and network accuracy.
