Relevant-features based Auxiliary Cells for Robust and Energy Efficient Deep Learning
Deep neural networks are complex non-linear models used as predictive analytics tool and have demonstrated state-of-the-art performance on many classification tasks. However, they have no inherent capability to recognize when their predictions might go wrong.There have been several efforts in the recent past to detect natural errors i.e. misclassified inputs but these mechanisms pose additional energy requirements. To address this issue, we present a novel post-hoc framework to detect natural errors in an energy efficient way. We achieve this by appending relevant features based linear classifiers per class referred as Relevant features based Auxiliary Cells. The proposed technique makes use of the consensus between RACs appended at few selected hidden layers to distinguish the correctly classified inputs from misclassified inputs.The combined confidence of RACs is utilized to determine if classification should terminate at an early stage.We demonstrate the effectiveness of our technique on various image classification datasets such as CIFAR10, CIFAR100 and Tiny-ImageNet.Our results show that for CIFAR100 dataset trained on VGG16 network, RACs can detect 46% of the misclassified examples along with 12% reduction in energy compared to the baseline network while 69% of the examples are correctly classified.
Improve the robustness and energy efficiency of a deep neural network using the hidden representations.This paper aims to reduce the misclassifications of deep neural networks in an energy efficient way by adding Relevant feature based Auxiliary Cells after one or more hidden layers to decide whether to end classification early.