  This work proposes a novel Energy-Aware Network Operator Search (ENOS)
approach to address the energy-accuracy trade-offs of a deep neural network
(DNN) accelerator. In recent years, novel inference operators have been
proposed to improve the computational efficiency of a DNN. Augmenting the
operators, their corresponding novel computing modes have also been explored.
However, simplification of DNN operators invariably comes at the cost of lower
accuracy, especially on complex processing tasks. Our proposed ENOS framework
allows an optimal layer-wise integration of inference operators and computing
modes to achieve the desired balance of energy and accuracy. The search in ENOS
is formulated as a continuous optimization problem, solvable using typical
gradient descent methods, thereby scalable to larger DNNs with minimal increase
in training cost. We characterize ENOS under two settings. In the first
setting, for digital accelerators, we discuss ENOS on multiply-accumulate (MAC)
cores that can be reconfigured to different operators. ENOS training methods
with single and bi-level optimization objectives are discussed and compared. We
also discuss a sequential operator assignment strategy in ENOS that only learns
the assignment for one layer in one training step, enabling greater flexibility
in converging towards the optimal operator allocations. Furthermore, following
Bayesian principles, a sampling-based variational mode of ENOS is also
presented. ENOS is characterized on popular DNNs ShuffleNet and SqueezeNet on
CIFAR10 and CIFAR100.
