  In this work, we introduce a control variate approximation technique for low
error approximate Deep Neural Network (DNN) accelerators. The control variate
technique is used in Monte Carlo methods to achieve variance reduction. Our
approach significantly decreases the induced error due to approximate
multiplications in DNN inference, without requiring time-exhaustive retraining
compared to state-of-the-art. Leveraging our control variate method, we use
highly approximated multipliers to generate power-optimized DNN accelerators.
Our experimental evaluation on six DNNs, for Cifar-10 and Cifar-100 datasets,
demonstrates that, compared to the accurate design, our control variate
approximation achieves same performance and 24% power reduction for a merely
0.16% accuracy loss.
