  There are many deep learning (e.g., DNN) powered mobile and wearable
applications today continuously and unobtrusively sensing the ambient
surroundings to enhance all aspects of human lives. To enable robust and
private mobile sensing, DNN tends to be deployed locally on the
resource-constrained mobile devices via model compression. The current practice
either hand-crafted DNN compression techniques, i.e., for optimizing
DNN-relative performance (e.g., parameter size), or on-demand DNN compression
methods, i.e., for optimizing hardware-dependent metrics (e.g., latency),
cannot be locally online because they require offline retraining to ensure
accuracy. Also, none of them have correlated their efforts with runtime
adaptive compression to consider the dynamic nature of the deployment context
of mobile applications. To address those challenges, we present AdaSpring, a
context-adaptive and self-evolutionary DNN compression framework. It enables
the runtime adaptive DNN compression locally online. Specifically, it presents
the ensemble training of a retraining-free and self-evolutionary network to
integrate multiple alternative DNN compression configurations (i.e., compressed
architectures and weights). It then introduces the runtime search strategy to
quickly search for the most suitable compression configurations and evolve the
corresponding weights. With evaluation on five tasks across three platforms and
a real-world case study, experiment outcomes show that AdaSpring obtains up to
3.1x latency reduction, 4.2 x energy efficiency improvement in DNNs, compared
to hand-crafted compression techniques, while only incurring <= 6.2ms
runtime-evolution latency.
