Title: Attribution-Guided Exit Policy for Reliable Early-Exit Inference
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: Early-Exit Neural Networks, Interpretable Deep Learning, Explainable AI (XAI), Exit Policy Optimization, Attribution Maps, Semantic Consistency, PFAM, Anytime Prediction

Abstract: Early-exit neural networks enhance efficiency by producing predictions at intermediate layers, but their utility depends critically on the exit policy that decides when to stop inference. Conventional policies, typically based on confidence thresholds, lack transparency and provide little justification for their choices. We address this limitation with two contributions. First, we introduce Progressive Feature Attribution Maps (PFAM), which capture the network’s evolving focus through both exit-wise and cumulative attributions. Second, we propose the Interpretability-Based Early-Exit Score (IEES), a composite metric that unifies confidence, attribution progression, and feature relevance. To eliminate attribution overhead at deployment, we further develop a lightweight proxy predictor that estimates IEES from inexpensive, attribution-free features. Experiments on ResNet-18 (CIFAR-10), MSDNet (CIFAR-100), and MobileNetV3 (ImageNet) show that our approach improves accuracy by up to 3\% while reducing computation by up to 26\% compared to full-depth inference. Analyses demonstrate that IEES-driven exits are semantically coherent and reliable, while PFAM achieves near-perfect monotonicity, convergence alignment, and strong faithfulness across benchmarks. Together, these advances enhance the transparency of anytime prediction and establish a foundation for explainable early-exit inference.

TL;DR: We propose IEES and PFAM, two modules that guide early-exit decisions in DNNs by combining confidence, attribution cues, and consistency—improving efficiency and explainability for real-world deployment.