  Despite their appealing flexibility, deep neural networks (DNNs) are
vulnerable against adversarial examples. Various adversarial defense strategies
have been proposed to resolve this problem, but they typically demonstrate
restricted practicability owing to unsurmountable compromise on universality,
effectiveness, or efficiency. In this work, we propose a more practical
approach, Lightweight Bayesian Refinement (LiBRe), in the spirit of leveraging
Bayesian neural networks (BNNs) for adversarial detection. Empowered by the
task and attack agnostic modeling under Bayes principle, LiBRe can endow a
variety of pre-trained task-dependent DNNs with the ability of defending
heterogeneous adversarial attacks at a low cost. We develop and integrate
advanced learning techniques to make LiBRe appropriate for adversarial
detection. Concretely, we build the few-layer deep ensemble variational and
adopt the pre-training & fine-tuning workflow to boost the effectiveness and
efficiency of LiBRe. We further provide a novel insight to realise adversarial
detection-oriented uncertainty quantification without inefficiently crafting
adversarial examples during training. Extensive empirical studies covering a
wide range of scenarios verify the practicability of LiBRe. We also conduct
thorough ablation studies to evidence the superiority of our modeling and
learning strategies.
