  With the successful application of deep learning models in many real-world
tasks, the model robustness becomes more and more critical. Often, we evaluate
the robustness of the deep models by attacking them with purposely generated
adversarial samples, which is computationally costly and dependent on the
specific attackers and the model types. This work proposes a generic evaluation
metric ROBY, a novel attack-independent robustness measure based on the model's
decision boundaries. Independent of adversarial samples, ROBY uses the
inter-class and intra-class statistic features to capture the features of the
model's decision boundaries. We experimented on ten state-of-the-art deep
models and showed that ROBY matches the robustness gold standard of attack
success rate (ASR) by a strong first-order generic attacker. with only 1% of
time cost. To the best of our knowledge, ROBY is the first lightweight
attack-independent robustness evaluation metric that can be applied to a wide
range of deep models. The code of ROBY is open sourced at
https://github.com/baaaad/ROBY-Evaluating-the-Robustness-of-a-Deep-Model-by-its-Decision-Boundaries.
