Title: How to Break Image Classification Models: Random Noise knows better than Saliency
Primary area: interpretability and explainable AI
Keywords: Model Degradation, Explainable AI, Attribution Methods, Saliency Evaluation, Model Robustness, Image Classification, Perturbation Analysis, Degradation Curves

Abstract: In safety critical domains such as autonomous driving or medical diagnosis, it is not enough to know what image classifiers focus on, we must also understand how vulnerable their decisions are to the removal of supposedly important evidence. We propose a degradation-based evaluation framework built on the central hypothesis that masking the most important input regions should cause maximal prediction collapse. Attribution methods such as SHAP, Grad-CAM, or Integrated Gradients are therefore treated as perturbation guides, and their effectiveness is measured by the extent to which they accelerate model degradation. Our framework produces degradation curves from which we derive three quantitative metrics: Area Under the Blindness Curve (AUBC), Sensitivity Slope, and Attribution Collapse Point. This shifts evaluation from judging saliency maps by visual plausibility to assessing their behavioral impact on model predictions. Strikingly, across standard architectures we find that random masking often surpasses attribution based masking, revealing a fundamental gap between visual explanations and true decision dependencies. By exposing this gap, our framework enables reproducible, model agnostic robustness analysis and opens the path toward vision models that are empirically aware of their own failure modes.

TL;DR: We show that popular attribution methods fail to reveal prediction-critical regions by introducing a degradation-based framework where random masking breaks vision models more effectively than saliency maps.