Before revision:  In this paper, we develop two quantitative evaluation procedures for saliency methods, using the fact that the Human Connectome Project (HCP) dataset contains functional magnetic resonance imaging (fMRI ) datafrom multiple tasks per subject to create ground truth saliency maps . We then introduce an adversarial training method that makes DNNs robust to small input noise, and demonstrate that it measurably improves interpretability .
After revision:  In this paper, we develop two quantitative evaluation procedures for saliency methods, using the fact that the Human Connectome Project (HCP) dataset . Our key finding is that saliency maps produced with different methods vary widely in interpretability, in both in synthetic and HCP fMRI data. Strikingly, even when DNN and linear models decode at comparable levels of performance, DNN saliency maps score higher on interpretability than linear model saliency maps (derived via weights or gradient). Finally, saliency maps produced with our adversarial training method that makes DNNs robust to small input noise, and demonstrate that it measurably improves interpretability .