Before revision:  We find that synthetic images indeed provide helpful information about feature map activations (82 \% accuracy; chance would be 50\%). However, natural images-originally intended to be a baseline-outperform synthetic images by a wide margin (92\% accuracy). Additionally, participants are faster and more confident for natural images, whereas subjective impressions about the interpretability of feature visualization are mixed. The higher informativeness of natural images holds across most layers, for both expert and lay participants as well as for hand- and randomly-picked feature visualizations. Even if only a single reference image is given, synthetic images provide less information than natural images (65\% vs. 73\%). In summary, popular synthetic images from feature visualizations are significantly less informative for assessing CNN activations than natural images. We argue that future visualization methods should improve over this simple baseline.
After revision:  We find that synthetic images indeed provide helpful information about feature map activations (82