Before revision:  We discuss the advantages and limitations of controversial stimuli as an experimental paradigm and how they generalize and improve onadversarial examples as probes of discrepancies between models and human perception.
After revision:  Controversial stimuli generalize the concept of adversarial examples, obviating the need to assume a ground-truth model. Unlike natural images, controversial stimuli are not constrained to the stimulus distribution models are trained on, thus providing severe out-of-distribution tests that reveal the models' inductive biases. Controversial stimuli therefore provide powerful probes of discrepancies between models and human perception.