SlowGuess/NeurIPS_2023_OCR huggingface dataset:
ascending efficiency order (marking MTME score, lower is better): Rank 5) Vanilla, which never reaches the  $10\%$  threshold within the considered range, Rank 4) Robusified  $\varepsilon_{tr}[1.0]$  (196), Rank 3) Image interpolation (47), Rank 2) Robustified  $\varepsilon_{tr}[10.0]$  (11), and the best, Rank 1) Robustified  $\varepsilon_{tr}[3.0]$  (9). This suggests that there exist an optimal robustification level for strongest modulation effects. While this precise optimization is beyond the scope of this work, we highlight that the Robust  $\varepsilon_{tr}[3.0]$  model is currently the most TM-efficient amongst all tested.
Other pixel budget constraints. In addition to the  $\ell_{2}$  perturbation constraint, we also explored the use of  $\ell_{\infty}$  constraint for TM. Visual inspection of the resulting images revealed that both constraints induce comparable modulation effects, with the  $\ell_{2}$  constraint giving rise to images that appeared slightly more visually "clean" and aesthetically pleasing. We also found that TM by  $\ell_{\infty}$ -trained models was significantly weaker compared to the  $\ell_{2}$ -trained models.
4.1 Distances in image space
We exclusively focused on  $224 \times 224 \times 3$  RGB pixel space ( $D = 150, 528$ ), where each value ranges between 0 and 1. Thus, the maximal distance between any two images is  $\| \delta \|_2^{max} \approx 388$ . For
reference, we found the typical distance between ImageNet images to be  $\approx 130$ . We also estimated the regime in which human object perception is empirically robust to random perturbations to be  $\| \delta \|_2 \leq 30$  (see Supplementary Materials; also see [35]).
4.2 Stimuli generation by model-guided perturbations
Disruption Modulation (DM). We used the Projected Gradient Descent (PGD) attack algorithm [31, 36, 37], using an adapted version of the robustness library [32]. Given an initial input image and its ground-truth label in 1-hot representation  $x$ ,  $y$ , and a classification model to logits for  $C$  classes,  $f: \mathbb{R}^{H \times W \times 3} \to \mathbb{R}^C$ , we optimize for the perturbation,  $\delta$ , such that to be disruptive of model classification to the ground-truth class label under a pixel budget restriction:
$$
\delta = \underset {| | \delta | | _ {p} <   \epsilon} {\operatorname {a r g m a x}} \mathcal {L} _ {C E} (f (x + \delta), y), \tag {1}
$$
where  $\epsilon$  is the pixel budget, and  $\mathcal{L}_{CE}$  is the cross-entropy loss. Unless otherwise mentioned, we focus on  $\ell_2$ -adversarially-trained models and attacks, i.e.,  $p = 2$ . Optimization is performed in steps of Stochastic Gradient Descent (SGD), each followed by a projection step to pixel budget  $\epsilon$ :
$$
\delta_ {k + 1} \leftarrow \operatorname {P r o j} _ {\epsilon} \left(\delta_ {k} + \eta \nabla_ {\delta} \mathcal {L} _ {C E} (f (x + \delta_ {k}), y)\right), \tag {2}
$$
where  $k$  denotes the step, and  $\eta$  is the step-size. All original input images are resized to  $256 \times 256$  (bilinear, anti-aliased) and center-cropped to  $224 \times 224$  [22]. We focused on the following range of  $\ell_2$ -pixel budgets in the low-budget regime: [0.1, 0.5, 1.0, 2.0, 3.0, 5.0, 7.5, 10, 15, 20, 25, 30, 40, 50]. We set the number of PGD steps and the step-size,  $(k_{steps}, \eta)$ , such to match the pixel budget [32], ranging from (200, 0.02) for  $\epsilon = 0.1$  to (2000, 2) for  $\epsilon = 50$ .
Targeted Modulation (TM). The algorithm is identical to DM, aside from  $y$  changing to be a given target class label, and the optimization solving for minimization in lieu of maximization (flipping of gradient sign). Notably, unlike DM, this modulation can be performed toward multiple target class labels from a single source image.
Hybrid-Class Targeted Modulation. To modulate toward non-cardinal directions we defined random composite directions involving multiple classes. These directions were constructed as probability vectors with all-zero except for  $k$  randomly selected classes, which were assigned equal probabilities. We generated four 2-composite directions: 'frog-bird', 'frog-insect', 'primate-crab', 'primate-fish'; Our objective function was based on cross-entropy similar to Eq 1, with additional logit maximization on the non-zero target classes, which drives higher usage of the allowable budget.
Features Targeted Modulation. To modulate images toward a target feature representation we compiled a subset of 60 images per class and computed their mean feature representation using the GM. This defines the target classes via class-centroids. The optimization criterion is then the MSE between the predicted feature representation the target class-centroid.
Image interpolation modulation. To generate baseline image perturbations using an image interpolation approach, we randomly drew images from a target class, used them for linear interpolation with the source image, and projected back the pixel budget envelope.
Dataset. We sought to focus on a dataset with the following key properties: (i) Has a tractable class-space for exploration; (ii) Will mitigate confounds originating from typical-human unfamiliarity with the class labels (as is commonly the case in ImageNet); (iii) Is widely used in the adversarial robustness context. Based on these considerations, we found a partial and mapped version of ImageNet to a basic set of nine classes, termed "Restricted ImageNet", to be the most suitable choice (see Supplementary Material for class mapping) [32].
Training robustified models. To adversarially-train models on ImageNet [38], we followed publicly released code [32]. Specifically, we trained ResNet50 models at  $\ell_2$  pixel budgets of 1.0, 3.0, and 10.0, using PGD (steps, step-size) of (7, 0.3), (7, 0.5), (10, 1.5), respectively.
Runtime. Our stimuli generation completes within 5min for a single batch of size 100 on a single A100 GPU. Adversarial-training of models completes within  $\sim 8$  days on 4 A100 GPUs.
4.3 Human behavioral measurements
We measured human behavior in a nine-way image categorization task. Our primary experimental objective was, for any given test image, to estimate the probability that humans would on average select each of the nine possible choices.
Task paradigm. Human subjects performed "sessions" comprising multiple trials. Each trial had three stages: (i) First, the subject pressed a button at the center of their screen, encouraging them to center-fixate. (ii) A test image (covering approximately 6 degrees of visual angle) appeared at the center of their screen for  $200\mathrm{ms}$ . This was followed by  $200\mathrm{ms}$  of a blank gray screen. (iii) An array of nine labeled buttons (one for each of the nine reportable categories) appeared. These buttons were arranged radially, outside of the location of where the test image appeared. The locations of the buttons were randomly permuted on each trial, and no time limit was imposed. The subject was asked to select the button which most accurately described the image.
In total, there were  $n = 130$  such trials in each session. Among these, there were four types of trials: (i) Warmup trials ( $n = 10$ ), which were always presented at the beginning of the session. Test images were randomly sampled "clean" (unperturbed) images drawn from the Restricted ImageNet validation set [32]. (ii) Calibration trials ( $n = 10$ ), included to measure the lapse rate of each human subject (see Lapse-rate correction). Stimulus images consisted of either a blue triangle or blue circle, and two randomly selected buttons on the choice screen were replaced with buttons for 'circle' and 'triangle'. (iii) Reference trials ( $n = 10$ ), consisting of unperturbed stimulus images randomly selected from Restricted ImageNet validation set. (iv) Main trials ( $n = 100$ ), consisting of potentially perturbed versions of source images. Trials types (ii)-(iv) were run randomly interleaved, following the warmup trials. In general, subject feedback was only delivered for unperturbed images. Upon an incorrect selection by the subject, a black "x" was displayed for 1000ms; No feedback was delivered upon a correct selection. We implemented the task paradigm using JSPsych, and presented the stimuli using the JSPsych-Psychophysics plugin.
Human data collection. We used Amazon Mechanical Turk to recruit and collect behavioral measurements in human subjects, following a protocol approved by the [anonymized IRB information] The protocol has no greater than minimal risk involved for the subjects. We did not attempt to collect any personally identifying information; subjects were kept anonymous. We screened subjects by their performance on a similar "demo" object categorization task (see Supplementary Material). In total, we recruited a population of  $n = 119$  subjects ( $n = 43$  female;  $n = 76$  male). Subjects were compensated for all time spent in this study, including in the recruitment phase. Subjects were free to opt out of the study at any time. We ensured that each image we included in our analysis had measurements from at least  $n = 2$  different subjects.
Lapse-rate correction. We assumed that measurements of human reports was possibly corrupted by a base "lapse rate", where a subject makes a random report independent of the presented test image's content. To account for that, we measured each subject's lapse rate, then adjusted their measurements using a formula that estimates their behavior under a lapse rate of zero (see Supplementary Material).
In this work, we systematically examine the prevailing assumption that human categorization is highly robust to low-norm image perturbations. Our findings challenge that assumption by establishing that the low-norm image perturbations suggested by robustified ANNs (but not vanilla ANNs) can strongly and precisely change human object category percepts. These results suggest two novel conclusions: (i) for arbitrary starting points in image space, there exists nearby "wormholes", local (i.e., low-norm) perturbations in image space that support "travel" from the current category perceptual state induced in the subject into a semantically very "distant" perceptual state, and (ii) contemporary scientific models of ventral visual processing are accurate enough to point us to those wormholes.
Our findings raise many new questions: given a start image, does there always exist a nearby human perceptual wormhole to at least one another category? Are there nearby wormholes to any possible category? Which wormholes are common to all humans and which are more idiosyncratic? And do robustified ANNs always find the wormholes that do exist? And how often do they predict wormholes that do not actually exist? We do not yet know the answers to these questions, but the results presented
here offer some insight. For example, Fig 2b right panel, showing a  $\sim 90\%$  disruption effect, is consistent with the hypothesis that wormholes are abundant in image space. Figs 3 and 4 ( $\sim 60\%$  effect) suggests that multiple category portals are nearby, but that portals to all categories are not always nearby. Further work could produce tighter lower-bounds on such probabilities.
We emphasize that these are lower bound estimates because we are probing human vision with models that are imperfect approximations of the underlying neurobiology. Indeed, we do not claim that robustified ANN models discover all low-norm wormholes or that they never point to wormholes that do not actually exist. Notably, because we still see a residual gap in behavioral alignment (Figs 1c and 2b), other models, which are even more behaviorally aligned, must still exist. Specifically, our analyses focused on the ResNet50 model architecture, and mainly on  $\ell_2$ -norm budget robustified models and image perturbation type (see Sec 3.2 for other perturbation constraints). While generalizing to other architectures or image generation algorithms is not foundational to the our claims presented in this paper, those are interesting directions to explore in future work.
Furthermore, while this paper supports that adversarial training (AT) gives rise to an improved estimate of the adults state of the ventral stream and its supported behavior, we do not claim that AT is the mechanism by which robustness in humans emerges. Future work may consider alternative, more biologically plausible mechanisms (e.g., Topographic ANNs [39]) that may give rise to a comparably aligned models in terms of predictivity.
5.2 Societal impact
Image generation methods such as those used here have potential societal benefits, but also potential risks [40]. The scientific knowledge of brain function contributed here may present additional future risks, in that it may widen the ability of malicious actors to subtly and non-consensually disrupt the perception of humans. Building defenses against those manipulations, and orienting the use of this knowledge toward societal benefits (e.g., consensually enriching perception, improving mental health, accelerating human visual learning) are crucial considerations in future work.
This work was partially funded by the Office of Naval Research (N00014-20-1-2589, JJD); (MURI, N00014-21-1-2801, JJD), the National Science Foundation (2124136, JJD) and the Simons Foundation (542965, JJD).