Contour Integration Underlies Human-Like Vision
Authors: Ben Lonnqvist
Elsa Scialom
Abdulkadir Gokce
Zehra Merchant
Michael Herzog
Martin Schrimpf
Venue: ICML 2025 poster (2025)
Keywords: psychophysics
machine vision
human vision
contour integration
robustness
visual perception
Primary area: applications->neuroscience_cognitive_science
TL;DR: Models fail at human-like contour integration; those that integrate contours in a more human-like way also reach better accuracy and robustness.

Abstract: Despite the tremendous success of deep learning in computer vision, models still fall behind humans in generalizing to new input distributions. Existing benchmarks do not investigate the specific failure points of models by analyzing performance under many controlled conditions. Our study systematically dissects where and why models struggle with contour integration - a hallmark of human vision -- by designing an experiment that tests object recognition under various levels of object fragmentation. Humans (n=50) perform at high accuracy, even with few object contours present. This is in contrast to models which exhibit substantially lower sensitivity to increasing object contours, with most of the over 1,000 models we tested barely performing above chance. Only at very large scales ($\sim5B$ training dataset size) do models begin to approach human performance. Importantly, humans exhibit an integration bias - a preference towards recognizing objects made up of directional fragments over directionless fragments. We find that not only do models that share this property perform better at our task, but that this bias also increases with model training dataset size, and training models to exhibit contour integration leads to high shape bias. Taken together, our results suggest that contour integration is a hallmark of object vision that underlies object recognition performance, and may be a mechanism learned from data at scale.

Contour Integration Underlies Human-Like Vision
Ben Lonnnvist Elsa Scialom
Abdulkadir Gokce" Zehra Merchant Martin Schrimpf
Michael H. Herzog
Abstract
Despite the tremendous success of deep learning in computer vision, models still fall behind hu- mans in generalizing to new input distributions. Existing benchmarks do not investigate the spe- cific failure points of models by analyzing per- formance under many controlled conditions. Our study systematically dissects where and why mod- els struggle with contour integration a hallmark of human vision - by designing an experiment that tests object recognition under various levels of object fragmentation. Humans (n=50) perform at high accuracy, even with few object contours present. This is in contrast to models which ex- hibit substantially lower sensitivity to increasing object contours, with most of the over 1,000 mod- els we tested barely performing above chance. Only very large scales (~ 5 training dataset size) do models begin to approach human perfor- mance. Importantly, humans exhibit integra- tion bias a preference towards recognizing ob- jects made up of directional fragments over direc- tionless fragments. We find that not only do mod- els that share this property perform better at our task, but that this bias also increases with model training dataset size, and training models to ex- hibit contour integration leads to high shape bias. Taken together, our results suggest that contour integration is hallmark of object vision that un- derlies object recognition performance, and may be mechanism learned from data at scale.
'pan"
"pan"
"pan"
"pan"
'pan"
"lamp
Figure I. Human and DNN categorization of (left): standard RGB image. (Middle): a contour-extracter image. (Right): a fragmented image requiring contour integration. The vast majority of over 1,000 tested models catastrophically fail at a categorization task the moment object contours are fragmented.
ments. You would most likely have no trouble recognizing the drawing as pan after all, your visual systems "fills in the blanks" and allows you to see the pan regardless of its fragmentation (Figure 1). This is an example of contour integration, a well-established mechanism in the primate brain, whereby disconnected elements are integrated into a continuous form, enabling recognition. Contour integration is highly relevant to naturalistic vision due to the number of occlusions in the natural environment, and it yet remains un- clear as to why the human brain is so robust to this particular type of image perturbation.
While deep neural networks (DNNs) reach superhuman per- formance standard image recognition benchmarks like ImageNet (Russakovsky et al., 2015), their generalization pabilities to unseen image distribution shifts lags behind hu- man generalization capabilities (Geirhos et al., 2021; Baker et al., 2018: 8; Bowers et al., 2023; Biscione & Bowers, 2023; Malhotra et al., 2022; Muttenthaler et al., 2023; Malhotra et al., 2023). Despite this large body of evidence suggesting differences between human and model generalization capa- bilities, the causes behind the differences remain unclear.
1. Introduction
Picture simple drawing of a pan where most of its con- tours are erased, leaving only a set of disconnected frag-
'Equal second authors 'École Polytechnique Fédérale de Lau- sanne (EPFL), Switzerland. Correspondence to: Ben Lonnqvist <benhlonnqvist@ gmail.com>.
A key reason for the difficulty in understanding model fail- ures is a lack of systematic studies that investigate individual well-established mechanisms in the primate brain. By lack of systematicity we mean that studies typically fall short on one of two axes: first, they may not investigate a set
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Contour Integration Underlies Human-Like Vision
of related experimental conditions that would allow them to know where exactly models diverge from human behav- ior. Second, studies simply may not include a sufficiently large model set to allow a quantitative explanation of model behavior at scale. For example, simply knowing whether models integrate contours would offer limited insight; know- ing how to improve them is crucial. In this work, we seek to bridge this explanation gap using contour integration as an exemplary test case.
statistical regularities in training data in naturalistic settings (Beery et al., 2018; Geirhos et al., 2020a). Geirhos et al. (2018) and Baker et al. (2018) tested several DNNs shape bias task and found that DNNs make categorization decisions largely based on image texture content, rather than shape content. This work was expanded later with larger set of experimental data spanning different data distribution shifts (Geirhos et al., 2021). A key finding of the work was the observation that amongst set of 52 models, the mod- els trained with the most data appeared to make the most human-likk like categorization errors, while models trained on less data made more idiosyncratic errors. (Hermann et al., 2020) argued that data augmentations can also help lead to less texture bias.
In short, we develop a new contour integration task that we tested 50 humans and over 1,000 DNN models on. Im- portantly, our task consists of a grand total of 20 different conditions: two baseline conditions with full color (RGB) images, and contour- filtered images; as well as two different sets of fragmented contour images with 9 sets of levels of fragmentation in each. Using this comprehensive experi- ment alongside our gigantic model set, we make several contributions:
Gestalt grouping (Wagemans et al., 2012; Biederman, 1987). the ability to combine object parts into wholes, has also seen rise in popularity, though experimental results are mixed. While some report that DNNs fail to capture principles of Gestalt grouping (Bowers et al., 2023; Malhotra et al., 2023), others report mixed or partial success (Biscione & Bowers, 2023; Pang et al., 2021; Lonnqvist et al., 2024). In the more naturalistic domain, Muttenthaler et al. (2023) studied model behavior using an odd-one-out task, finding model objective function and training dataset size to drive performance, while model scale and architecture had no effect.
1. We found that most models struggle on our task that is relatively easy for humans, especially in more chal- lenging settings. For example, humans still achieved 50% accuracy with only 35% of fragmented elements remaining, while models were barely above chance even with decoder adaptation.
2. Models that were larger and trained on more data per- formed better our task. For instance, GPT-4o's 's per- formance was near- saa--inditinguushabbe from humans; but this trend held across all models.
Contour integration in humans and primates. Contour integration in general has been studied behaviorally exten- sively in humans. There are several well-established hall- marks of contour integration, most important for us is the effect of contour alignment when individual fragments align, human performance substantially increases and de- tection becomes possible (Polat & Sagi (1994); Kovács & Julesz (1993); Roelfsema (2006); Elder & Zucker (1993); see also Wagemans et al. (2012); Biederman (1987)). Con- tour integration has also been studied extensively in animal models using electrode arrays to make direct neural record- ings of populations of neurons. Of particular note is primary visual cortex V1, where neurons respond strongly to arrange- ments of aligned fragments (Li et al., 2006; Kapadia et al., 1995; Bosking et al., 2000; 1997). In addition, evidence of contour integration has also been found in visual area V4 (Chen et al., 2014).
3. We found that models that are trained on more data had a larger integration bias (performance on directional segment stimuli minus performance on directionless phosphene stimuli), suggesting that contour integration is critical for human-like object recognition perfor- mance indeed, a larger integration bias translated into higher accuracy as well as better robustness.
4. We trained models on contour integration, and found that they not only acquired an integration bias, but also shape bias greater than that of shape-trained models.
5. Together, these suggest that contour integration un- derlies human like vision, and may be mechanism learned from large-scale data, rather than an inbuilt mechanism inherent to biological vision.
Contour integration in DNNs. While some of the earli- est noN-DNN models of contour integration were studied earlier (Li, 1998), classical contour integration was studied in DNNs by Linsley et al. (2018), who showed that tradi- tional convolutional neural networks struggle to learn to integrate line segments across the visual field in an abstract task. Inspired by horizontal connectivity in the primate brain, they developed horizontal gated recurrent unit net- work, which was able to learn and solve the contour inte-
2. Related work
Studies of visual behavior in DNNs. Most work study- ing visual behavior in DNNs has focused on a number of out-of-distributtin datasets examining different cases of human-mode alignment due to DNNs' tendency to exploit
2

---

Contour Integration Underlies Human-Like Vision
12%
16%
21%
27%
35%
46%
59%
77%
100%
Cantours RGB
Figure 2. Stimulus examples. Numbers refer to the number of fragments the stimulus relative to the 100% maximum. 100% is the maximum number of fragments placed along the object contours without overlap. Top: Phosphene stimuli ranging from 12% to 100% number fragments. Bottom: Segment stimuli. Right: Contour and RGB stimuli.
gration task. (Funke et al., 2021) took a different approach: they fine-tuned an ImageNet-trainedt ResNet-50 (He al., 2016) on line segment images with open and closed con- tours. They found that while the model did well on the task in-distribution, generalization proved catastrophically difficult.
as both the contour and full color (RGB) conditions. Stimuli were presented in ascending order of fragmentation starting from 12% up to 100% followed by contours and RGB. All stimuli were presented foveally spanning 8 degrees of visual angle. Stimuli were presented for 200ms, followed by 200ms 1/f noise mask. After the noise mask, partici- pants could select their response option from 12 categories arranged in a circle around the center of the screen. In to- tal, we report results for 50 participants, totalling 26,400 trials. For more details on experimental procedure and stim- ulus selection, see Appendix A.2. Additionally; 10 new participants performed additional control experiment to control for learning effects due to the ascending stimulus presentation order (details in Appendix A.3).
Our work differs from previous work on several critical axes. First, we design dataset that allows for a more detailed examination of model and human behavior than previous datasets, such as shape bias datasets (Geirhos al., 2018). This is due to the fact that we collected data to stimuli across many different conditions, and many different levels of fragmentation in our stimuli. Second, the scale of our model evaluation eclipses previous studies by an order of magnitude and enables us to quantitatively establish links that were not possible prior. Third, the combination of the dataset design and large model set allow us to analyze our data in novel ways. We are able quantitatively show that contour integration leads to improved object recognition, and better robustness to image perturbations.
Stimulus synthesis. For our RGB base images we used the BOSS dataset (Brodeur et al., 2010; 2014) which consists images of everyday
of high-quality background-extractt objects. We generated total of 19 different datasets (Roter- mund et al., 2024) from these images: contour-extracted images, as well as nine different levels of fragmentation for each of our two experimental conditions (directionless phosphenes and directional segments). The levels of frag- mentation ranged from 100% (meaning the maximum num- ber of elements we have in any image), down to 12% of that maximum following a log scale. We used 4 objects per category in experiment for each of our 12 object cate- gories, resulting in 432 fragmented stimuli for phosphenes and segments respectively. Combined with the RGB and contour stimuli, this resulted in a total of 528 stimuli per human participant. At the time of submission, these stimuli are not public and could therefore not have been used for model pre-training.
3. Methodology
Human psychophysical experiment. We recruited 50 human participants to take part in a 12-alternative forced choice object recognition task. All participants performed the task in controlled laboratory setting, with stimuli pre- sented on a high-quality 1920x 1080 pixel monitor. Upon arrival, the participants were sat in a darkened room where vi-
they were first verified for normal or corrected-to-norce sion using the Freiburg acuity test (Bach, 1996). Participants also performed a short language test with unrelated object categories, well as short set of 24 familiarization trials with object categories not included in our main experiment, intended to familiarize themselves with the response options and trial structure of the task. In the main experiments, par- ticipants were divided into two groups of 25 participants, with each group being presented either phosphene stimuli (Figure 2 top) or segment stimuli (Figure 2 bottom), as well
Model responses. To investigate whether models could do our task out-of- box, as well as whether their rep- resentations could support our task, we used Brain-Score (Schrimpf et al., 2020) to extract model responses in two different ways:
Zero-shot: To directly test DNNs' capability to perform the
3

---

Contour Integration Underlies Human-Like Vision
(b)
Humans
Models
distributions
Accuracy
6a-
59
Decodends
0.--
Decoder-ft
04
5A
044
62-
12-
02-
Zero-shet
shot
Chorke
Chance
125
35%
218
19%
49%
39% Ih 10th One.
12%
18%
25%
20H
30%
44h
55%
17%
100%
Ovet
12%
14%
25%
20
Mi
59%
1h% 1000
Cort AOO
Percentage Elements (log scale)
Percentage Elements (log scale)
Percentage Elements (log scale)
Figure 3. Human and model performance. Individual dots represent individual participants' or models' average performance, and error bars show 95% confidence intervals. Task chance performance is 8.33%. (a): Human object recognition performance. The x-axis shows different conditions (Figure 2), combined across phosphenes and segments. (b): Model categorization performances. Dark blue: Zero-shot models with responses mapped from ImageNet labels without additional fitting. Light blue: Linear decoder-fit models with additional 120 supervised trials for linear decoder within-condition. Violet: GPT-4o zero-shot performance. (c): The same data as in (b), visualized as distributions.
fragmented object recognition task in a human- comparale manner, we mapped model responses from ImageNet cate- gories (Russakovsky et al., 2015) to our 12 object categories zero-shot using a WordNet synset mapping (Geirhos et al., 2021). This was only possible for models that output Ima- geNet 1000-class labels. We also tested GPT-4o zero-shot, for which you find details in Appendix A.12.
ate statistical tests throughout. There are several reasons for this choice. In recent years there has been trend to move away from accuracy towards fine- ome-grained error-based metries (like error consistency, Geirhos et al. (2020b)). key motivation for this shift has been that is increasingly difficult find benchmarks on which humans outperform leading deep learning models. While these metrics are traordinarily valuable, they also come with downsides such as a large amount of noise in fitting estimation that requires up to hundreds of trials per condition a standard experi- mental setup. As such, analysis at the level of conditions, rather than stimuli, is unintuitively more difficult using these metries. Because the goal of our study tire range of behaviors across set of conditions, to rather increase the difficulty of our task to avoid ceiling effects, and estimate performance simply using accuracy.
Decoder-fit: For each of our 12 object categories, we first selected 10 ImageNet images from the corresponding Im- ageNet categories. We then removed backgrounds from these images (Gatis, 2023) and generated 120 novel frag- mented images for all percentage levels; 10 images per object category. We fit linear decoders on the penultimate layer activations.
explain the en- chose
Model selection. selection, Our model set consisted of several sources of models: 505 pre-trained models from the timm library (Wightman, 2019) and 23 pre-trained models from the taskonomy library (Zamir et al., 2018). In addition, we also added 514 models trained ourselves (see Appendix A.7 for details). These additional models ranged from very small networks trained a few hundred samples, to medium- sized models trained on ImageNet-21k. The largest pre- trained models from the timm library were trained on over 5 billion images. In total, we include 1,038 models from 13 architecture families and 18 datasets in our work, including Vision Transformers (Dosovitskiy et al., 2021), Convolu- tional Neural Networks (Liu et al., 2022), and many others, trained on datasets such as LAION-2B (Schuhmann al., 2022), CLIP (Radford et al., 2021), and ImageNet (Rus- sakovsky et al., 2015). For quick list of models analyzed in this study, as well as their architectures, see Table For more details about the models, including information about their training datasets and compute, see Appendix A.7.
Table 1. Number of models and subjects in the main experiments.
Architecture
N Zero-shot
N Decoder-fit
ViT
122
257
ConvNeXt
107
189
ResNet
75
179
EfficientNet
45
87
ResNeXt
37
39
Vix
26
30
SwinTransformer
26
34
Mobile
16
16
CORnet-S
15
30
AlexNet
15
30
Fist
14
14
RegNet
13
16
Tiny VIT
8
11
Total
621
1038
Model and human comparison. We computed model and human accuracy across conditions and perform appropri-
Humans
50+10 control

---

Contour Integration Underlies Human-Like Vision
(a)
t.
(b)
(c)
(d)
Decoder-fit (n=1038)
Highox
0.6
0.6
Reletive Importance
moan
0.4
0.4
0.4
humon
0.2
02-
02-
9000
38
10B 1005
IX B0OK 100M 10C8
IN 10 10m000n a1 10T80оT xoq FLOPs Training dataset size (Millions)
FLOPs
Training dataset size
2001
Figure Dataset scale and model compute explain performance. (a) and (b): Model performance on fragmented objects is explained moderately well by model compute measured in total floating point operations (FLOPs) per sample (r = 0.334, < 0.001, and very well by training dataset size (r = 0.814, p < 0.001). (c): Considering both variables together (visualized by multiplying them) results in smooth trend. (d): Dataset size and model compute per sample are more important for explaining model results than architecture. y-axis shows t-values from a multiple regression. To get the average effect of architecture, we averaged the absolute values of architecture t-values.
4. Results
Dataset size and model compute explain differences in model performance. To understand the individual differ- ences between the models, we studied whether model per- formance could be explained by three simple factors: model training dataset size, model compute per sample, and model architecture (Figure 4). In addition to the 528 pre-trained models in our selection, we also trained an additional 514 models primarily on the lower end of the training dataset size and model compute axes to investigate the full range of scaling performance across pre-existing as well- cell-controlled training diets. Since model performance across the board is substantially higher for decoder-fit models, we focus the rest of our analysis these models. For full details models and training diets, see Appendix A.7.
Humans substantially outperform models. We first an- alyzed human performance and compared it to the perfor- mance of a set of 528 pre-trained models from the timm and taskonomy libraries (Figure 3). Humans performed at ceiling accuracy on the contour and RGB conditions, simi- larly to most models. This is in contrast our experimental conditions where we fragmented the object contours- hu- man performance scaled approximately log-linearly (Fig- ure 3(a); accuracy = 0.29 log(x)) - 0.51, R² = 0.73, < 0.001) in the number of elements. While model per- formance followed a similar scaling function in the number of fragmented elements, performance scaled substantially slower than in humans, especially zero-shot (Figure 3(b,c): zero-shot accuracy 0.04 log(=) 0.03, R 0.23, p < 0.001; decoder-fit accuracy- 0.09 * log(x) (z) + 0.03, R = 0.39, p < 0.001). Of particular note are zero-shot models, which catastrophically failed on our task, with only few models reaching above-chance performance in general. However, GPT-40 stood as an outlier in zero-shot models, reaching near human levels of performance and being the best zero-shot model by a large margin.
Across our entire set of 1,038 models, we find that both model compute per sample (reported as floating point op- erations or FLOPs per input image), as well as the training dataset size (reported as the number of unique images in the training dataset of the model) are both important factors explaining model performance (Figure 4(a,b)). Training dataset size is especially important regressing training dataset size on model results on our task performance re- sults in a correlation of r = 0.814, explaining most of the variance in the data. Due to the high multicollinearity between FLOPs and training dataset size, we also show a simple multiplication of the two for visualization purposes (Figure 4(c)).
Decoder-fit models performed substantially above zero-shot models, with both larger slope and intercept than zero-shot models. However, while decoder-fit models' performance started at approximately the human level on the most dif- ficult condition (12% percentage elements), their perfor- mance on average did not scale the human level at the easiest fragmented condition (100% percentage elements). Although the overall distribution of pre-trained models did not align with the human distribution of results, there was individual variation in model results, with a small number of models substantially outperforming others.
The importance of architecture for performance. We considered the relative importance of model compute per sample, model training dataset size, as well as the average effect of architecture (Figure 4(d)), as t-values of a mul- tiple regression. We averaged the absolute values of the t-values of the effects of architecture families model performance, and observe that architecture average is
5

---

Contour Integration Underlies Human-Like Vision
relatively unimportant for contour integration performance on our task compared to model compute or training dataset size.
performance across our model set.
This is exactly what we found in Figure 6(b): the models with the largest integration biases are the most human- -ike in performance, and model average performance increased with integration bias. Together these suggest that contour integration is crucially related to human-like object recogni- tion regardless of the underlying architecture the computa- tion is implemented in.
Directional segments reveal contour integration in hu- mans and models. To analyze the degree of contour inte- gration caused by the effect of adding directional segments (Figure 2, bottom) instead of directionless phosphenes (Fig- ure top), we compared human and model performance on these two conditions. Results are shown in Figure 5.
Contour integration is learned from data. We tested whether the size of the model's training dataset directly cor- related with its integration bias across our 1,038 decoder-fit models (Figure 6(c)). The results revealed a strong positive correlation (r 0.594, p < 0.001), indicating that contour integration is mechanism models learn from the data dis- tribution automatically, rather than mechanism that must be hand-engineered in the architecture.
Humans exhibited clear preference for directional seg- ments over directionless phosphenes, with a large effect size (Cohen's s d = 1.96).. Given the overall poor average performance of models in the task in both the zero-shot and decoder-fit fit regimes compared to human performance, we were somewhat surprised to find that both types of model responses exhibited a human-like preference for directional segments, with statistically significant effect sizes (Cohen's d = -0.84; Cohen's 's d = - 1.55 respectively). This sug- gests that while models were not as performant on our con- tour integration task as humans, they share a similarity in how they solve the task.
This finding carries an important implication. It has tradi- tionally been thought that contour integration is largely a product of horizontal connectivity in primary visual cortex (Bosking et al., 1997; 2000; Kapadia et al., 1995). Here we demonstrate that not only are early horizontal connections not necessary, but that this mechanism can and is learned directly from large enough training dataset.
1.0
Cohen's = 11.6 0.001
0.8-
Contour integration leads to increased robustness. We selected a subset of 50 models from our set of models to evaluate on robustness to image perturbations (ImageNet- C-topl). The models were selected linearly based on per- formance on our fragmented object task. We find that the larger the integration bias the model had, the better its per- formance was against image perturbations (Figure 6(d); r= 0.671, p < 0.001).
conditions
Cohen's c - -1.55
23,09
Accuracy
0.8
0.4-
Cohen's = =0000 [m-11.41
0.2-
Chance
Humans
Zero-shot
Decoder-Fe fit
0.0
Contour integration leads to shape bias. We wanted to study whether it would be possible to directly train models to group elements, and whether that would induce both inte- gration bias and shape bias. To that end, we trained several ResNet-18 (He al., 2016) models ImageNet- 1k (Rus- sakovsky et al., 2015), and several different manipulations of ImageNet-Ik, 1k, training on the following data setups:
Phosphenes Segments
Phosohenes
Segments
Phosphenes
Segments
Figure 5. Human and model performance phosphenes and segments. Humans exhibited higher performance when presented with segments. rather than phosphenes. Dots show human and model means, and error bars are 95% confidence intervals.
IN only: ImageNet- 1k baseline with no additional images.
Contour integration explains human-level performance. To understand why larger models trained on more data do substantially better than smaller models trained on less data, we sought to analyze what we call integration bias: the performance difference between the segment task and the phosphene task (Figure 6(a)). Since model integration bias was smaller than that of humans' on average (Figure 5), and because models exhibited a wide range of performance levels largely explained by their training dataset size, we conjectured that these model-to-model performance differ- ences actually stem from the models' ability to integrate contours. In other words, if integration bias is a crucial driver of overall performance, should correlate with model
IN + Phosphenes:
ImageNet- 1k combined with grayscale images with directionless phosphenes.
ImageNet- Ik combined with both segment and phosphene stimuli.
IN Segments + Phosphenes:
In addition, we tested Shape-biased model (Geirhos et al., 2018). All of our fragmented object models were trained for 100 epochs using two 1100 GPUs per model, with a total batch size of 512. For more training and data preparation details, see Appendix A.9, and for additional controls on contour training, see Appendix Figure 12.
6

---

Contour Integration Underlies Human-Like Vision
(a)
0.5
(b)
30%
(C)
0.-
(d)
57%
Todu to getturbalicoe OrsgelennChett
0.-
20%
0.5
Integration blas
43%
Accuracy
10%
SEGMENT PHOSPHENE
0.2
57%
43%
02
r-0.694 (p=0.001)
0.594 (p=0..001
HUMAN INTEGRATION BIAS
-10%
(p=0.001)
=14%
0.0
-10%
Integration Blas
0%
10%
20%
30%
Training dataset size
10"
10"
10
0.00
Integration bias
0.03
0.12
0.13
0.8
Figure 6. Integration bias explains model performance. (a): We defined integration bias as the difference in performance between the segment condition and the phosphene condition. For example, model achieving 60% performance on segments and 50% performance on phosphenes would have 10% integration bias. (b): Light blue dots are individual decoder-fit fit models. Green lines are human 95% confidence intervals centered around the mean. Model integration bias correlated strongly with model performance. Darker color indicates larger model training dataset size. (e): Model training dataset size correlated with integration bias. (d): Model integration bias led to higher robustness as measured by ImageNet-C (topl accuracy). Included are 50 models selected based on performance on our task (Appendix A.))).
We found that while is possible to improve performance on fragmented objects by directly training models sim- ilar types of stimuli, it comes at a cost unlike previous studies (Geirhos et al., 2018), we were not able to simultanc- ously improve performance on full color images while also training models to exhibit improved robustness to contour fragmentation (Figure 7). Rather, models that improved performance fragmented stimuli (IN Segments, IN + Phosphenes, IN + Segments Phosphenes) also minor drops in full color image recognition (RGB). While this is expected, it also paves the way to study contour inte- gration mechanistically in smaller DNN models.
across evaluation of over 1,000 DNNs, they fail capture human behavioral performance, even when allowing for ad- ditional decoder- decoder-fitting trials. We found that the root cause of the model failure is an inability to integrate contours as measured by the difference in performance on images with directional segments and directionless phosphenes: models that had a larger integration bias also performed better on contour integration task in general. Our definition of integration bias builds vast body of evidence both in human behavior and neural data that shows the specificity of contour integration to directionally aligned line segments (Bosking et al., 2000; 1997; Roelfsema. 2006; Kovács & Julesz, 1993; Kapadia et al., 1995). We found that inte- gration bias arises from model training on large datasets models trained on more data exhibited larger integration bias. Finally, training models fragmented images led not only to integration bias, but also to a larger shape bias than shape-trained models (Geirhos et al., 2018). This allows for researchers to study the computations underlying human- like integration bias in DNNs, and to obtain a high model shape bias simultaneously with high integration bias, by training on fragmented objects.
What is surprising, however, that not only did our models acquire shape bias (Figure right; measured by Geirhos conflict accuracy), but their shape bias
et al. (2021) was higher than that of shape-bias trained ResNet-50 (Geirhos et al., 2018). Simultaneously, the integration bias acquired by the shape-biased model was barely half of the human equivalent. Taken together with evidence from neu- ral (Roelfsema, 2006; Kapadia et al., 1995) and psychophys- ical (Polat Sagi, 1994; Elder & Zucker, 1993) results, these findings point us conclusion: contour integration leads models to acquire shape bias.
Limitations. In this work, we focused on observational findings. We have made several efforts to remedy this: first, our model set covered much of the extant landscape of open access pre-trained vision models and, combined with mod- els we additionally trained ourselves, constituted what is the largest duman--odee comparison to date by approximately order of magnitude. Second, synthesized new per- turbed versions of ImageNet and trained models different
5. Discussion
Summary. This work presented a large-scale comparison of human and DNN performance on contour integration task. By carefully designing an experiment meant to specifi- cally test the failure points of DNN models, we found that

---

Contour Integration Underlies Human-Like Vision
Humans
1.
0.14
0.30
0.3
Accuracy
0.6
Integration Bias
0.10
Shape bias
0.20
0.4
0.05
Seg. + Phos.
0.30
Shape blazed
phoo..
Shape blosed
IN only
Phos.
0.7
only
Chance
0.0
12%
10%
21%
40%
59%
77%
100%
Cont.
RGB
0.09
0.50
Percentage Elements (log scale)
Models
Models
Figure 7. Model performance of models trained on ImageNet-Ik and fragmented ImageNets. All models were evaluated zero-shot. Left: Our trained model performance the fragmented object task. Middle: Individual models' integration bias. Right: Model shape bias as measured by the cue conflict dataset (Geirhos et al., 2021). Our models trained on phosphenes, and a combination all reach higher shape bias than the Stylized ImageNetttrraneed model (Geirhos al., 2018).
combinations of ImageNet and its perturbed counterparts.
improve contour integration performance, full color image object recognition performance decreased. Nevertheless, the models exhibited greater shape bias than even those trained specifically for while the shape- biased model did not have as large of an integration bias as models trained for contour integration. This demonstrates that shape bias emerges in models that also achieve contour integration. Human visual behavior astonishingly complex, and our work shows that some of that elusive complexity nevertheless emerges from large enough training datasets. In other words, findings challenge the notion that hand-engineered features or induc- tive biases are necessary to replicate even some of the most fundamental mechanisms of human-like visual processing.
Another limitation of our study was that we did not eval- uate leading vision-language models the same scale we evaluated pure vision models. We took steps to address this: we included several CLIP- models in our training set, and while CLIP-trained vision encoders are not joint vision-language models, their representations are neverthe- less guided by language. We also included GPT-4o in our model comparison, which we found to be the most human- like zero-shot model.
Finally, since we used pre-trained models in most of our evaluation, it is possible that some models may have been trained on images that are similar, but not identical, to the ones we used to test models. We took great care to avoid leakage to any models' training datasets: we synthesized the stimuli ourselves using stimulus rendered (Rotermund et al., 2024), and none of the stimuli we used in the study were published at the time of model evaluation.
We believe this highlights a potential way forward for build- ing human-like vision models. our study we saw that models, particularly zero-shot, fail replicate human be- havior catastrophically across the board. In addition, results show that the primary reason for this is integration bias, and particularly how it improves with dataset scale. This highlights an interesting possibility: perhaps the reason models fail to replicate human- behavior not because they are not large enough per se, but because their train- ing datasets have systematic differences human visual input (see e.g. Long et al. (2024)). One such example is the relative absence of occlusions in internet photographs photographers often prefer taking photos with better form and figure than the visual input that humans observe on a momentary basis (Leung et al., 2021), but other biases exist too (Torralba & Efros, 2011). Instead of attempting to build better human like models through architectural changes, our results highlight the possibility that in search of human-like models, must seek human-lite like training data.
Future directions. Our study presents an interesting out- look for the field of human vision modeling. On one hand, our results may give clues as to why many efforts model- ing the edge cases of human vision, such as Gestalt group- ing, have failed (Bowers al., 2023; Baker al., 2018; Malhotra et al., 2022). A task as simple contour integra- tion, believed to occur mainly in primates in primary visual cortex VI (Kapadia et al., 1995; Li et al., 2006), requires gigantic models trained on billions of images, as well as additional decoder trials to reach human performance. Fur- thermore, attempts to model human behavior directly by fitting relatively simple model with standard hyperparame- ters yielded mixed results, given that while we were able to
8

---

Contour Integration Underlies Human-Like Vision
Acknowledgements
e385, January 2023. ISSN 0140-525X, 1469-1825. doi: 10. 10.1017//0040555522003333
We thank Yingtian Tang for helpful discussions, and Marc Repnow for technical support.
Brodeur, M. B., Dionne-Dostics E., Montreuil, T., and Lep- age, M. The bank of standardized stimuli (boss), new set oft 480 normative photos of objects be used as visual stimuli in cognitive research. PLOS ONE, 5(5):1133, 05 2010. doi: 10.137 ho37://oomaal..oe0000007777
Impact Statement
Our findings challenge conventional assumptions about the role of architectural design in visual tasks, highlighting instead the critical role of large-scale data and specific biases like contour integration. These insights pave the way for more human-like and robust AI vision systems.
Brodeur, M. B., Guérard, K., and Bouras, M. Bank of Stan- dardized Stimuli (BOSS) Phase II: 930 New Normative Photos. PLOS ONE, 9(9):1-10, September 2014. doi: 10. 10.137//j 0tt.umm6al.po.0000005533333 Publisher: Public Library of Science.
References
Bach, M. The freiburg visual acuity automatii measurement of visual acuity. Optometry and Vision Science, 73(1):49-53, January 1996. doi: 10.1097/ 00006324- 199601000-00008
Cavanaugh, J. R., Bair, W., and Movshon, J. A. Na- ture and interaction of signals from the receptive field center and surround in macaque vI neurons. Jour- nal of Neurophysiology, 88(5):553--2446, 2002. doi: 10. 10.1152/jn.0009922000. URL
org/
Baker, N., Lu, H., Erlikhman, G., and Kellman, P. J. Deep convolutional networks do not classify based on global object shape. PLOS Computational Biology, 14((2):1-43, 12 2018. doi: 10.1337/jourmall 1006.100661...
11.1152/ ittp://n..00000 www.000662.0000
Chen, M., Yan, Y., Gong, X., Gilbert, C. D., Liang, H., and Li, W. Incremental integration of global contours through interplay between visual cortical areas. Neuron, 82(3): 682-694, 2014. doi: 10. ne0oo//.euro..2214...0
Beery, S., Van Horn, G., and Perona, P. Recognition in terra incognita. Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
Chen, X., Wang, X., Changpinyo, S., Piergiovanni, A., Padlewski, P., Salz, D., Goodman, S., Gryener, A., Mustafa, B., Beyer, I.., Kolesnikov, A., Puigcerver, J., Ding, N., Rong, K., Akbari, H., Mishra, G., Xue, L., Thapliyal, A., Bradbury, J., Kuo, W., Seyedhosseini, M., Jia, C., Ayan, B. K., Riquelme, C., Steiner, A., Angelova, A., Zhai, X., Houlsby, N., and Soricut, R. Pali: A jointly- scaled multilingual language-image model, 2023.
Biederman, I. Recognition-by-comoomeee bycoooonntsssss A theory of human image understanding. Psychological Review, 94 12):115-147, 1987. ISSN 1939-1471. doi: 10.1037/ 0033-295X 1..-2955.94.21155. Place: US Publisher: American Psychological Association.
Biscione, V. and Bowers, J. S. Mixed Evidence for Gestalt Grouping in Deep Neural Networks. Computational Brain & Behavior, 6(3):438-456, September 2023. ISSN 2522-087X. doi: 10. 00010077/42113--0300000666
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. 2009 IEEE conference on computer vision and pattern recognition, pp. 248 255, Icee. 2009.
Bosking, W. H., Zhang, Y., Schofield, B., and Fitzpatrick, D. Orientation selectivity and the arrangement of hori- zontal connections in tree shrew striate cortex. Journal of Neuroscience, (766)2212221222 1997. ISSN 0270-6474. doi: 10. 000.522/NEEUURSSC11.7-66622222
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An image is worth 16x16 words: Transformers for image recognition scale. In International Conference Learning Representations, 2021.
Bosking, W. H., Kretz, R., Pucak, M. L., and Fitzpatrick, D. Functional specificity of callosal connections in tree shrew striate cortex. Journal of Neuroscience, 20(6):2346 2359, 2000. ISSN 0270-6474, doi: 10. 10.1523/NEURROCC. 00-06-02346.20000
Elder, J. and Zucker, S. The effect of contour closure on the rapid discrimination of wwo-dimensional shapes. Vision Research, 33(7))981 1993. ISSN 0042-6989. doi:
Bowers, J. S., Malhotra, G., Dujmović, M., Montero, M. L.., Tsvetkov, C., Biscione, V., Puebla, G., Adolfi, F., Hum- mel, J. E., Heaton, R. F., Evans, B. D., Mitchell, J., and Blything, R. Deep problems with neural network models of human vision. Behavioral and Brain Sciences, 46:
Fang, A., Jose, A. M., Jain, A., Schmidt, L., Toshev, A., and
Shankar, Data filtering networks, 2023.

---

Contour Integration Underlies Human-Like Vision
Funke, C. M., Borowski, J., Stosio, K., Brendel, W., Wallis, T. S. A., and Bethge, M. Five points check when comparing visual perception in humans and machines. Journal of Vision, 21(3):16-16, 03 2021. ISSN 1534- 7362. doi: 10. 10.1167/jov.2133.16
Hermann, K., Chen, T., and Kornblith, S. The origins and prevalence of texture bias in convolutional neural networks. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Infor- mation Processing Systems, volume 33, pp. 19000-19015. Curran Associates, Inc., 2020.
Gadre, S. Y., Ilarco,, G., Fang, A., Hayase, J., Smyrnis,
G. Nguyen, T., Marten, R., Wortsman, M., Ghosh, D.,
Kapadia, M. K., Ito, M., Gilbert, C. D., and Westheimer, G. Improvement in visual sensitivity by changes in local context: Parallel studies in human observers and in vl of alert monkeys. Neuron, 15(4))843-885) 1995. ISSN 0896-6273, doi: https://8b.or////000000066666669 90175-2.
Zhang, J. Orgad, E., Entezari, R., Daras, G., Pratt, S., Ramanujan, V., Bitton, Y., Marathe, K., Mussmann, S.,
Vencu, R., Cherti, M., Krishna, R., Koh, P. W. Saukh, O., Ratner, A., Song, S., Hajishirzi, H., Farhadi, A., Beau- mont, R., Oh, S., Dimakis, A., Jitsev, J., Carmon, Y., Shankar, V., and Schmidt, L. Datacomp: In search of the next generation of multimodal datasets. arXiv preprint arXiv:2304. 14108, 2023.
Kovács, I. and Julesz, B. A closed curve is much more than an incomplete one: effect of closure in fguur-grrunde segmentation. Proceedings of the National Academy of Sciences, 90(16),7495-748 1993. doi: 10. 10.1073/pnas 10.16.7495.
Gatis, D. rembg: A tool to remove image background using pre-trained deep learning models. https: //g.tub..... com/ comm/danieei lgatis/////sss rembg, 2023. Accessed: 2023- 04-15.
Krizhevsky, A., Sutskever, I., and Hinton, G. E. Imagenet classification with deep convolutional neural networks. In Pereira, F., Burges, C. J. C., Bottou, L., and Weinberger, K. Q. (eds.), Advances in Neural Information Process- ing Systems 25, pp. 1097-1105 105. Curran Associates, Inc., 2012.
Geirhos, R.,, Rubisch, P., Michaelis, C., Bethge, M., Wich- mann, F, A., and Brendel, W. ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. September 2018.
Geirhos, R., Jacobsen, J. H., Michaelis, C., and et al. Shorteut learning in deep neural networks. Nature Ma- chine Intelligence, 2:665-673, November 2020a. doi: 10. 10.1088|<42225-00000005557
Kubilius, J., Schrimpf, M., Hong. H., Majaj, N. J., Rajaling- ham, R., Issa, E. B., Kar, K., Bashivan, P., Prescott-Roy, J., Schmidt, K., Nayebi, A., Bear, D., Yamins, D. L. K., and DiCarlo, J. J. Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs. In Wallach, H., Larochelle, H., Beygelzimer, A., D' Alché-Buc, F., Fox, F., and Garnett, R. (eds.), Neural Information Pro- cessing Systems (NeurIPS), pp. 12785-12796 Curran Associates, Inc., 2019.
Geirhos, R., Meding, K., and Wichmann, F. A. Beyond accuracy: quantifying trial-by-trial behaviour of cnns and humans by measuring error consistency. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.). Advances in Neural Information Processing Sys- tems, volume 33, pp. 13890-13902. Curran Associates, Inc., 2020b.
Leung, B., Ho, C.-H., Persekian, A., Orozco, D., Chang, Y., Sandstrom, E., Liu, B., and Vasconcelos, N. Oow1500: Overcoming dataset collection bias in the wild, 2021.
Geirhos, R., Narayanappa, K., Mitzkus, B., Thieringer, T., Bethge, M., Wichmann, F. A., and Brendel, W. Partial success in closing the gap between human and machine vi- sion. In Advances in Neural Information Processing Sys- tems, volume 34, pp. 23885 23885-23899. Curran Associates, Inc., 2021.
Li, W., Piëch, V., and Gilbert, C. D. Contour saliency in primary visual cortex. Neuron, 50(6))951-99) 962, 2006. doi: 10. www.:iijnnerron22000.44444
Li, Z. A neural model of contour integration in the primary visual cortex. Neural Computation, 10(4):903-944) 1998. doi: 10. 10.11622089977666830000111113111 URL https:// doi
Goyal, P., Caron, M., Lefaudeux, B., Xu, M., Wang, P., Pai, V., Singh, M., Liptchinsky, V., Misra, I., Joulin, A., and Bojanowski, P. Self-supervised pretraining of visual features in the wild, 2021.
Linsley, D., Kim, J., Veerabadran, V., Windolf, C., and Serre, T. Learning long-range spatial dependencies with horizontal gated recurrent units. In Bengio, S., Wal- lach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018
He, K., Zhang, X., Ren, S., and Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recog- nition, CVPR '16, pp. 770-778. IEEE, June 2016. doi: 10.1109/CVRR.2016.90 URL http: httLttttp//eeep/////e /ieeexppereee leee.org/ weaeee.rr//cccmme///88800
10

---

Contour Integration Underlies Human-Like Vision
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. Swin transformer: Hierarchical vision transformer using shifted windows, 2021.
Oquab, M., Darcet, T., Moutakanni, T., Vo, H. V., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., Howes, R., Huang, P.-Y,, Xu, H., Sharma, V., Li, S.-W., Galuba, W., Rabbat, M., As- sran, M., Ballas, N., Synnaeve, G., Misra, I., Jegou, H., Mairal, J., Labatut, P., Joulin, A., and Bojanowski, P. Dinov2: Learning robust visual features without supervi- sion, 2023.
Liu, Z., Mao. H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., and Xie, S. A convnet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
Long, B., Xiang, V., Stojanov, S., Sparks, R. Z., Yin, Z.,, Keene, G. E., Tan, A. W. M., Feng, S. Y., Zhuang, C., Marchman, V. A., Yamins, D. L. K., and Frank, M. C. The babyview dataset: High-resolution egocentric videos of infants' and young children's everyday experiences, 2024.
Pang. Z., O'May, C. B., Choksi, B., and VanRullen, R. Predictive coding feedback results in perceived illusory contours in a recurrent neural network. Neural Networks, 144: 164-17,, 2021. ISSN 0893-6080. doi: https://doi. 111.1:.011/jj..ueet....
Polat, U. and Sagi, D. The architecture of perceptual spatial interactions. Vision Research, 34(1):73-78, 1994, ISSN 0042-6989, doi: https//dbi.rg1110011060002222220 90258-5.
Lonnqvist, B., Wu, Z., and Herzog, M. Latent noise seg- mentation: How neural noise leads to the emergence of segmentation and grouping. In Forty-fire first Interationall Conference on Machine Learning. 2024.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. Learning transferable visual models from natural language supervision. In Meila, M. and Zhang, T. (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 8748 8763. PMLR, 18 24 Jul 2021.
Loshchilov, I. and Hutter, F. Decoupled weight decay reg- ularization. In International Conference on Learning Representations, 2019.
Malhotra, G., Dujmović, M., and Bowers, J. S. Feature blindness: A challenge for understanding and modelling visual object recognition. PLOS Computational Biology, 18(5):1-27, 05 2022. doi: 10. 1/j/ra/aaaalll 100.000 1009572.
Malhotra, G., Dujmović, M., Hummel, J., and Bowers, J. S. Human shape representations are not an emergent prop- erty of learning to classify objects. Journal of Experimen- tal Psychology: General, 152(12):3380-3402 2023. doi:
Radosavovic, I., Kosaraju, R. P., Girshick, R., He, K., and Dollár, P. Designing network design spaces. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 00428-10436, 2020.
10.1037/xge0001440.
Ridnik, T., Ben-Baruch, E., Noy, A., and Zelnik-Manor, L., ImageNet-21KK Pretraining for the Masses. June 2021.
Marques, T., Schrimpf, M., and DiCarlo, J. J. Multi-scale hierarchical neural network models that bridge from sin- gle neurons in the primate primary visual cortex object recognition behavior. bioRxiv, 2021. doi: 10.1101/2021. 03.01 433495. URL https: / /ooi. 00gg/10.11111 2021. 03 01. 433495. Preprint.
Roelfsema, P. R. Cortical Algorithms for Perceptual Group- ing. Annual Review of Neuroscience, 2923222332 2006. doi: 10.1 144///mnurr..nuuuo299.000 112199 12939.
Rotermund, D., Scialom, E., Repnow, M., Herzog, M., and Ersst, U, davrot/percept simulator 2023: V1.0.0 (neuro- prosthesis), April 2024.
Mehrer, J., Spoerer, C. J., Jones, E. C., Kriegeskorte, N., and Kietzmann, T. C. An ecologically motivated image dataset for deep learning yields better models of human vision. Proceedings of the National Academy of Sciences, 118(8), 2021. doi: 10. 10.1073/pnas..00 201 1417118.
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A. C., and Fei-Fei, L. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3)2111-52, 2015. doi: 10.10077s11263--15--00
Mehta, S. and Rastegari, M. Mobilevit: weigh, general-purpose, and mobile-friendly vision transformer. In International Conference on Learning Representations, 2022.
Schrimpf, M., Kubilius, J., Lee, M. J., Ratan Murty, N. A., Ajemian, R., and DiCarlo, J. J. Integrative Benchmark- ing to Advance Neurally Mechanistic Models of Human Intelligence. Neuron, 108(3):413-44) November 2020. ISSN 0896-6273. doi: 10. (2.1016/j.neuror aeeuron.202..00 2020.07.040.
Muttenthaler, L., Dippel, J., Linhardt, L., Vandermeulen, R. A., and Kornblith, S. Human alignment of neural network representations. In The Eleventh International Conference on Learning Representations, 2023.

---

Contour Integration Underlies Human-Like Vision
Schuhmann, C. and Beaumont, R.
Laion-
Wagemans, Elder, J. H., Kubovy, M., Palmer, S. E., Peter- son, M. A., Singh, M., and der Heydt, R. A Century of Gestalt Psychology in Visual Perception I. Perceptual Grouping and Figure-Grount Organization. Psychologi- cal bulletin, 133(6)): 1120))217722 November 2012. ISSN 0033-2909. doi: 10.1037/a002933..
https:/ htt/aaaann... Lon.ai/blog. Laion-aae aestheiiss 2022. Accessed: 2025- 01-30.
aesthetics.
Schuhmann, C., Vencu, R., Beaumont, R., Kaczmarezyk,
R., Mullis, C., Katta, A., Coombes, T., Jitsev, J., and Komatsuzaki, A. Laion-400m: Open dataset of clip- filtered 400 million image-text pairs, 2021.
Wightman, R. Pytorch image models, httpt:://ttstthiaa com rwi rww wtt.://www/ptoo.ot-mmoeee--ooooo 2019.
Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C. W., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., Schramowski, P., Kundurthy, S. R., Crowson, K., Schmidt, L., Kaczmarczyk, R., and Jitsev, J. LAION-5b: An open large-scale dataset for training next generation image-text text models. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022.
Wu, K., Zhang, J., Peng, H., Liu, M., Xiao, B., Fu, J., and Yuan, L. Tinyvit: Fast pretraining distillation for small vision transformers, 2022.
Xie, S., Girshick, R., Dollár, P., Tu, Z., and He, K. Aggre- gated residual transformations for deep neural networks, 2017.
Xu, H., Xie, S., Tan, X. E., Huang, P.-Y., Howes, R., Sharma, V., Li, S.-W., Ghosh, G., Zettlemoyer, L., and Feichten- hofer, C. Demystifying clip data. 2023.
Singh, M., Gustafson, L., Adcock, A., de Freitas Reis, V., Gedik, B., Kosaraju, R. P., Mahajan, D., Girshick, R., Dollár, P., and van der Maaten, L. Revisiting weakly su- pervised pre-training of visual perception models, 2022.
Yalniz, I. Z., Jégou, H., Chen, K., Paluri, M., and Maha- jan, D. Billion-scale semi-supervised learning for image classification, 2019.
Tan, M. and Le, Q. EfficientNet: Rethinking model scaling for convolutional neural networks. In Chaudhuri, K. and Salakhutdinov, R. (eds.), Proceedings of the 36th Inter- national Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 6105- 6114. PMLR, 15 Jun 2019.
Zamir, A. R., Sax, A., Shen, W. B., Guibas, L., Malik, J., and Savarese, S. Taskonomy: Disentangling task transfer learning. In 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2018.
Thomee, B., Shamma, D. A., Friedland, G., Elizalde, B., Ni, K., Poland, D., Borth, D., and Li, L.-J. Yfcc100m: the data in multimedia research. Communications of the ACM, 59(2):64-73, January 2016. ISSN 1557-7317. doi: 10.1145/2812888
Torralba, A. and Efros, A. A. Unbiased look at dataset bias. In CVPR 2011, pp. 1521-1528, 2011. doi: 10.1109) EVPR.2011.5995320
Tu, Z., Talebi, H., Zhang, H., Yang, F., Milanfar, P., Bovik, A., and Li, Y. Maxvit: Multi-axis vision transformer. ECCV, 2022.
Vasu, P. K. A., Gabriel, J., Zhu, J., Tuzel, O., and Ranjan, A. Fastvit: fast hybrid vision transformer using structural reparameterization, 2023.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. u., and Polosukhin, I. At- tention is all you need. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
12

---

Contour Integration Underlies Human- Vision
ImageNct
100
80
67. p00000000
-0.10
-0.05
0.00
0.05
0.10
0.15
0.20
Integration
Figure Integration bias explains ImageNet classification performance. Correlation between ImageNet performance (topl) and integration bias.
A. Appendix
A.1. Integration bias predicts classification accuracy
We evaluated model performance on ImageNet and found the same results as we did on ImageNet-C (Figure 4). Integration bias predicts performance across tasks, from fragmented stimuli (Figure 6a) to ImageNet (r 0.67, 0.01).
13

---

Contour Integration Underlies Human-Like Vision
A.2. Human experiment category selection
We included 12 categories in our task: truck, cup bowl binoculars glasses. hat pan, sewing machine, shovel. banana. booz, aamp The object category selection was done the basis of pi- lot experiment, where 46 different in-lab participants performed free-naming trials of similar objects without strict time limit. The 12 categories tested here all reached at least 90% free-name naming agreement among the pilot participants.
10
0.8
A.3. Human control experiment
To control for whether our experimental results could have been influenced by any human unsupervised learning or habituation during the experiment, we ran an additional control experiment. We recruited 10 human participants (5 females; mean age 21.3 years, a = 3.31). We followed the same experimental method- ology as in the main experiment, splitting the participants into two equal groups of segments and phosphenes. The only difference was that these participants only performed the condition with 100% fragments alongside the contour and RGB conditions, instead of all conditions from 12% to 100%. We found no difference in pilot participants' performance in the 100% condition compared to the main experiment, ruling out the possibility of unsupervised learning or habituation. Results are shown in Figure
0.6
0.4
0.2
0.0
100
contours
RGB
Percentage elements
A.4. Human performance scaling across conditions
We analyzed the difference how performance across the segments and the phosphenes conditions scales. We found little difference (phosphenes vs segments) in humans- the difference in performance in these Iwo cases is mostly a shift of intercept instead of change in slope across the number of elements (figure ??), though while the effect of difference in slopes is small, it is still technically significant (t = 2.87. < 0.01). See Figure 10 for details. This means that the directionality of the segments only slightly affects performance positively over the directionless prosphenes as element density increases.
Figure Human control experiment. Individual dots represent individual par- ticipants
A.5. Integration is not predicted by receptive field size alignment
We evaluated the same subset of models as in Figure 6 on two Brain-Score benchmarks that test for the similarity of the effective receptive field size of model to a primate counterpart. These are the Grating summation field (GSF) and surround diameter (Marques et al., 2021; Cavanaugh et al., 2002) benchmarks measured in Macaque V1. See Figure short, we do find statistically significant relationship between either measure of receptive field size similarity and fragmented object recognition accuracy (GSF 0.2655, p 0.0653; surround diameter 0.276, 0.055)
A.6. Contour-trained models do not achieve high shape bias or integration bias
As in Figure we also trained additional control models investigate whether contour-based models, and the distribution shift in image statistics caused by that, could be at cause for the results in Figure 7. We trained the following models:
IN only: ImageNet-1 1k baseline with no additional images.
Contours only: Contour-extracted grayscale images.
IN + Contours: ImageNet-Ik 1k combined with contour-extractoe images.
IN Binary contours:
ImageNet- 1k combined with dataset where we use Gaussian filter with Otsu thresholding contour-extracted images that binarizes contours in the image.
IN Segments: ImageNet-Ik combined with grayscale images with directional segments.
IN + Phosphenes: ImageNet- 1k combined with grayscale images with directionless phosphenes.
14

---

Contour Integration Underlies Human-Lik, Vision
Humans
0.8
Phosphenes
0.6
Semment
0.4
0.2
Chance
0.0
12%
16%
21%
27%
35%
46%
59%
77%
100%
Cont.
RGB
Percentage Elements (log scale)
Figure 10. Human accuracy split by condition (phosphenes: light green; segments: dark green).
GSF
Surround Diameter
1.0
Alignment
0.8
0.6
0.4
0.2
0.1
0.2
0.3
0.4
0.5
0.1
0.2
0.3
0.4
0.5
Accuracy
Figure Model performance correlated alignment in benchmarks: the Grating summation field (GSF) and surround diameter (Marques et al., 2021; Cavanaugh et al., 2002).
15

---

Contour Integration Underlies Human-Like Vision
IN Segments Phosphenes: ImageNet-Ik 1k combined with both segment and phosphene stimuli.
We found that the effect of training solely on contours is minimal to non-existent, and thus the shift in image statistics does not account for the observed differences in integration and shape biases (Figure 12).
1-
Models
only
ony
0.25
0.8
0.20
IM Prospheres
IM Segnert: Phosphenes
Integration Blas
Accuracy
0.6
+-
Shape-2used
0.15 H..Humass
Shape blas
0.25
0.00
0.20
0.2
3.5-
Chance
0.55
0.0
Percentage Elements (log scale)
35%
59%
100%
RGB
0.00
Models
0.20
Models
Figure 12. Model performance of models trained on ImageNet-1k and fragmented ImageNets. All models were evaluated zero-shot. Left: Our trained model performance the fragmented object task. Middle: Individual models' integration bias (hars matched in order color to left; first two bars show only and Contours only, which are both 0). Right: Model shape bias as measured by the cue conflict dataset (Geirhos al., 2021). Our models trained phosphenes, segments, and combination all reach higher shape bias than the Stylized ImageNet-tranned model (Geirhos et al., 2018)
A.7. Further model details
Pre-trained model selection. We include some statistics on our model selection. Table 2 includes information about model averages. N denotes the number of models or participants analyzed, Dataset, denotes the average number of samples in the model architecture family's training dataset. FLOPs,, denotes the average compute per sample used by the model architecture. Acc.,. denotes the average performance of models or participants in the group, and Acc. max denotes the maximum any individual model of participant achieved across conditions.
The full list of model architecture families we used is: Vision Transformer (Vaswani et al., 2017; Dosovitskiy et al., 2021); ConvNeXt (Liu et al., 2022); ResNet (He et al., 2016); EfficientNer (Tan & Le, 2019); ResNeXt (Xie et al., 2017); Max ViT (Tu et al., 2022); Sranfraasformer (Liu et al., 202 I); MobileViT (Mehta & Rastegari, 2022); CORnet-S (Kubilius et al., 2019); AlexNet (Krizhevsky et al., 2012); FastVIT (Vasu et al., 2023); RegNet (Radosavovic al., 2020); Tiny (Wu al., 2022).
Model training details for main analyses. In addition to pre-trained model set from timm (Wightman, 2019) and taskonomy (Zamir et al., 2018), we trained set of 514 models from multiple architecture families. The architectures we trained included ResNet-18. 18, 34, 50, 101, 152 (He et al., 2016); ViTT, S, B, L (Dosovitskiy et al., 2021); ConvNeXIT, S, B, L (Liu et al., 2022); EfficientNet-BO. 1, 2 (Tan & Le, 2019); CORnet-S (Kubilius et al., 2019); AlexNet (Krizhevsky et al., 2012).
We trained these datasets using three datasets: ImageNet- Ik (Russakovsky et al., 2015). ImageNet-21k (Ridnik et al., 2021). and EcoSet (Mehrer al., 2021). 1. We trained models full datasets, well subsets of the datasets ranging from ==500 training samples to the full dataset. For graphical illustration of all training dataset sizes, see Figure 13. ConvNeXt and VIT models were trained using the original training recipes suggested by the original authors. All other models were trained for 100 epochs using a batch size of 512 with the SGD optimizer with cosine decay learning rate. The learning rate started at 0.1 and warmed up over epochs, with a weight decay of 10 4. The horizontal flip and random resized crop augmentations were used.
16

---

Contour Integration Underlies Human-Like Vision
Table Summary of model dataset, compute, and performance on our task. All conditions, including RGB and contours included. N denotes the number of models or participants analyzed, Dataset, denotes the average number of samples in the model architecture family's ' training dataset. FLOPsy, denotes the average compute per sample used by the model architecture. Acc,, denotes the average performance of models participants in the group, and Acc. max denotes the maximum any individual model of participant achieved across conditions.
Zero-shot
Decoder-fit
Architecture
N
FLOPSp
Dataset,
Acc.,.
AcC.max
N
FLOPs|,
Dataset,
Acc.yu
Acc. max
Human
50
0.59
0.73
VIT
122
6.4 1020
1.2 x 10a
0.16
0.36
257
1.2 10
1.7 X 10
0.34
0.54
ConvNeXt
107
6.4 x 10th
1.1 x 10a
0.17
0.36
189
6.1 x 10
1.2 x 10a
0.33
0.55
ResNet
75
1.2 x 10T0
3.3 X 107
0.12
0.18
179
1.3 x 1000
3.3 x 107
0.31
0,45
EfficientNet
45
1.6 x 109
7.4 x 10c
0.12
0.18
87
1.6 x 109
7.4 10
0.29
0.39
ResNeXt
37
3.4 10TH
8.1 107
0.18
0.25
39
3.7 10
8.4 x 107
0.39
0.46
MaxViT
26
1.7 X 10T1
1.3 X 10a
0.20
0.24
30
1.6 x 10S1
1.4 10t
0.40
0.46
SwinT
26
5.1 10
8.0 X 107
0.20
0.23
34
5.4 10
9.4 10?
0.43
0.48
VibileVIT
16
1.0 10TH
9.7 X 10t
0.17
0.19
16
1.0 x 10
9.7 X 10t
0.33
0.36
CORnet-S
15
3.3 X 1040
5.3 X 107
0.11
0.17
30
3.3 x 10
5.3 X 107
0.29
0.37
AlexNet
15
1.4 10
6.1 X 107
0.12
0.16
30
1.4 X 109
6.1 X 107
0.30
0.33
FastVIT
14
6.7 X 109
1.9 X 107
0.19
0.21
14
6.7 X 109
1.9 X 107
0.39
0.42
RegNet
13
9.5 1000
1.2 X 10a
0.19
0.23
9.2 10
1.3 x 10
0.39
0.46
TinyViT
8
1.2 10
1.5 107
0.18
0.20
11
1.0 10
1.7 107
0.41
0.45
Total
621
1038
17

---

Contour Integration Underlies Human-Like Vision
A.8. Additional details training datasets.
Our of models from the timm library (Wightman, 2019) include models trained on various datasets. In addition, include 23 pre-trained models from the taskonomy library (Zamir et al., 2018). A full list of timm datasets used in study, in addition to their references, can be found in Table 3. We also include information the number of models from each training dataset, including our own, which can be found in Figure 13. Models trained on our datasets are mix of ImageNet-Ik Ik, ImageNet-21I Ik, and EcoSet models.
18

---

Contour Integration Underlies Human-Like Vision
Dataset
Citation
Size
ImageNet-1k 1kk
(Deng al., 2009)
1.28M
Ecoset
(Mehrer et al., 2021)
1.5M
1/12/Net-12/21k
(Ridnik al., 2021)
13.6M
YFCC100m
(Thomee et al., 2016)
100M
LVD142M
(Oquab et al., 2023)
142M
LAION-400M
(Schuhmann et al., 2021)
400M
OpenAI
(Radford et al., 2021)
400M
LAION-A
(Schuhmann Beaumont, 2022)
939M
IG-IB
(Yalniz al., 2019)
940M
Datacomp
(Gadre et al., 2023)
1.0B
SEER
(Goyal et al., 2021)
1.0B
WebLI-1B
(Chen et al., 2023)
1.0B
600M-2B
(Schuhmann et al., 2022)
2.0B
DFN2B
(Fang et al., 2023)
2.0B
LAION-2B
(Schuhmann al., 2022)
2.2B
MetaCLIP
(Xu al., 2023)
2.5B
SWAG
(Singh et al., 2022)
3.6B
DFNSB
(Fang et al., 2023)
5.0B
Table 3. Model training datasets used our study (M = million, billion).
19

---

Contour Integration Underlies Human-Like Vision
600M-26
70
? in1k: 245
dfn2b
laiona
swag
yfcc100m
dfnSb
ig1b
seer
Number of models
50-
webli
datacompxi
Ivd142m
2pt5b
lalon-400M
30
in12k
laion2b
in21k
inw21k
in22k
in1k
10
10a
10
10
10
10.00
Dataset size (log scale)
Figure Training datasets counts.
A.9. Training models on fragmented ImageNet
Fitting Data Preparation. As first step, we use the buckground-removy library (Gatis, 2023) to remove the background from ImageNet images. Next, employ the tool from (Rotermund al., 2024) convert the Background-removed RGB images into contour images. We then the same tool (Rotermund al., 2024) generate two additional transformations: segmented objects and phosphene images. To obtain binary contours, we first employ 3 3 Gaussian filter and Otsu thresholding on contour images. During this process, subset of ImageNet images could not be processed. Consequently, we ended up with 1240077 training phospheee//eegmen/commmee images out of the original 1281167.
Joint Training. We train multiple instances of ResNet-18 18 (He et al., 2016) on various combinations of four datasets: ImageNet, ImageNet-contours, SmageNet-phospheet and Imagenet-seemmens. We use the standard ResNet-18 architecture without modifications, and optimize the following multi-task objective:
C = A1 LimageNet + Ceoooutours + AaCphosphenes + X4 C4
where each X; set to when the corresponding dataset is included in the training process.
During joint training on multiple datasets, we first select batch of images and load the variations of those that A 0. For instance, if the model is trained on base ImageNet and contours, load batch of RGB images and their contour counterparts. For phosphene and segment variations, we only use the 100% condition, and load the image accordingly.
Except the ImageNet-trained variant, which is trained using SGD, all model variants are trained with the AdamW (Loshchilov Hutter, 2019) optimizer using a lcarning rate 10-3, a weight decay schedule. We employ linear warm-up for the first epochs and train for a total 100 epochs. The batch size 512, and standard ImageNet augmentations (random resized cropping and horizontal flipping) are applied throughout training.
10-?, and a cosine learning rate decay
A. Model set for joint behavioral analysis
In Section 4 we analyzed models on ImageNet-C-topl, and in Appendix A. we evaluated the same models on ImageNet- top1. These models were selected primarily based on performance on our fragmented object task. The full procedure for selection was follows:
We first chose small set of common and powerful architecture families: VIT, ConvNeXt, Swin, CvT, EfficientViT,
20

---

Contour Integration Underlies Human-Like Vision
ConvNeXtv2.
We then select 20 models from the selected architectures linearly based on their scores on our fragmented object recognition zero-shot task.
We finally add the best model for any architectures that were not produced above, resulting in a total of 22 models in the first batch.
We also included in our analysis an additional 28 models trained ourselves from [citation removed for anonymity; relevant details in Appendix A.7], resulting in a total of 50 models analyzed jointly on our task, and common neural and behavioral tasks.
A.11. Stimulus synthesis
We implemented stimulus synthesis using (Rotermund et al., 2024), which implements the following algorithm to filter image Czz, y, :
(1) Go(a,y,9) = =exp(- x² +2² -x²
cool 2n (cccmnttssing) - -1) - co cos 1)
(2)
202
(2) C(I,3,9) = (oa.)) * 17 (2, + [Go * (2 v.:p)
(3)
Where
exp(ex2p202 -2 =x=exp(-
C0 = exp
22()))))
The convolution operation is implemented via a 2D Fast Fourier Transform, and / (z, y) is the grayscale image pixel value at (z, y). Thus, the resulting contour image C(2, y, sp) contains the value of the local contour of orientation yp at position (z, y). We set the spatial filter size to = 0.06 aredeg and the spatial scale to À = 0.12 arcdeg.
For the placement of phosphenes and segments, we place elements on the contours of the object preferentially depending on the strength the contour and its directionality. We use this algorithm both for our experimental stimuli, as well as ImageNet-Ii Ik images. For ImageNet-Ile 1k images, we also perform background removal using rembg (Gatis, 2023) before applying this algorithm.
A.12. GPT-4o experimental methodology
GPT-4o (checkpoint: message:
got 2ot-4o-2024-05-11 was initialized with temperature=0.0 0 and presented with a system
You are invited participate in a research project. The research project aims study visual perception, i.c. the processing of visual information in participants. In experiments, visual stimuli will be presented you images. Your task will be evaluate some properties of these stimuli and to give your answer using the provided answer options only. We will measure your behavioral responses (accuracy of your responses, correctness, etc.). Answer as truthfully as possible, and if you are not sure, make your best guess.
Following that, GPT-4o received exactly the same instructions as humans. Experimental procedure followed human experimental procedure exactly. messages was kept in shared message history.
A.13. Image background color control experiment.
To test whether the specific image color statistics affect model performance cause difficulties, we tested small subset of models on red background images. We performed t-test to test for the difference in the group performance of the fragmented conditions and found that there was no difference model accuracy under these two conditions (t = 0.323, = 0.749). This shows that the specific model training distribution not the reason for model performance here, and that the results are robust against a change of background (Figure 155).
21

---

Contour Integration Underlies Human-Like Vision
5ic mane
Back silkgruus
las
Figure Red background control. (Left): Examples of images with red, instead black, background. (Right); Results show no sensitivity to the background color.
A.14. Differences between ViT and Convolutional architectures.
We conducted several analyses comparing transformers and CNNs. To make results comparable, we restricted the analysis to only models trained on ImageNet-lk.
We found that the overall performance is no different (CNNs: 32.49% on average, ViTs: 31.24% on average, t-test of means p = 0.1108: not significant). Integration biases between transformers and CNNs are comparable: ViT (7.8%) and CNN (10.1%) are not statistically significantly different 2.45, p = 0.016) at the 0.01 criterion.
The way model performance scales across the number of elements the same across models and across conditions: only condition across all conditions is different between CNNs and ViTs (the 16 16-phosphenes condition), while the differences between the rest are statistically non-significant.
6.14
Transformar
Vision Transformer
CNN
0.12
Score
Score
0.04
0.02
0.00
las
overageEng
averageEng
averageEng
12-phasuhe 16 (iphosphe
21 2--phosphe
phenshe 27 T-phoshhe
4b phosphe 5P 7/ ptosphe
100 phosph
12. D-semmnnt
16. segment
21 segnient
27 seeeent
35 degmenn
segment
59 segnert
77 i-senment
100 contoursfa
rgbEnginee
Benchmark
Benchmark
Figure Vit vs CNN analysis. (Left): Across models trained ImageNet, VITs and CNN do not exhibit differences across any measures. A: Integration bias. B: Average performance (both conditions). C: Average performance (phosphenes). D: Average performance
(segments). (Right): Performance scaling across percentage elements.
22


Reviews:
[{"review_id": "8v76MmHKPn", "reviewer": "Reviewer_QTFh", "summary": "The paper presents evidence suggesting that contour integration—a fundamental feature of human vision—remains largely absent in artificial vision models. To demonstrate this, the authors tested human performance on contour integration tasks and evaluated over 1,000 computational models to identify trends in machine vision systems. Notably, they found that models trained on larger datasets exhibited better contour integration capabilities. Intriguingly, certain advanced models like GPT-4 achieved performance levels comparable to humans, underscoring the role of scale in bridging this perceptual gap.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your favorable review. We are happy to read that you found the paper insightful and our experiments sound, and that you see how it helps guide future research. We address your comments point by point below, and point to an external link for additional figures: https://drive.google.com/drive/folders/1M_nUONfTXLmUZCHHL0PfIlIaEhpu0vLo?usp=drive_link\n\n> Concerns about training data, which could be remedied by testing models on the task with e.g. a different background that is more unlikely to be in training datasets\n\nThank you for the nice suggestion. We followed your proposal and made a new version of our dataset, this time with a red background instead of the standard black background (image examples in the folder __*red_images/*__). We tested the subset of pre-trained models also in figures 6d and figure 8. We did a t-test to test for the difference in the group performance of the fragmented conditions and found that there was no difference in model accuracy under these two conditions (_t=0.323, p=0.749_). This shows that the specific model training distribution is not the reason for model performance here, and that the results are robust against a change of background. See figure __*normal_vs_red.png*__.\n\n> Concerns about the analysis regarding how important architecture is and how robust the analysis is\n\nWe have taken two additional steps to remedy this:\nWe now also run a mixed effects regression, which allows for the architecture type to be treated as a random variable. This analysis also finds that training dataset size is most important (_z=14.66, p<0.001_), while model compute is less important but significant (_z=7.177, p<0.001_). Importantly, the random effect for architecture type under this more controlled analysis is dominated by error rather than systematic variance (variance=0.002, standard error = 0.008), suggesting that the random factor of architecture family plays little systematic role in the mixed effects model.\nWe analyzed ViT vs CNN-based architectures in more detail for a fixed training dataset size (ImageNet-1k) -- due to space constraints, we detail this analysis in our response to reviewer __eQjn__.\nThese further analyses provide additional evidence to our claim that architecture is less important than training dataset size.\n\n> Different downstream tasks, such as segmentation\n\nWe thank the reviewer for this idea. We analyzed additional models from the taskonomy (Zamir et al. 2018) model set, which  provides a great degree of breadth in terms of tasks, with the downside that we only have one model per task. Nevertheless, we report these additional results here (Figure __*taskonomy_scores.png*__).\n\nIn summary:\n- The base object classification and scene classification models reach 31% & 32% accuracy respectively\n- Unsupervised segmentation reaches 29%%, surface normal estimation 33%, and depth estimation reach 34% accuracy\n- Interestingly, the edge-computing model reaches only 21% accuracy\n\nIn general, it seems that while the task itself appears to play a role, object recognition stands out as a good objective, with more specialized objectives often falling slightly short. Nevertheless, the differences are rather small.\n\n> Novelty of the work: “Specifically, the authors highlight how integration bias—a phenomenon extensively studied in models by Geirhos et al.—can arise from large-scale training [...]”\n\nWe would like to point out that Geirhos et al. (2018) did **not** study integration bias, but rather shape bias, which is a different metric measured using an entirely different dataset. Specifically, Geirhos et al. (2018) studied the human and model responses to images with a conflicting texture and shape cue. This gives a point estimate of the amount of shape bias a model has. \n\nOur work does not study shape bias, but rather contour integration. Shape bias is a human preference to detect object categories by their shape, rather than their texture, while contour integration is the ability to integrate elements despite their discontinuity in image space. As such, while contour integration allows the recognition of images that would otherwise be unrecognisable, shape bias is a description of choice preferences. Our experimental setup is also different. Instead of several different conditions that are not related to each other on a single continuous scale (like in Geirhos et al.), we varied the image fragmentation. This allowed us to understand where and why neural networks fail (at the fragmentation of the contour), and to pinpoint this specifically to contour integration, a well-established “algorithm” in humans. Taken together, our work is very different from previous work, with a novel experiment, metric, and findings.\n\nWe thank you again for your review. We believe these comments address your concerns, but please let us know if additional analyses would be helpful. If all concerns are addressed, we ask the reviewer to consider increasing their score.", "claims_and_evidence": "I am sympathetic to the argument that models trained on vast datasets may develop shape biases conducive to contour integration. However, given that many state-of-the-art models (e.g., GPT-4o) are trained on proprietary, non-public datasets, there remains ambiguity about whether their training data included images resembling the experimental stimuli. While the paper notes that the exact stimuli used are not publicly disclosed, the possibility of \"close encounters\" between test stimuli and training data—even unintentional ones—raises concerns about ecological validity. A possible way could be  to verify whether performance persists under novel conditions, such as with modified backgrounds or added noise, which would reduce the likelihood of prior exposure influencing model behavior.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "No proofs were provided in the paper.", "experimental_designs_or_analyses": "The experiments appear methodologically sound. I specifically reviewed the human data collection protocols and model evaluation framework, which encompass:\n\nZero-shot evaluation via the BrainScore pipeline.\n\nDecoder fitting using fragmented ImageNet subsets, with label remapping to align with the 12 stimulus classes.\n\nDataset size analysis to correlate scale with performance.\n\nArchitecture size comparisons: While insightful, my primary concern lies in the potential unfairness of comparing models like RNNs—which lack access to modern large-scale training datasets—against newer architectures (e.g., transformers) trained on vastly larger corpora. This discrepancy in data availability complicates direct performance comparisons, as differences may reflect dataset scale rather than architectural superiority, leading to t values that undermined possible architecture impact. \n\nExperiments about integration biases and how countour integration leads to robustness.", "relation_to_broader_scientific_literature": "The paper delivers valuable insights for the community by systematically evaluating how artificial vision models align with human visual processing. It  contextualizes its findings against prior work, contrasting architectural approaches (e.g., Linsley et al.’s biologically constrained networks) with data-driven explanations for emergent capabilities. Specifically, the authors highlight how integration bias—a phenomenon extensively studied in models by Geirhos et al.—can arise from large-scale training, thereby enabling certain architectures to solve contour integration tasks without explicit architectural mimicry of human vision. This dual focus on architecture and data provides a nuanced framework for understanding the mechanisms behind model performance, guiding future research toward more human-like machine vision systems.", "essential_references_not_discussed": "I think the paper cites the most important literature.", "other_strengths_and_weaknesses": "The paper is well writen and provides good justification for the different experimental choices. It is transparent about its limitations and provide some possible way to mitigate some of the problems that can be raised.   I think it is a good example of a task that can help understanding the gap between human and artificial vision. \n\nMaybe I missed it, but perhaps would be to comment on models that have been trained on other downstream tasks such as object detection, image segmentation, and how the hypothesis of data diets apply or not in them.", "other_comments_or_suggestions": "No.", "questions_for_authors": "*  Although its mention in the limitation section, I think the potential problem of data leackage in the larger datasets can be an important point for the validity of the study. It would be good to run at least one control, perhaps under different background colors or under some noise.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6b18r571Mg", "reviewer": "Reviewer_ToRh", "summary": "This paper conducts a nuanced analysis of the extent to which vision models are human-like by conducting an experiment involving categorization of degraded images, where those images are reduced to lines or to fragments that are either points or line segments. Humans are able to recognize the images with fragmented contours while these pose a challenge for many vision models. Very large models nonetheless approach human performance, although they do not show a bias for directional fragments that is shown in humans.\n\n## update after rebuttal\n\nThank you for clarifying. These points do not change my opinion of the paper and I will keep my score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review. We are glad you find our experiments justified and carefully conducted, and that you found the findings interesting. We respond to each of your comments below point-by-point.\n\n> It is not clear whether there are actionable insights for improving models based on our results\n\nWe believe there are quite a number of actionable insights for improving models based on our results: \n\n**First**, many attempts in the literature for reproducing alignment to low-level human visual cortex have focused on architectural changes in lieu of other approaches (such as data-based approaches), see e.g. Kubilius, Schrimpf et al. (2019). We show that these approaches do not pay off when trained and evaluated at scale. This gives a direct actionable insight: for modelers seeking to improve models of basic human vision, a more fruitful approach is to focus on making the training diet of the models more human-like (especially in scale) than to improve the model architecture in a specific way.\n\n**Second**, we demonstrate success in training models for contour integration directly. While this in itself is the less surprising insight, it certainly is valuable for those who simply want models that exhibit human-like contour integration for further experiments. Interestingly, we also find that training for contour integration in models also leads to shape bias.\n\n**Third**, we find that models with a more human-like integration bias exhibit improved accuracy on a downstream object classification task (Figure 6b, Figure 8) – indicating that selecting models for their ability to integrate contextual information is a useful validation signal when building artificial neural networks. \n\nTaken together, our work substantially advances our understanding of contour integration in vision science, and provides a clear path models that do exhibit human-like contour integration: large training diets, or direct training. This finding contrasts previous work that focus on the architectural role of horizontal connectivity in human visual cortex and models as the mechanistic source of contour integration (e.g. Linsley et al, 2018).\n\n> “Very large models nonetheless approach human performance, although they do not show a bias for directional fragments that is shown in humans” \n[...] \n“ [the paper demonstrates] that while large models actually approach human performance even in this new task there is still an interesting bias in human vision that differentiates it from the models”\n\nWe would like to clarify that the best models do indeed show a human-like bias (Figure 6b), demonstrating that large training diets can yield human-like contour integration behavior in models. Integration bias separates almost all models from humans due to their integration bias that is not human-like. This shows that object recognition itself is possible without contour integration (a surprising finding in itself: e.g. Field 1993 as pointed out by reviewer eQjn; Kovacs & Julesz 1993; Grossberg & Mingolla 1985), but the largest models learn to do contour integration nonetheless. \n\n\nThanks again for your review. We believe we have addressed your comments and ask you to consider raising your score?\n\n\n\n**References from all rebuttals**\n\nCavanaugh, J. R., Bair, W., & Movshon, J. A. (2002). Nature and interaction of signals from the receptive field center and surround in macaque V1 neurons. Journal of Neurophysiology, 88(5), 2530–2546. https://doi.org/10.1152/jn.00692.2001\n\nZamir, A. R., Sax, A., Shen, W. B., Guibas, L. J., Malik, J., & Savarese, S. (2018). Taskonomy: Disentangling task transfer learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). \n\nGatis, D. (n.d.). rembg [Computer software]. GitHub. Retrieved March 31, 2025, from https://github.com/danielgatis/rembg​\n\nLinsley, D., Kim, J., Veerabadran, V., Windolf, C., and Serre, T. Learning long-range spatial dependencies with horizontal gated recurrent units. NeurIPS, volume 31. Curran Associates, Inc., 2018.\n\nGrossberg, S., Mingolla, E. Neural dynamics of perceptual grouping: Textures, boundaries, and emergent segmentations. Perception & Psychophysics 38, 141–171 (1985). https://doi.org/10.3758/BF03198851\n\nKubilius, J., Schrimpf, M., Kar, K., Rajalingham, R., Hong, H., Majaj, N., ... & Dicarlo, J. (2019). Brain-like object recognition with high-performing shallow recurrent ANNs. NeurIPS (pp. 12785-12796).\n\nField DJ, Hayes A, Hess RF. Contour integration by the human visual system: evidence for a local \"association field\". Vision Res. 1993 Jan;33(2):173-93. doi: 10.1016/0042-6989(93)90156-q. PMID: 8447091.\n\nMarques, T., Schrimpf, M., & DiCarlo, J. J. (2021). Multi-scale hierarchical neural network models that bridge from single neurons in the primate primary visual cortex to object recognition behavior. bioRxiv. https://doi.org/10.1101/2021.03.01.433495", "claims_and_evidence": "The core claims are justified through careful experiments and statistical analyses. Inferential test statistics and error bars are included.", "methods_and_evaluation_criteria": "In general the methods made sense for this problem and the experiment design was creative and sensible.", "theoretical_claims": "There were no theoretical claims to evaluate as the primary claims are empirical.", "experimental_designs_or_analyses": "The basic experimental designs are relatively simple, involving construction of a set of stimuli and evaluating model performance across those stimuli. The set of models used is extensive.", "relation_to_broader_scientific_literature": "There is a fairly extensive literature covering comparisons of models to human performance in image classification tasks. The paper does a good job of summarizing that literature. The key contributions here are use of a novel paradigm to provide a more stringent test of these models and the demonstration that while large models actually approach human performance even in this new task there is still an interesting bias in human vision that differentiates it from the models.", "essential_references_not_discussed": "I did not see any major omissions.", "other_strengths_and_weaknesses": "The primary strengths of this paper are its novel experimental approach and interesting findings about human vision. The main weakness is that it is not clear whether there are actionable insights for improving vision models -- the main focus in the paper in addressing this point is suggesting that there may be omissions from the training data, but this hypothesis is not explored in detail.", "other_comments_or_suggestions": "In general the paper was clear and I appreciated the use of color in the text to differentiate models.", "questions_for_authors": "No questions other than the issues identified above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9l6vtOABI1", "reviewer": "Reviewer_RkAR", "summary": "The work investigates the difference between the human ability to generalise object recognition and DNNs. The study builds on experiments that test the ability to recognise objects even in the presence of fragmentation, particularly by contour integration, and the ability of DNNs to perform the same task. The experiments conducted tests with 50 individuals and 1038 models from 13 architecture families and  18 datasets; the largest models were trained on more than 5B images. The tests showed that, in general, people perform better than DNNs and that the performance trend is related to the amount of data, with a correlation of 0.814. On the other hand, architectures are less important than data in contour integration. The set of experiments shows that contour integration is learned automatically from the distribution of data and does not depend directly on horizontal connectivity in the primary visual cortex, as previously assumed, as this mechanism can be learned from the data.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your review. We are happy to hear that you found our experiments meaningful, and the paper well-written and very interesting. We reply point-by-point to your comments below and point to an external link for additional figures: https://drive.google.com/drive/folders/1M_nUONfTXLmUZCHHL0PfIlIaEhpu0vLo?usp=drive_link\n\n> Contour extraction is not explained in the paper\n\nWe apologize for not including a detailed description of this step in our paper. We use a phosphene rendering algorithm (Rotermund et al., 2023, as cited in the paper) for extracting contours and rendering stimuli with phosphenes or segments. The contour extraction is simple: images are transformed from RGB to grayscale, and then convolved with a Gabor filter bank of 8 different orientations. Specifically, our contour image is defined by:\n\n__see *contour_equation.png*__\n\nFor the placement of phosphenes and segments, we place elements on the contours of the object preferentially depending on the strength of the contour and its directionality.\nWe use this algorithm both for our experimental stimuli, as well as ImageNet-1k images. For ImageNet-1k images, we also perform a background removal using `rembg` (Gatis) before applying this algorithm. We will include this description in the updated manuscript.\n\n> The problem of obtaining contours, which here is considered resolved and easy is, in fact, still an open problem even in the case of supervised models\n\nHow exactly we obtain contours from RGB images is not central to our argument in any way. This is for two reasons: \n\n1) all models and humans see the same images, regardless of how the contour was extracted.\n\n2) The contour condition is merely a control condition meant to show that the large drop in model performance is not merely due to superficial changes in data distribution (i.e., most of the image being black). \n\nThe contour extraction merely serves as an image preprocessing step that we can use to render images which are shown to humans and models to compare their performance and behavioral characteristics.\n\n> The paper is well-written and very interesting. However, I found that attempting to train models directly to group elements is a bit naive.\n\nThe goal of this experiment was to causally show the primacy of training data in achieving a human-like contour bias, which we successfully demonstrate with the improved performance of the resulting models. Despite the lack of horizontal connectivity or other architectural biases, we were able to train a model simply using segments and phosphenes to exhibit a human-like integration bias. This also resulted in high shape bias, exceeding previous shape biases from other similar direct training approaches (Geirhos et al., 2018). While the approach itself is simple, it serves to strengthen our claim about the importance of dataset. \n\n> There are attempts like this in the literature, and obtaining binary contours using Gaussian filters and Otsu threshold is not quite skillful. \n\nWe are not exactly sure what is meant by it not being “quite skillful”. The motivation here was to simply turn our non-binary contours extracted using Gaussian filters into binary contours, and for this purpose the methodology works very well.\n\n> Perhaps the only theoretical claim concerns the demonstration that contour integration is not necessarily a product of horizontal connectivity in the primary visual cortex (referring to the literature) and that the mechanism is learned from the amount of data inducing learning. I am not sure that the tests effectively support these conclusions.\n\nThis is a very interesting thought – which experiments do you have in mind that would better support these conclusions? We believe our evidence is strong. We show that:\n\n- human-like contour integration emerges in models trained on large datasets despite any explicit architectural mechanism being present (Figs. 6b, 7). This provides proof of existence that such a mechanism is not strictly necessary.\n- We causally show that human-like contour integration is possible to directly train for without the use of human behavioral response data (Figure 7). This is causal evidence in a controlled setting that such a mechanism is not necessary.\n\nTaken together, we believe these facts strongly support the conclusion that horizontal connectivity is not necessary for contour integration. Of course it is still possible that the human visual system implements contour integration using a mechanism of horizontal connectivity. Our results challenge the prevailing view that this is the only way to implement contour integration, or that the existence of this mechanism is the key reason for why contour integration exists - clearly, it emerges in other settings too.\n\nWe thank you again for your review, and believe that we have addressed your comments. In light of this, we hope that you consider raising your score.", "claims_and_evidence": "The work follows an interesting track leading to conclusions on contour integration through meaningful experiments on humans in a controlled laboratory setting and on models.\n\nEach experiment step is documented in detail, and metrics and statistics explain the analysis.", "methods_and_evaluation_criteria": "Benchmarks are made on a large amount of data collected, both on the side of humans and models, significantly validating the claims.", "theoretical_claims": "Perhaps the only theoretical claim concerns the demonstration that contour integration is not necessarily a product of horizontal connectivity in the primary visual cortex (referring to the literature) and that the mechanism is learned from the amount of data inducing learning. I am not sure that the tests effectively support these conclusions.", "experimental_designs_or_analyses": "In my opinion, the experimental design is valid and also quite interesting.", "relation_to_broader_scientific_literature": "This is probably the first systematic model on the subject. It will certainly induce discussion and further research; there is little or no prior literature.", "essential_references_not_discussed": "All required references for the main task seem to be discussed.", "other_strengths_and_weaknesses": "The paper is well-written and very interesting. \nHowever, I found that attempting to train models directly to group elements is a bit naive. \n\nThere are attempts like this in the literature, and obtaining binary contours using Gaussian filters and Otsu threshold is not quite skillful. Furthermore, besides *IN only*, it is not clear how contours were extracted in the other combination, as there is no reference to the literature. The problem of obtaining contours, which here is considered resolved and easy is, in fact, still an open problem even in the case of supervised models..", "other_comments_or_suggestions": "I'm unsure about the attempt to train models directly to group elements. Several attempts like this have been made in the literature. Moreover, obtaining binary contour using Gaussian filters and Otsu threshold is quite naive.\n\nAlso, apart from  \"IN only\"  for all the other cases, it is unclear how contours were extracted, as no literature is mentioned. \nThis ease in extracting contour is a bit superficial since no model performs well on this task yet. \nThis blurs the conclusions in lines 368-378.", "questions_for_authors": "Please explain clearly how you have extracted the contours on all objects in ImageNet-1K", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "P0pNDYZekv", "reviewer": "Reviewer_eQjn", "summary": "The authors systematically dissected where and why models struggle with contour integration by designing an experiment that tested object recognition under various levels of object fragmentation. It was found that humans exhibited an integration bias – a preference towards recognizing objects made up of directional fragments over directionless fragments. It was also found that not only models that shared this property performed better, but that this bias also increased with model training dataset size, and training models to exhibit contour integration leads to high shape bias.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their review. We have addressed your comments individually below and point to an external link for additional figures: https://drive.google.com/drive/folders/1M_nUONfTXLmUZCHHL0PfIlIaEhpu0vLo?usp=drive_link\n\n> Long-range interactions in humans being normally associated to contour integration, and how that relates to our task.\n\nIn our paper, we have intentionally omitted discussion around long-range interactions as our task was not designed to investigate these effects directly:\n\n- Our stimuli spanned only an 8x8 degree window of visual angle.\n\n- Our stimulus presentation time was only 200ms, which was followed by a 1/f noise mask. This ensured that no long-lasting integration or tracing is possible. \n\nThe goal of these steps was to ensure that the types of long-range connectivity you refer to are controlled in this setting - see also below for our analysis of the human data.\n\n> The difference in human perception between two sets of stimuli should be analyzed. In my opinion, perception of directionless outlines should have a strong relationship with proximity. However, this is not the case for the directional version. \n\nThank you for this suggestion, which we have now done. In summary, we found little difference in the scaling of the two conditions (phosphenes vs segments) in humans. The difference in performance in these two cases is mostly a shift of intercept instead of a change in slope across the number of elements (figure __*human_accuracy_split.png*__), though note that while the effect of difference in slopes is small, it is still technically significant (_t=2.87, p<0.01_). This means that the directionality of the segments only slightly affects performance positively over the directionless prosphenes as element density increases.\n\n> Comparison between CNN and transformer architectures, since CNNs do not have long-range interactions while transformers do\n\nThank you for this suggestion. We indeed included transformers in our original work (the single largest architecture family in our paper is the vision transformer (ViT), of which we had 257 decoder-fit variants; Table 1, row 1).\n\nWe have now conducted several new analyses comparing transformers and CNNs. To make results comparable, we restricted the analysis to only models trained on ImageNet-1k, although we also provide figures for all models in the figures too.\n\n- We found that the overall performance is no different (CNNs: _32.49%_ on average, ViTs: _31.24%_ on average, t-test of means _p=0.1108_: not significant). This is also true even when not controlling for dataset (CNNs: _29.08%_ on average, ViTs: _29.98%_ on average, t-test of means _p=0.1889_: not significant); figure __*vit_vs_cnn_imagenet.png*__\n\n- Integration biases between transformers and CNNs are comparable: ViT (_7.8%_) and CNN (_10.1%_) are not statistically significantly different (_t=2.45, p=0.016_) at the 0.01 criterion; figure __*vit_vs_cnn_imagenet.png*__\n\n- The way model performance scales across the number of elements is the same across models and across conditions: only one condition across all conditions is different between CNNs and ViTs (the 16-phosphenes condition), while the differences between the rest are statistically non-significant. __*vit_vs_cnn_imagenet_scaling.png*__\n\nTaken together, these analyses show that long-range interactions do not play a crucial role in our task either in humans nor in models. \n\n> Receptive field sizes and how they relate to task performance should be studied\n\nWe thank the reviewer for this nice suggestion. In addition to the experiments where we compared ViTs to CNNs (effectively two extremes of this spectrum), we have now added an explicit test for receptive field size and its relationship to the primate visual system. \nWe evaluated the same subset of models as in Fig 6d and 8 on two Brain-Score benchmarks that test for the similarity of the effective receptive field size of a model to a primate counterpart. These are the Grating summation field (GSF) and surround diameter (Marques 2020, Cavanaugh 2002) benchmarks measured in macaque V1. In short, we do not find a statistically significant relationship between either measure of receptive field size similarity and fragmented object recognition accuracy (GSF _r=0.2655, p=0.0653_; surround diameter _r=0.276, p=0.055_); figure __*receptive_field.png*__\n\nWe believe these additional results further bolster our conclusion that architecture plays a minimal role in contour integration, and that the receptive field size of models is not a crucial component in this study.\n\nWe also thank you for pointing us to missing references, which we have included. \n\nThank you again for your review. We believe we have addressed all of your concerns and would be grateful if you considered raising your score.", "claims_and_evidence": "The authors did not explain why the deep learning models performed worse than humans. In vision science, contour integration is normally associated with long-range interactions which has been exploited by humans. However, the authors did not mention this in the paper. The authors may need to investigate the size of the actual receptive field utilized by those CNN models. It is known that CNNs cannot capture long-range interactions/dependencies well while Transformer was developed to address this issue. In this context, it would be interesting to compare CNNs and Transformer networks with regard to the spatial extent that they utilize.", "methods_and_evaluation_criteria": "I would like to see the size of the actual receptive field utilized by those CNN models in the experimental results. It is really interesting to explore the relationship between the performance of those models and the size of receptive field.", "theoretical_claims": "The role of long-range interactions used by HVS should be discussed because they are important to contour integration. The difference in human perception between two sets of stimuli should be analyzed. Note that perception of the outline which consists of directionless points should have a strong relationship with proximity. However, the directional version is different.", "experimental_designs_or_analyses": "Again, the authors are encouraged to examine the maximal spatial extent that the deep models exploited and the relationship between this value and the performance of the model on contour integration.", "relation_to_broader_scientific_literature": "The human subject study was similar to that conducted in (Panis et al. 2008). However, the authors did not mention this work at all. Also, they missed many important references, such as (Field et al. 1993), (Dong et al. 2021).", "essential_references_not_discussed": "Field et al. 1993, Contour integration by the human visual system-- evidence for a local ‘association field’\nPanis et al. 2008, Identification of everyday objects on the basis of fragmented outline versions\nDong et al. 2021, Perceptual Texture Similarity Estimation An Evaluation of Computational Features", "other_strengths_and_weaknesses": "Strength:\n-A large number of Models were examined.\n-The experimental results were analyzed using statistical methods.\n-Two sets of stimuli were used.\n\nWeakness:\n-The difference in human perception between two sets of stimuli should be analyzed. In my opinion, perception of directionless outlines should have a strong relationship with proximity. However, this is not the case for the directional version.\n-Contour integration is normally associated with long-range interactions which can be exploited by humans. But the authors did not mention this in the paper. The authors are encouraged to investigate the size of the actual receptive field utilized by CNN models. It is known that CNNs cannot capture long-range interactions/dependencies well while Transformer was developed to address this issue. Therefore, it would be interesting to compare CNNs and Transformer networks in terms of the spatial extent that they can exploit.", "other_comments_or_suggestions": "-The difference in human perception between two sets of stimuli should be analyzed.\n-The authors are encouraged to investigate the size of the actual receptive field utilized by CNN models.\n-CNN and Transformer models should be compared in terms of the spatial extent that they can exploit.", "questions_for_authors": "None.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This work introduced some ideas from cognitive psychology to improve visual recognition models. Focusing on the idea of contour integration, the proposed additions to the models lead to a subtle improvement in recognition accuracy. There is overall minimal but positive enthusiasm for the contributions of this work.
Decision: Accept (poster)