Brain-like representational straightening of natural movies in robust feedforward neural networks
Authors: Tahereh Toosi
Elias Issa
Venue: ICLR 2023 poster (2023)
Keywords: Perceptual straightening
invertibility
robustness
visual cortex

Abstract: Representational straightening refers to a decrease in curvature of visual feature representations of a sequence of frames taken from natural movies. Prior work established straightening in neural representations of the primate primary visual cortex (V1) and perceptual straightening in human behavior as a hallmark of biological vision in contrast to artificial feedforward neural networks which did not demonstrate this phenomenon as they were not explicitly optimized to produce temporally predictable movie representations. Here, we show robustness to noise in the input image can produce representational straightening in feedforward neural networks. Both adversarial training (AT) and base classifiers for Random Smoothing (RS) induced remarkably straightened feature codes. Demonstrating their utility within the domain of natural movies, these codes could be inverted to generate intervening movie frames by linear interpolation in the feature space even though they were not trained on these trajectories. Demonstrating their biological utility, we found that AT and RS training improved predictions of neural data in primate V1 over baseline models providing a parsimonious, bio-plausible mechanism -- noise in the sensory input stages -- for generating representations in early visual cortex. Finally, we compared the geometric properties of frame representations in these networks to better understand how they produced representations that mimicked the straightening phenomenon from biology. Overall, this work elucidating emergent properties of robust neural networks demonstrates that it is not necessary to utilize predictive objectives or train directly on natural movie statistics to achieve models supporting straightened movie representations similar to human perception that also predict V1 neural responses.


BRAIN-LIKE REPRESENTATIONAL STRAIGHTENING OF NATURAL MOVIES IN ROBUST FEEDFORWARD NEURAL NETWORKS
Tahereh Toosi & Elias B. Issa
Department of Neuroscience, Zuckerman Mind Brain Behavior Institute Columbia University New York, NY, USA
tooereh.toooii el das. issa [ssa)]coll umbia edu
ABSTRACT
Representational straightening refers to decrease in curvature of visual feature representations of a sequence of frames taken from natural movies. Prior work established straightening in neural representations of the primate primary visual cortex (V1) and perceptual straightening in human behavior as hallmark of bi- ological vision in contrast artificial feedforward neural networks which did not demonstrate this phenomenon as they were not explicitly optimized to pro- duce temporally predictable movie representations. Here, we show robustness noise in the input image can produce representational straightening in feedforward neural networks. Both adversarial training (AT) and base classifiers for Random Smoothing (RS) induced remarkably straightened feature codes. Demonstrating their utility within the domain of natural movies, these codes could be inverted to generate intervening movie frames by linear interpolation in the feature space even though they were not trained on these trajectories. Demonstrating their bio- logical utility, we found that AT and RS training improved predictions of neural data primate V1 over baseline models providing a parsimonious, bio-plausible mechanism noise in the sensory input stages for generating representations in early visual cortex. Finally, we compared the geometric properties of frame representations in these networks better understand how they produced repre- sentations that mimicked the straightening phenomenon from biology. Overall, this work elucidating emergent properties of robust neural networks demonstrates that it is not necessary to utilize predictive objectives or train directly on natural movie statistics to achieve models supporting straightened movie representations similar to human perception that also predict V1 neural responses.
INTRODUCTION
In understanding the principles underlying biological vision, a longstanding debate in computational neuroscience is whether the brain is wired to predict the incoming sensory stimulus, most notably formalized in predictive coding (Rao & Ballard, 1999; Friston, 2009; Millidge et al., 2021), or whether neural circuitry is wired to recognize or discriminate among patterns formed the sensory epithelium, popularly exemplified by discriminatively trained feedforward neural networks (DiCarlo al., 2012: Tacchetti al.. 2018; Kubilius al., 2018). Arguing for role of prediction in vision, recent work found perceptual straightening of natural movie sequences in human visual perception (Hénaff et al., 2019). Such straightening diagnostic of system whose representation could be linearly read out to perform prediction over time, and the idea of representational straightening res- onates with machine learning efforts to create new types of models that achieve equivariant, linear codes for natural movie sequences. Discriminatively trained networks, however, lack any prediction over time in their supervision. It may not be surprising then that large-scale ANNs trained for clas- sification produce representations that have almost improvement in straightening relative to the input pixel space, while human observers clearly demonstrated perceptual straightening of natural movie sequences (subsequently also found in neurons of primary visual cortex, V1 (Hénaff al.,

---

Natural movies
Natural images
Representation
straightening
invertibility
perception
cat
dog
V1 cortex X Feedforward ANN
pixel space
Training objective
1+1
+oise
t1
feature space
boundary
Learning to Linearize, Temporal Prediction
Robustness to Input Perturbation
Figure I: Perceptual straightening of movie frames can be viewed as invertibility of latent represen- tations for static images. Left: straightening of representations refers decrease in the curvature of the trajectory in representation space such neural population in the brain or human perceptual space, but standard ANNs do not show straightening (Hénaff et al., 2019; 2021). Right: Invertibility of latent representation refers to interpolation between representation of two images (e.g. an image of a dog and an image a cat), where the invertible interpolations show the main features of a dog morph into the main features of cat. Invertible representations emerge in robust ANNs (Engstrom et al., 2019b), obviating the need to directly train for temporal straightening.
2019; 2021)). This deficiency in standard feedforward ANNs might suggest a need for new models trained on predictive loss functions rather than pure classification to emulate biological vision.
Here, we provide evidence for an alternative viewpoint, that biologically plausible straightening can be achieved in ANNs trained for robust discrimination, without resorting to a prediction objective or natural movies in training. Drawing on insights from emergent properties of adversarially-traiee neural networks in producing linearly invertible latent representations, we highlight the link between perceptual straightening images (Figure 1). We examine straightening in these robust feedforward ANNs finding that their properties relate to those the biological vision framework. The contributions this work are as follows:
natural movies to invertible latent representations learned from static
We show that robust neural networks give rise to straightened feature representations for natural movies in their feature space, comparable to the straightening measured in the pri- mate brain and human behavior, and completely absent from standard feedforward net- works.
2. We show that linearly interpolating between the start and end frames a movie in the output feature space of robust ANNs produces synthetic frames similar to those of the original natural movie sequence in image space. Such invertible linear interpolation is precisely the definition of temporally predictive feature representation.
3. Compared prior models of early visual cortex, robustness input noise (corruption or adversarial robustness) is significantly better at explaining neural variance measured from V1 neurons than non-robustly trained baseline models, suggesting new hitherto unconsid- ered mechanism for learning the representations in early cortical areas that achieves natural movie straightening.
2

---

2 RELATED WORK
2.1 MECHANISMS FOR PRODUCING BRAIN-LIKE REPRESENTATIONS
Feedforward ANNs as models of biological vision. Standard feedforward ANNs, although lacking a number of bio-plausible features such as feedback connections or local learning rule (Whittingon& & Bogacz, 2019), still can explain the neural variance (Schrimpf et al., 2018) recorded from rodent (Bakhtiari et al., 2021), monkey (Yamins et al., 2014; Bashivan et al., 2019), and human visual cortex (Khaligh-Razavi which are considered more bio-plausible by using a prediction objective function (e.g.. PredNet and CPC (Zhuang et al., 2021; Schrimpf et al., 2020)). Thus, to learn the representations in the brain, regardless of bio-plausibility of mechanisms, feedforward ANNs provide a parsimonious more tractable class of leading models for object recognition in the visual cortex.
Kriegeskorte, 2014; Cichy et al., 2016) better than alternatives
Models of primary visual cortex. In neuroscience, rather than rely soicly a on top-down training objective like standard ANNs do, there has been a tradition of explaining early visual representations using more fundamental principles such as sparse coding and predictive coding as well as invoking unsupervised training (Olshausen & Field, 1996; Rao & Ballard, 1999). For example, unsupervised slow feature analysis extracts the slow-varying features from fast-varying signals in movies based on the intuition that most external salient events (such as objects) are persistent in time, and this idea can be used to explain the emergence of complex cells in V1 (Berkes & Wiskott, 2005). Recent work in machine learning has attempted to blend more bottom-up principles with top-down training by experimenting with swapping out ANN early layers with VI-like models whose filters are inspired from neuroscience studies (Dapello et al., 2020). This blended model turns to have benefits for classification robustness in the outputs. However, it remains unclear whether there is a form of top-down training that can produce V1-like models. Such mechanism would provide fundamentally different alternative prior proposals of creating VI through sparse coding or future prediction (Hénaff et al., 2019; 2021).
2.2 TEMPORAL PREDICTION AND INVERTIBILITY IN NEURAL NETWORKS
Learning to predict over time. Changes in architecture, training diet (movies), and objective (predicting future frames) have all been explored mechanisms to produce more explicit equiv- ariant representations of natural movies (Lotter et al., 2016; van den Oord et al., 2018). Directly related the idea of straightening, penalizing the curvature of representations of frames was used in Learning to linearize (Goroshin et al., 2015) to learn straightened representations from unlabeled videos. This class of models does not need supervision which makes them more dio-plausible in nature; however, as mentioned in the previous section, they lag behind supervised feedforward ANNs both in terms of learning effective representations for object recognition and in producing feature representations that predict neural data.
Learning invertible latents. In deep learning applications, invertibility is mostly discussed in generative neural networks as a constraint to learn a prior to address applications in signals and systems such image de-noising, signal compression and image reconstruction from few and noisy measurements or to be able to reconstruct or modify real images. Usually invertibility is implemented by carefully designing dedicated architectures (Jacobsen al., 2018b; Chen et al., 2019). However, recently it has been shown it can be implemented in standard feedforward ANNs when they undergo training for adversarial robustness (Engstrom al.. 2019b;c). These works showed empirically that adversarially robust training encourages invertibility as linear interpolation between classes (e.g., cat to dog) results semantically smooth image-to-image translation (Engstrom et al., 2019b) as opposed to blurry image sequences produced by standard ANNs.
We reasoned that robust networks which encourage invertibility may also lead to straighten- ing as this a property that would be related to improved invertibility of a network, so we sought to extend prior work and study the behavior of robustly trained networks specifically in the domain of natural movies. We report on how these networks straighten natural movies in their features spaces and can invertibly reproduce movie frames in natural sequence.
3

---

Non-robust Neural Networks
Robust Neural Networks
Curvature change (degrees re: pixels)
(210"
RandSmooth 4,:0 0
RandSmooth 6y:07
AdvTroin 07
Untrained
SupMoccouggm SupMokugm R50
Standard R50
Se (-Sup I50
Standard R152
-30
byye3
Figure 2: ANNs show straightening of representations when robustness noise constraints (noise augmentation or adversarial attack) is added to their training. Measurements for straightening of movie sequences (from (Hénaff et al., 2019), in each layer of ResNet50 architecture under different training regimes; supervised training (standard), no training (random parameters). self self-supprvised training (Zhontar al., 2021), supervised training with no augmentations, supervised training with extensive augmentations, supervised training with noise augmentation (base classifiers for RS) (Co- hen et al., 2019), and supervised training with adversarial training (Engstrom al., 2019a)
3 METHODS
3.1 BASELINE MODELS
We consider the class of feedforward convolutional neural networks, typically restricting to the ResNet-50 (He et al., 2015) architecture trained on ImageNet for the main analyses. Baseline net- works (not trained for robustness) include a supervised ResNet-50/ResNet- 101e--/////ss and self-supervised (Barlowtwins (Zhontar et al., 2021)). We trained ResNet-50 for imagenet classifica- tion without augmentations and with extensive augmentations (Chen et al., 2020), labeled as Sup- NoAugm and SupMocoAugm, respectively. We also consider Voneresnet (biological V1 front-end (Dapello et al., 2020)) and ResNet-50 trained as a base network for action recognition (Chen et al., 2021) but include these as separate examples the Appendix since they use modified architecture.
Table I: Clean accuracy and robust (attack: L2, = 0.1) accuracy for the models used. Except for the custom models, all the other models were obtained from the repository of the references. Note that RS here refers the base classifier random smoothing without probabilistic inference.
Models
Clean accuracy
Robust accuracy
Model reference
RN50 AT Le :
58,50
57.81
(Engstrom et al., 2019a)
RN50 AT Loo
62.80
61.40
(Engstrom et al., 2019a)
RN50 AT Loo
48.29
47.01
(Engstrom et al., 2019a)
RN50 RS Lg c 0.25
39.40
36.01
(Cohen et al., 2019)
RN50 RS L2 : 0.5
23.75
22.21
(Cohen al., 2019)
RN50 RS Lz2
1
10.62
10.17
(Cohen et al., 2019)
RN50 Standard
75.43
52.32
(He al., 2015)
RN50 No augmentation
64.35
28.13
custom
RN50 Extensive augmentation
75.27
53.08
custom
RN50 Self-supervised
70.18
41.73
(Zbontar al., 2021)

---

3.2 MODELS TRAINED FOR ROBUSTNESS
We consider two forms of models trained for minimizing classification loss Coe in the face of input perturbations 8 hxx subject constraints on the overall magnitude of perturbations in the input space, where v, y, 0 are the network input, output, and classifier parameters, respectively:
Lee (0, x + 5,
(I)
In adversarially trained networks, projected gradient descent from the output space finds maximal directions of perturbation in the input space limited to length 6, and training entails minimizing the effect of these perturbation directions the network's 's output (Madry et al., 2018). In random smoothing (Lecuyer et al., 2018; Cohen et al., 2019), a supervised network trained but in the face of Gaussian noise added to the input space as the base classifier before performing probabilistic inference. In this work, we only use the representations as learned in base classifiers without the probabilistic inference. The perturbations in the base classifiers ō thus follow:
Grand NN00, o2),
δade == arg max Lee (0, + 6,
(2)
These defenses imput noise have different motivations. Adversarial robustness provides defense against white box attacks whereas random smoothing is protecting against general image corrup- tions. However, prior work has suggested a connection between corruption robustness and adver- sarial robustness (Ford et al., 2019). Theoretically, random smoothing leads to certified robustness (Cohen al., 2019) and trains a condition of invertible networks (Jacobsen et al., 2018a), while adversarial robustness has been shown empirically to lead to invertible latent representations in net- works (Engstrom al., 2019b).
3.3 REPRESENTATIONAL METRICS
Representational straightening estimates the local curvature in a given representation of a se- quence of images (natural or artificial) of length N, Cseg {Ti, Itaa.-- "aaa.--.y Ftx } as the angle between vectors connecting nearby frames, and these local estimates are averaged over the entire movie quence for the overall straightening in that representational trajectory (same as (Hénaff et al., 2019)):
N- l Σ
Pe. rt-1 |r: T=-1ll
Ft. Tt+1 -7
= arccos
| - -rell Oseg
(3)
N
Lower curvature (angle between neighboring vectors) indicates a straighter trajectory. and in the results we generally reference curvature values the curvature in the input space (i.e., straightening relative to pixel space). This metric has been utilized in neuroscience showing that humans tend to represent nearby movie frames in a straightened manner relative to pixels (Hénaff et al., 2019). This curvature metric is also closely related to objectives used in efforts to train models with equivariance by linearizing natural transformations in the world as an alternative to standard networks trained for invariant object classification (Goroshin et al., 2015; Sabour et al., 2017).
Expansion. We define the radius a sequence of images from a movic clip as the radial size of the minimum covering hyper-sphere circumscribing all points representing the frames in (Gärtner, 1999). We use this measure to supplement geometrical characterization of a movie sequence in pixel space and in model's representational spaces. Like representational straightening values, expansion values for models in the main text are referenced to the radius measured in pixel space or the radius measure for the same layer in a baseline network by simply dividing by those references. We used mini-ball, a publicly available python package based on (Gärtner, 1999) to measure radius of the covering hyper-sphere.
RESULTS
4.1 ROBUST ANNS EXHIBIT REPRESENTATIONAL STRAIGHTENING
With insights from connections to invertibility (see Figure/ 1), we hypothesized representational straightening of movie trajectories could be present in robustly trained neural networks. We took
5

---

Natural Movie
Invertibility of Representations 0.6
Pixels
SSIM Real Movie Frames
Standard
0.4-
RandSmooth L, :0 0.25
RandSmooth Ly o 0.50
RS
0.2
SupNoAugm R50
SupMocoAugm R50
AdvTrain La o? 3
RandSmooth lg302
Adv o
Standard R50
Self-Sup R50
AdvTrain L...
AT
t-6
t-4
t-2
t Original Frame
Interpolated Frames
Figure 3: Invertibility as measured by the SSIM (Wang et al., 2004) the actual in-between frames (labeled as Natural Movie) and the pixel-space projected linear interpolations between the first and the last frame labeled Pixels, standard ResNet50, RS (ResNet50, L2 a² = 0.5) and AT (ResNet50, L2 o² 3). Interpolating the representations of the first and last frames in an invertible repre- sentation space generates sequence of frames that are more similar to the ground-truth in-between frames, but for ion-invettible representation the generated frames are blurry and more similar to the interpolation in pixel space (a.k.a. artificial sequence). Interpolating the first and last frames in pixel space, second row, gives exactly what called artificial sequence in studies of straight- ening (Hénaff et al., 2019; 2021), as opposed to natural sequence which were the actual in-between frames.
the same movie stimuli publicly available (Hénaff al., 2019))A.4.1, Figure 12) and the same met- rics, and we tested the same architecture, ResNet50 (He et al., 2015)) trained under different loss functions Table to perform controlled head-to-head comparisons. Figure shows representational straightening of natural movies measured in layers of ResNet50 trained under AT (Engstrom et al., 2019a) and RS (Cohen et al., 2019) at different adversarial attack or noise levels, respectively. Ro- bust neural networks in contrast other ANNs decreased the curvature of natural movies. Straight- ening for artificial sequences as measured (Hénaff et al., 2019) (A.1, Figure 7) and other models (A.2, Figures 9 and 8) are provided in Appendix. Importantly, although most models, whether standard ResNet-50 or one with a V1-like front-end, may display an initial dip in curvature for nat- ural movies the very earliest layers, this except for robustly trained networks (A.2, Figure vs. A.1, Figure 7) and those trained on action recognition from temporally instructed training, which we include here as a proxy for movie- like training though its feedforward architecture deviates from ResNet50 by additional temporal processing components (A.2, Figure 8).
sustained in feature representations of later layers
Perceptual Straightening measured as invertibility of latent representations. Next, we sought to empirically test how well robust networks can invert natural movies given that they contain lin- earized feature representation of movie frames in their high level feature spaces and given the general conceptual benefit linearity for invertibility Figure 1. We measured invertibility of each model on the same movie sequences used for measuring straightening as follows. We linearly interpolated between latent representation the first and last frame each movie and used the same procedure that used previously in (Engstrom et al., 2019b;a) to obtain the pixel-space correspondence of those interpolated representations. Whereas those generated pseudo-frame frames can be assessed by for their pixel-by-pixel distance to the actual movie frame, chose metric, Structural Similarity In-

---

Primary visual cortex
Neural predicitivity (V1 explained variance)
r=-0.53
0.35
0.35
0.30
0.30
0.25
RandSmaath 0.25
RanuSmaath Ly pf 1.00
0.20
0.25
SupMocoAug R50
0.15
araightnigg
SupNoAugm A50
Ade -0"
Ly
0.20
Standard R50
0.10
Sel-SSu
AdvTrain
AdrTrain
0.05
0.15
*Best fitting layer in each model
-30
-20
-10
Curvature change (degrees re: pixels)
Figure 4: Left: RS and AT are more predictive of V1 neural responses than other non-robust models of the same architecture (ResNet50). Right: each dot represents a layer in ResNet50 trained under different loss function (color codes same left). Higher representational straightening (negative curvature change) associates with higher V1 predictivity. Intriguingly, the highest VI predictiv- ity corresponds to layers that exhibit comparable straightening that measured from VI neurons (-100 on average) (Hénaff et al., 2021). Explained variance is noise-corrected and computed as in (Schrimpf al., 2018)
dex Measure (SSIM (Wang et al., 2004)). that utilizes intermediate- statistics motivated from biological vision and putatively more related to some aspects human perception than simple pixel space correspondence. Figure 3 shows an example of such inverted frames for standard ResNet50, RS (I22 a² 0.5) and AT (T,2 a² = 3), and a summary of average measured invertibility using the SSIM metric pseudo-frames from each model. As expected, inline with the findings of previous work (Engstrom et al., 2019b), AT models scored relatively higher on invertibility of frames than baseline discriminative model. However, what had not been previously shown is that RS models, using merely the benefits of their robustness to noisy augmentation (base classifier on top of learned representation; no probabilistic inference), also exhibit higher invertibility scores compared to standard trained models. Invertibility scores were consistently improved in RS and AT models across variety of movies tested including those with relatively stationary textures and not just dynamic objects (see A.4.4, Figure 13 for further examples and A.4.3, Table for scores across all 1I movies). Thus, RS models along with AT models exhibit invertibility of representations for movie frames which further demonstrates their ability to support perceptual straightening of natural movies their highest layers that may be functionally similar to perceptual straightening previously measured from human subjects (Hénaff et al., 2019).
4.2 RANDOM SMOOTHING AND ADVERSARIAL TRAINING IN EXPLAINING NEURAL REPRESENTATIONS IN THE PRIMATE VISUAL SYSTEM
Robustness to noise as a bio-plausible mechanism underlying straightening in primary visual cortex. As shown above, straightening which is a constraint for brain-like representations visual cortex manifests in robust neural networks. Both classes of RS and AT training for robustness to Ta norm generate straightened representations of movie sequences. However, to distinguish among models of object recognition, we can measure how well they explain variance in patterns of neural activity elicited different visual cortical areas. Here, for all neural comparisons in our analyses, measured the Brain-Score (Schrimpf et al., 2018) using the publicly available online resource to assess the similarity to biological vision each model, which is a battery of tests comparing models against previously collected data from the primate visual system (see Brain-Score.ong We found that RS and AT models provided a better model of V1 (in terms explained variance) compared to non-robust models Figure 4, On other benchmarks,
go up the ventral stream hierarchy from V1

---

Pixel Space Radius vs. Representational Straightening
Minimum Covering Sprere
corr(radius curvature)
0.2
0.1
pixel space
0.0
Standard
M
-0.1
Ad/frain L.. L.:0"=4
Testure straightening
0.2
RandSimoan ty: 0.25
RandSmmors fare"
Figure 5: Can straightening for movie sequence be explained by the size of the hyper-sphere bounding the frames (i.e. radius in pixel space)? While RS exhibits small but positive correlation, the rest of the models, including AT, show negative or no correlations. Positive correlation means the smaller the size of the bounding hyper-sphere in pixel space, the more straightened the representation over the layers the model.
to IT again, keeping the layer assignment fixed across models for proper comparison, we observed a decrease in explainability of robust models (A.3, Figure 11), in part presumably because robust models have lower object classification performance which is known to drive fits in higher brain areas like V4 and IT supporting object recognition (Yamins et al., 2014). Previous work (Dapello et 2020; Kong et al., 2022) linked adversarial robustness in models to their higher Brain-Scort for V1, but we found that it may not be specifically driven by adversarial robustness per se, rather (I2) noise robustness is also sufficient (as in base classifiers of RS tested here). More broadly, looking neural fits across all models and their layers, we find that straightening in a particular model-layer correlates with improved explanatory power of variance in cortical area VI (Figure 4, middle panel, each dot is layer from a model), being even more strongly predictive than robustness of the overall model (A3, Figure 10). The level of straightening reached by best fitting layers of RS and AT models was comparable to the 10 degree straightening estimated in macaque VI neural populations (black dashed reference line in Figure 4). This complements the fact that robust models peak near the 30 degree straightening measured in perception (Figure 2), suggesting that robust models can achieve brain-like level of straightening to VI and perception.
Does the geometry of movie frame representations in pixel space dictate straightening in down- stream representations? The connection between two properties the same representation mani- fold, robustness to independently sampled noise and straightened trajectories of smooth input tem- poral sequences, not immediately clear. Because robustness is achieved by adding noise bounded by norm (I2, T2, T in pixel space, natural question is whether the radius of the bounding hyper-sphere of the frames of the tested movies in pixel space (see Expansion in Methods) was cor- related with the measured straightening in feature space in each layer of the robustly trained models (Figure 5; also see A.5, Figure 14). We found, however, that there seemed to be different mech- anisms at play for RS versus AT in terms of achieving straightening. RS models showed (small but) positive correlations, which means the smaller the ball containing all the frames of the movie in input space, the larger the straightening effect for the representations of frames of that movie in the model. While in AT models we see the opposite (negative) or no correlation. These divergent patterns underscore differences between these models and suggest that geometric size in pixel space is not strongly constraining the degree to which movie can be straightened.
Geometry of movie frame representations in feature space is relevant for capturing neural representations in VI Between different RS models tested different input noise levels, RS L2 x² = 0.5 stands out as it gives a better model of VI than those using smaller or larger magnitude input noise (Figure 4). For this model, we found that in addition to its intermediate level of straightening, the expansion score of movie frames, which is the radial size in its represen- tation normalized to size in the same layer of baseline ResNet50, was highest compared to the
8

---

AStraightning (re: Standardd RSD)
AExpaasion Siandard R30)
ABrain- BBrain-Scoree Score (re: Standard
01-62
18
21-10
005-
000-
conv2
eyen
layar2
ergpooy
conv2
laprr
aayrza
layer-
oyer4
conv
syerr
fc
Figure 6: Geometric characteristics, straightening and curvature, of RS models related to VI
plainability. A means quantity is referenced to the same measure in a standard ResNet50.
other RS models (Figure 6, middle panel; measures are referenced to layers standard ResNet50 to highlight relative effect of robustness training rather than effects driven by hierarchical layer). This demonstrates a potential trade-off between improving straightening in a representation while avoiding too much added contraction of movies by robust training relative standard training. This balance seems to be best achieved for o² = 0.5, where we also see the significantly higher predic- tivity of VI cortical data (Figure right panel). The best AT model also shows little contraction of movies coupled with high straightening (A.5, 15).
DISCUSSION
We have demonstrated novel properties of robust neural networks how they represent natural movies. Conceptually, this work establishes a seemingly surprising connection between disparate ideas, robust discriminative networks trained on static images on one hand, to work learning to linearize by training natural movies, the other. These modeling paths could both result in linearized, or straightened, natural movie representations (Figure 1). From machine learning per- spective, the invertibility and concomitant representational straightening of robust networks suggests that they learn explainable representations of natural movie statistics. Biologically, the emergence of straightening these networks as well their ability better explain V1 data than baselines relatively lacking in straightening Figure 4 provides new insights into potential neural mechanisms for previously difficult explain brain phenomena.
Biological constraints could lend parsimony to selecting among models, each with different engi- neering goal. On face, RS by virtue of utilizing Gaussian noise instead of engineered noise gains traction over adversarial training as a more simple, and powerful way of achieving robustness in ANNs, which is inline with a long history of probabilistic inference in visual cortex of humans (Pouget al., 2013). Indeed, looking across the range of robust models tested, the best fitting model of VI was not necessarily the most robust but tended toward more straightened representations that also showed the least contracted representations consistent with known dimensionality expansion from the sensory periphery to V1 in the brain (Field, 1994). Future work exploring a wider variety of robustness training conjunction with more bioplausible architectures, objectives, and training diets may yet elucidate the balance of factors contributing to biological vision.
At the time, our work does directly address how straightened representations in the visual system may or may not be utilized to influence downstream visual perception and behavior, and this connection is an important topic for future work. On the one hand, for supporting dynamical scene perception, behaviors that predict (extrapolate) or postdict (interpolate) scene properties over time (e.g., object position) may be supported by straightened natural movie representations. Indeed, both explanations, prediction and postdiction, have been invoked to account for psychophysical phe- nomena like the flash-lag illusion which present an interesting test case of how the brain processes complex stimuli over time (Cagleman & Sejnowski, 2000). However, even for relatively stationary scenes such as those containing textures, we observed benefits for straightening and invertibility in robustly trained networks (see A.4, Tables 2 and 3). Further work is needed to explore how spatially local versus global features the presence of simple versus complex motion are affected in their relative straightening by model training.

---

ACKNOWLEDGMENTS
This work was supported by Stingenstein-Simons fellowship, Sloan Foundation fellowship, and Grossman-Kavii Scholar Award as well as a NVIDIA GPU grant and was performed using the Columbia Zuckerman Axon GPU cluster. We thank all three reviewers for their constructive feed- back that led to an improved final version of the paper.
10

---

REFERENCES
Shahab Bakhtiari, Patrick Mineault, Tim Lillicrap, Christopher C Pack, and Blake Aaron Richards. The functional specialization of visual cortex emerges from training parallel pathways with self-supervised predictive learning. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021. URL https:
/
Pouya Bashivan, Kohitij Kar, and James DiCarlo. Neural population control via deep image synthesis. Science, 364(6449))eaav4336 May 2019.
P Berkes and L Wiskott. Slow feature analysis yields rich repertoire of complex cell properties. J. Vis., 5(6):9-9, July 2005.
Chun-Fu Chen, Rameswar Panda, Kandan Ramakrishnan, Rogerio Feris, John Cohn, Aude Oliva, and Quanfu Fan. Deep analysis of enn-based spatio-temporal representations for action recogni- tion. In The IEEE Conference Computer Vision and Pattern Recognition (CVPR), June 2021.
Ricky T. Q. Chen, Jens Behrmann, David K Duvenaud, and Joern-Henrik Jacobsen. Residual flows for invertible generative modeling. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d d'Alché- Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, vol- ume 32. Curran Associates, Inc., 2019. URL https pttp//eeeeeeedeeeeeeooeepcc/. neurips..coo file/ waleee/d0055994400 F00 00 Pawp:eeaww.adaaaape.p.... pdf.
paper/ /ttp://200///lle/66055922222
Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning. March 2020.
Radoslaw Martin Cichy, Aditya Khosla, Dimitrios Pantazis, Antonio Torralba, and Aude Oliva. Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence. Sci. Rep., 6(1), September 2016.
Jeremy M Cohen, Elan Rosenfeld, and J. Zico Kolter. Certified adversarial robustness via random- ized smoothing, 2019. URL https: / //rxivivv..// org/ abg/abs/////a / 1902.. 02918.
Joel Dapello, Tiago Marques, Martin Schrimpf, Franziska Geiger, David D. Cox, and James J. Di- Carlo. Simulating primary visual cortex at the front of enns improves robustness image per- turbations. bioRxiv, 2020. doi: 10.1101/2020.06 154542. URL https: // www.//wwkkkk. biorxiv. org/cont htts:/oonennteer/l///2//22/
22 2020. 06. 16 154542.
James J DiCarlo, Davide Zoccolan, and Nicole Rust. How does the brain solve visual object
recognition? Neuron, 73(3):415 434, February 2012.
David M. Fagleman and Terrence J. Sejnowski. Motion Integration and Postdiction in Vi- sual Awareness. Science, 287(5460)2036-20888 March 2000. doi: 1216/cience..287 5460.2036. URL htthtt:: //ww://wwww//// science. org/doi/ full/10. /126/sccencee. 287. 5460. 2036. Publisher: American Association for the Advancement of Science.
Logan Engstrom, Andrew Ilyas, Hadi Salman, Shibani Santurkar, and Dimitris Tsipras. Robustness (python library), 2019a. URL https::t///.mm httpt //gtthub.////u.//e////// com. wwl.ccom///a/.. MMa//MaadryLa/aaa http//ooouustssssee...
Logan Engstrom, Andrew llyas, Shibani Santurkar, Dimitris Tsipras, Brandon Tran, and Aleksander Madry. Adversarial robustness as prior for learned representations. June 2019b.
Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Brandon Tran, and Alek- sander Madry. Learning perceptually-alignes representations via adversarial robustness. In ArXiv preprint arXiv:1906.0099 2019c.
David Field. What Is the Goal of Sensory Coding? Neural Computation, 6(4):559-601, July 1994. ISSN 0899-7667. doi: 10. 10.1162/neco.199. 1994.6.4.59,, URL https: or1/ 1162 1994. 4.59...
Nic Ford. Justin Gilmer, Nicolas Carlini, and Dogus Cubuk. Adversarial examples are a natural consequence of test error in noise. January 2019.

---

Karl Friston. The free-energy principle: rough guide to the brain? Trends Cogn. Sci., 13(7):
293-301, July 2009.
Bernd Gärtner. Fast and robust smallest enclosing balls. In Jaroslav Nešetřil (ed.), Algorithms ESA' 99, pp. 325-338, Berlin, Heidelberg, 1999. Springer Berlin Heidelberg. ISBN 978-3-540-
48481-3.
Ross Goroshin, Michael F Mathieu, and Yann LeCun. Learning to linearize under uncertainty. C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 28. Curran Asso- ciates, Inc., 2015. URL httpt//t///rrooeediigg..c pdf.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. CoRR, abs/1512.00336 2015. URL
1ttp:/www..r/aab//1.......... 1331000..33385 03385.
Olivier J Hénaff, Robbe L. T Goris, and Eero P Simoncelli. Perceptual straightening of natural videos. Nat. Neurosci, 22(6))984-991, June 2019.
Olivier J Hénaff, Yoon Bai, Julie A Charlton, Ian Nauhaus, Eero P Simoncelli, and Robhe L Goris. Primary visual cortex straightens natural video trajectories. Nat. Commun., 12(1):5982, October 2021.
Jörn-Herrik Jacobsen, Jens Behrmann, Richard Zemel, and Matthias Bethge. Excessive invariance causes adversarial vulnerability. November 2018a.
Jörn-Henrik Jacobsen, Arnold Smeulders, and Edouard Oyallon. i-RevNet: Deep invertible net- works. February 2018b.
Seyed-Mahdi Shaligh-Razavy and Nikolaus Kriegeskorte. Deep supervised, but unsupervised, models may explain IT cortical representation. PLoS Comput. Biol., 10(1 (e11))ee000991 November 2014.
Nathan C Kong. Eshed Margalit, Justin L Gardner, and Anthony M Norcia. Increasing neural network robustness improves match to macaque VI eigenspectrum, spatial frequency preference and predictivity. PLoS Comput. Biol., 18()):e0,,9739 January 2022.
Jonas Kubilius, Martin Schrimpf, Aran Nayebi, Daniel Bear, Daniel K Yamins, and James DiCarlo. CORnet: Modeling the neural mechanisms of core object recognition. September 2018.
Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. Certified robustness to adversarial examples with differential privacy. February 2018.
William Lotter, Gabriel Kreiman, and David Cox. Deep predictive coding networks for video pre- diction and unsupervised learning. May 2016.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. To- wards deep learning models resistant adversarial attacks. In International Conference Learn- ing Representations, 2018. URL ht tps::p:: wwwpoeviee....eee. net www.moormiioo faforummiddrJaBffffa..e
Beren Millidge, Anil Seth, and Christopher L Buckley. Predictive coding: A theoretical and experi- mental review. July 2021.
A Olshausen and D Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 38 381(683))66776699 June 1996.
Alexandre Pouget, Jeffrey M Beck, Wei Ma, and Peter E Latham. Probabilistic brains: knowns and unknowns. Nat. Neurosci., 100011170-117 September 2013.
R P Rao and D H Ballard. Predictive coding in the visual cortex: functional interpretation of some extra-classical receptive-field effects. Nat. Neurosci., 2(1):79-87, January 1999.
Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton. Dynamic routing between capsules. October 2017.
12

---

Martin Schrimpf, Jonas Kubilius, Ha Hong. Najib J. Majaj, Rishi Rajalingham, Elias B. Issa, Kohitij Kar, Pouya Bashivan, Jonathan Prescott-Roy, Franziska Geiger, Kailyn Schmidt, Daniel L. K. Yamins, and James J. DiCarlo. Brain- Which artificial neural network for object recognition is most brain-like? bioRxiv preprint, 2018. URL https://p///.. http://www..brrrro. www.:bioxxiv..rr. conten//nnt//////////mm 10.1 1001 00.1101440700v22 4070077v.
Martin Schrimpf. Jonas Kubilius, Michael Lee. Apurva Ratan Murty, Robert Ajemian, and James DiCarlo. Integrative benchmarking to advance neurally mechanistic models of hu- man intelligence. Neuron, 2020. URL https:: /ttp///wwww/../ www.well.....wwwww. www.////////oocc/eee wwwpnn//fuflltlttoooo
50896-6273 (20) 30605-X.
Andrea Tacchetti, Leyla Isik, and Tomaso A Poggio. Invariant recognition shapes neural represen- tations of visual input. Annu. Rev. Vis. Sci., 4:403-422, September 2018.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. July 2018.
Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600-612, 2004. doi: 10. 20011000
James C.R. Whittington and Rafal Bogacz. Theories error back- back-propagation in the brain. Trends in Cognitive Sciences, 23(3):235-255) 2019. ISSN 1364-6613. doi: https://doi.org/10..00 tii.tss... 2018.12.005 URL https: /www. wcwencedi con/sccience/arttcclee pii/ $1364661319300129
Daniel L. K. Yamins, Ha Hong. Charles Cadieu, Ethan A. Solomon, Darren Seibert, and James J. DiCarlo. Performance-optimizee hierarchical models predict neural responses in higher visual cortex. Proceedings of the National Academy of Sciences, 111((3):8619 8624, 2014. doi: 10. 1403112111111 14C3112111.
1003///aaa..411211111 URL https:tpp/...m h/tp::/wwwwwaaaaaee owg.orr///oooo.ooo http://///abb//../oo/// ab..ab.. http///1...00///meeeee/m ppas.
Jure Zbontar, Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Self-supervised learning via redundancy reduction. March 2021.
Chengxu Zhuang, Siming Yan, Aran Nayebi, Martin Schrimpf. Michael C. Frank, James Di- Carlo, and Daniel L. K. Yamins. Unsupervised neural network models of the ventral visual stream. Proceedings of the National Academy of Sciences, 11803):20144161118, 2021. doi: 10.1073/pnas 20144199 18. URL https: //www pnas wttp://///////o//0/////o 10t3///1...000/011 / 20141961
13

---

A APPENDIX
A.1
STRAIGHTENING FOR BOTH NATURAL AND ARTIFICIAL SEQUENCES
Curvature change of natural (degrees re: pixels)
Standard
Curvature change of artificial (degrees re: pixels)
30
60
30
60
Untrained RS0
-30
layer1
layer2
layer3
layerd
60
-30
Standard R101
supervised
Standard R152
AdvTrain
RandSmooth
60
330
-60 330
-60
Figure 7: ANNs show straightening representations when robustness to noise constraints (noise augmentation robustness adversarial attack) is added to their training. Counterclockwise from top left, measurements for straightening movie sequences (from (Hénaff et al., 2019), natural sequence: green, artificial sequence: black) in each layer of ResNet50 architecture un- der different training regimes: supervised training (standard), supervised training with adversarial training (I2, o² = 3) (Engstrom al., 2019a) and supervised training with noise augmentation (L2, 0² 0.5) (Cohen et al., 2019). Top right shows straightening for artificial (open circles) and natural (closed circles) sequences using ResNet architecture with no training (random parameters), self-supervised training (Chen et al., 2020) or additional layers.
14

---

A.2
STRAIGHTENING IN OTHER ARCHITECTURES
RasnetSO_TAM TS2
modes
Resnet101, 01_TAM
Quiver ure change (fngreee re pleeld)
Figure 8: Straightening for ResNet50 and ResNet101 trained the base architecture for action recognition (Chen et al., 2021) They were trained on video clips for action recognition. Although these models were not trained for straightening or predicting the next frame, they exhibit small but measurable straightening for natural movies. However, the curvatures for artificial sequences were increased as much as curvature increase for artificial sequences in robust neural networks (Figure 7).
Vone_Reanet-Sa 50
Durviture change (clggroes ne pizels)
Artifiaal mones
Artifciall
-21
Figure 9: Lack of straightening for natural movies in ResNet50 trained with a biologically-inpping model of V1 in the front-end (Dapello et al., 2020). Vone-ResNet50 exhibits robustness to adversar- ial attack, but the fact that it does not exhibit straightening (except for the front-end) provides further evidence that adversarial robustness does not always accompany straightening.
15

---

A.3
MORE ON NEURAL DATA PREDICITIVITY
Primary visual cortex
r=0006 (p=0.2)
r=0.28 (p=2.9e-12)
Neural predicitivity (V1 explained variance)
0.3
0.3
0.1
0.1
20
40
60
0
20
40
60
Clean accuracy (%)
Robust accuracy (%)
Figure 10: Clean accuracy (left) and robust accuracy (right) vs. V1 predictivity (same color conven- tion as used in main text).
Cortical area V1
Cortical area
Neural predicitivity (explained varfance)
0.38
0.35
0.39
025
0.
0.20
0.35
028
0.30
018
0.55
Nagthe abuan
0.8
Cortical area V4
Cortical area IT
06
0.5
0.0
04-
0.45
01
layer 3
layer 4
0.25
layer
layer
Brain-score (data and metric publicly available (Schrimpf et al., 2018)) for the models used this study. The bar plot Figure 4 is summary of this plot. Since focus this work was on V1 which is the only visual area for which neural straightening has been measured (Hénaff et al., 2021), we measured the brain-score for more layers for V1.
Figure 11:
16

---

A.4 MOVIE CHARACTERISTICS AND ADDITIONAL INTERPOLATION EXAMPLES
A.4.1 MOVIES
Figure 12: Movies used evaluate straightening and invertibility (11 frames each).
17

---

A.4.2
TABLE FOR AVERAGE STRAIGHTENING
Table 2: Average curvature change (re: pixels) for each movie. RN stands for ResNet. The architec- ture of all robust models used was ResNet50.
water
cam.
walk.
dogv.
egomo.
chiron.
hees
leaves
smile
chiron.
prair.
RNI8
45.29
36.25
-27.58
12.39
-34,82
13.30
-5.94
-8.03
9.28
48.45
RN34
36.63
-2.43
-35.67
12.39
-5.81
-8.65
48.14
RN50
48.85
37.33
-28.13
14.00
-34.68
13.45
-5.66
-8.53
10.79
0.76
49.72
RN50 Self-sup
52.05
40.32
-2.733
18.98
-33.19
14.37
-5.68
-8.22
19.42
0.52
49.90
RN50 MocoAugm
49.03
36.15
-26.04
16.75
-32.88
13.82
-5.22
-57.69
14.00
0.57
49,30
RN50 NoAugm
46.38
33.64
-24.79
15.89
-33.58
14.71
-5.10
-57.32
8.87
48.31
RN101
48.81
37.73
-35.03
12.68
-6.17
-1.05
49.63
RN152
48.91
40.38
-29.09
13.27
-35.53
12.23
-6.60
-99.75
13.80
-1.00
50.68
AT Ly c
17.43
2.78
40..66
-3,61
-55.42
-2.54
-13.44
-73.01
-25,09
-13.06
30.34
AT Loo 1E= 4
25.93
16.51
-36.55
3.48
-50.46
5.45
-12.90
-68.74
-11.49
-7.91
38.46
Loo0 : 8
25.64
13.15
-40.79
1.10
-54.99
4.11
-15.59
-71.48
-17.99
-10.23
39.09
E 0.25
12.23
-31.81
5.38
-45.86
-9.37
-63.84
-9.08
-5.22
RS 0.5
16.25
1.34
-3.65
1.76
-51.28
0.80
-12.57
-70.21
-14.35
-10.09
24.26
RS L2 :
10.65
-0.61
-42.40
-7.27
-58.04
-4,68
-66.57
-79.82
-23,55
-14.62
18.03
A.4.3 INVERTIBILITY MEASURE FOR EACH MOVIE
Table 3: Average SSIMs for each movie. RN stands for ResNet. Architecture of all robust models used was ResNet50.
waler
cam.
walk.
dogv.
egomo.
chiron.
bees
leaves
smile
chirono.
prair.
RNIS
0.47
0.34
0.21
0.22
0.13
0.28
0.31
0.20
0.29
0.31
0.34
RN34
0.46
0.34
0.21
0.22
0.13
0.27
0.30
0.20
0.29
0.30
0.34
RN50
0.47
0.34
0.22
0.22
0.14
0.28
0.30
0.20
0.29
0.31
0.34
RN50 Self-sup
0.49
0.34
0.22
0.23
0.14
0.29
0.33
0.33
0.34
RN50 MocoAugm
0.44
0.20
0.21
0.12
0.25
0.27
0.28
RN50 NoAugm
0.43
0.32
0.20
0.21
0.12
0.26
0.32
0.28
0.28
0.32
RN101
0.48
0.35
0.22
0.23
0.14
0.28
0.31
0.21
0.30
0.30
0.35
RN152
0.49
0.36
0.22
0.23
0.14
0.29
0.31
0.21
0.30
0.31
0.36
AT L2 E 3
0.76
0.59
0.52
0.67
0.22
0.62
0.58
0.35
0.81
0.63
0.59
AT Loc =
0.72
0.56
0.43
0.60
0.25
0.57
0.53
0.32
0.69
0.55
0.56
AT Lix :
0.78
0.59
0.45
0.66
0.29
0.65
0.56
0.35
0.78
0.61
0.62
RS La : 0.25
0.53
0.48
0.35
0.47
0.18
0.44
0.44
0.29
0.57
0.46
0.42
RS L2 : 0.5
0.60
0.49
0.37
0.51
0.22
0,44
0.45
0.30
0.54
0.49
0.46
RS L.2 =
0.50
0.35
0.53
0.24
0.44
0.48
0.30
0.52
0.49
0.48
18

---

A.4.4 ADDITIONAL MOVIE INTERPOLATION EXAMPLES
Real
Real
Pixels
Pixels
Standrd
Standrd
RS
RS
AT
AT
Real
Pixels
Standrd
RS
AT
Figure 13: Three more example for interpolations for movies: chironomous. dogville, and egomo- tion, respectively. The gray dot in the middle of all frames known as fixation spot where subjects (humans or monkeys) are instructed to keep their gaze toward during the experiment.
19

---

A.5
EXPANSION METRIC VS. STRAIGHTENING ACROSS MODELS AND LAYERS
Uyyel
layer?
Syerr
Inyert
50
26
26
Curvature (re: pixels)
10
20
Standaid
Staneis
Standaid KAB
10-
Acctrac
Acxtion ta rfhedd
Acstrinn officel
Fandorrohh kgrol 0ad
RendOmashh coc w0.00
Kandor wo00
Expansion (re: pixels)
Figure 14: For each layer in each model, the expansions (re: pixels) and curvature change (re: pixels) were plotted for first row: all movies, second row: average over movies.
AStraightening Standard R50)
ABrain-Sooi V1 (re: Standard R50)
AExpansion (re: Standard R50)
10-
2.04
2.22
Adversorial
-1.0
-1as
convi
layera
layera
layare
augpool
[c-
syer1
4942
1900
conv1
layer1
laye2
loye-34
Figure 15: Geometric straightening and expansion of AT models and their relation VI explain-
ability. means quantity is referenced to the same measure in standard ResNet50.
20

---

A.6 REPRODUCIBILITY INFORMATION
Almost all data (models, movies, and metrics) used in this work are publicly available and we pro- vided references them in the text (for instance Table 1). We will release the code reproduce the main results in this work at https: h//gittus com/te (toosi/Braing e and we provide pointers to the publicly available resources used in this work as listed below. Movies and images. We used the same movies used the original studies on human perception and monkey primary visual cortex (Hénaff et al., 2019; 2021) which are available from first author Github as referenced in their papers. Images used measure the clean accuracy and robust accuracy were taken from ImageNet validation set.
Models. All the models used in this study were from ResNet family and checkpoints for the main robust models the only two custom trained models (supervised with no augmentions and supervised with Moco augmentation) will be made publicly available along with the code.
publicly available as references in the main text (Table 1). The checkpoints for
Neural predictivity metric. We used brain-score, which is publicly available benchmark to eval- uate how well model predicts variance in neural data (Schrimpf et al., 2018).
21


Reviews:
[{"review_id": "kX2hiQ4V8qb", "reviewer": "Reviewer_JB2C", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "[response to author's rebuttal: see https://openreview.net/forum?id=mCmerkTCG2S&noteId=mpYhm0LBTQD]\n\nThe paper \"Brain-like representational straightening of natural movies in robust feedforward neural networks\" investigates to which degree standard ImageNet-trained models and \"robust\" networks (in terms of adversarial training, training with random smoothing) straighten representational trajectories of short movie sequences, which is an effect that is known from neuroscience. They find that robust networks indeed straighten representational trajectories. A few other experiments investigate links to fitting brain data and inverting representations.", "summary_of_the_review": "I am leaning towards acceptance since investigating the link between adversarial training / noise robustness and representational straightening is a clear, novel and important finding. At the same time, the paper currently has a number of issues that should be improved (see suggestions above). I would be happy to raise my score if my concerns are sufficiently addressed.", "strength_and_weaknesses": "Strengths:\n+ establishes a connection between adversarial training/random smoothing and perceptual straightening, which is novel and very interesting\n+ consistent color scheme, (mostly) clear figures, especially Figure 2 conveys the main point of the paper\n\nWeaknesses:\n- Brain-score comparison may be impacted by different baseline accuracies between adversarially trained and RS models (had to tell, since I couldn't find the RS model's baseline accuracy in the paper)\n- some claims / wording not backed up by data (\"brain-like representational straightening\")\n- multiple issues regarding clarity, see suggestions below\n- poor reproducibility\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "k9PG2jTFOM", "reviewer": "Reviewer_ePuu", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents an interesting and important finding regarding representational straightening of natural movies in artificial neural networks (ANN). The authors showed that representational straightening, which was previously shown in human perception of natural movies and neuronal representations in primary visual cortex, doesn’t necessarily require predictive training of ANNs with movies, and can emerge in ANNs trained with robust supervised learning algorithms (image classification plus adversarial training or random smoothing). \n", "summary_of_the_review": "$\\textbf{Questions}$:\n\n1- The volume of the hypersphere in the latent space seems to have a relationship with the representational curvature of movies, based on the results presented in the paper (figure 6). Can the authors show a more thourough comparison of curvature and expansion metrics across different models and different layers of the models? It’s possible that the measured curvatures are, in general, smaller in more contracted latent hyperspheres which would also explain the results shown in figure 6.\nThis would also suggest an explanation for the observed relationship between robust training and straightening: robust training pushes similar images (eg consecutive frames of a movie) close together in the latent space. This could be considered as an implicit predictive training for movies, which is not dissimilar to self-supervised learning algorithms such as SimCLR and contrastive predictive coding. I'm interested to hear what the authors think about this possibility. \n\n2- The outperformance of the RS model in accounting for V1 responses is very interesting. Given the significant difference between RS and AT in explaining V1 responses, how could we know if the outperformance of RS is caused by robustness, and not specifics of the RS algorithm? \n\n3- The inverted linear interpolation experiments are very successful in giving a more direct evidence for the straightened representations learned via robust training. However, the SSIM metric used to quantify the similarity of natural movies and synthetic sequences mainly relies on lower-order statistics of the sequences. One could imagine that perceptual straightening relies on more abstract representations, in addition to lower-order statistics. In an ideal situation with enough time for more experiments, human observers would be the best evalutors for sequence similarities. In the absence of that possibility, could the authors provide more generated samples in the appendix. Especially, samples where the first and the last frames are more visually different. Also, which layers of the ANNs were used for the linear interpolations? How would the results look like for the early, intermediate, and late layers (both the appearance of the sequences and the SSIM results)? The most straightened representations seem to be in the intermediate layers. Would we also get the highest SSIM with the intermediate layer interpolations?\n\n\n$\\textbf{Minor Comments}$:\n\n1- I suggest that the authors check the citation formats throughout the manuscript. There are a number of cases where the citation format is not correct, or the reference’s authors names is attached to the word before with no space in between. For example, see the last two lines of page 3. \n\n2- figure 2 right: why not all AT models are shown in the curvature plots?\n\n3- figure 5 caption: “Positive correlation means the smaller the correlation,…” correlation doesn’t seem to be the right word here.\n\n4- page 7 last paragraph: “Between different RS models tested on different input noise levels, RS L2 stands out…” which RS L2?  Based on table 1, all RS models are L2.\n", "strength_and_weaknesses": "$\\textbf{Strengths}$: \n\nThe results of this paper shed light on an important, previously observed phenomenon: namely, natural movie sequences are straightened both in human perception and primate V1 which is beneficial for linear prediction. These empirical observations implied that perception and neuronal representations underlying it are optimized for prediction of natural visual steams. Therefore, both predictive training and natural movies are both necessary for learning straightened representations in ANNs. This paper, interestingly shows that neither of these two factors are necessary. Rather, natural, static images and robust training are sufficient for learning straightened representations in ANNs. In addition to measuring the curvature of latent representations in a trained ANNs, authors use different metrics and experiments to explore this phenomenon from different viewpoints, which make the findings even more convincing. On top of that, the paper presents a computational model for primate V1 that finally shows a significant improvement compared to the previous models. \n\n$\\textbf{Weaknesses}$:\n\nAlthough the empirical results shown in the paper are quite convincing, the paper doesn’t provide an intuitive understanding of why robust training leads to representational straightening of movies. The ANNs don’t see any natural movies during training, and there is no direct (or indirect) intuitive relationship between noise robustness in pixel space and linearizing sequence of frames in latent space. In the absence of such an explanation in the paper, my own efforts for finding an explanation directed me toward a possible relationship between robust training of natural images and predictive training of natural movies. I elaborate below in my comments.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "bHz8B41qDN", "reviewer": "Reviewer_J1tD", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper tackles the question of straightening of natural movies in deep neural networks. It starts from previous observations both in visual psychophysics and neurophysiology that the curvature of the internal representations (abstract perception, V1 neural activity) is lower than the one of the initial movie representations (evolution of pixels in space-time). Such a reduction in curvature is not observed in the feature space (ouputs at different layers) of deep neural network (resnets) with standard training on large image datasets.\n\nThe authors shows that robust training (adversarial and random smoothing) is sufficient to obtain a curvature reduction in the feature space. According to the authors, this observation stems from a notion of \"invertibility\" which is used to describe a neural network in which linear interpolation in the feature space lead to perceptually smooth interpolation in the image space. They show that interpolating between movie frames generate perceptually plausible movies (using SSIM). Finally, the authors also shows that such robustly trained (random smoothing) neural networks are better at explaining the variance of neurophysiological recordings in the primary visual cortex.\n", "summary_of_the_review": "This a good empirical contribution for ICLR filling a gap between vision studies and artificial neural network. The connection between a static feature of NN (robustness) and a dynamic feature of perception is appealing.\n\nI will recommend acceptance more strongly once the authors have accounted for my remarks (ie improving the discussion section and referring to supplementary figures + fixing typos).\n\n", "strength_and_weaknesses": "Strength:\n- leverage behavioral and biological vision studies to question deep neural networks,\n- relate robustness to input noise to perceptual straightening of movies,\n- surprising observation : networks trained robustly on static images reproduces an observed dynamical feature of perception  \n- compare two types of robustness (random smoothing, adversarial training). \n\nWeaknesses:\n- incremental work\n- results are not sufficiently discussed with the visual perception literature ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster