Exploring perceptual straightness in learned visual representations
Authors: Anne Harrington
Vasha DuTell
Ayush Tewari
Mark Hamilton
Simon Stent
Ruth Rosenholtz
William T. Freeman
Venue: ICLR 2023 poster (2023)
Keywords: adversarial robustness
deep learning
representation learning
computer vision
neuroscience
human vision

Abstract: Humans have been shown to use a ''straightened'' encoding to represent the natural visual world as it evolves in time (Henaff et al. 2019). In the context of discrete video sequences, ''straightened'' means that changes between frames follow a more linear path in representation space at progressively deeper levels of processing. While deep convolutional networks are often proposed as models of human visual processing, many do not straighten natural videos. In this paper, we explore the relationship between network architecture, differing types of robustness, biologically-inspired filtering mechanisms, and representational straightness in response to time-varying input; we identify strengths and limitations of straightness as a useful way of evaluating neural network representations. We find that (1) adversarial training leads to straighter representations in both CNN and transformer-based architectures but (2) this effect is task-dependent, not generalizing to tasks such as segmentation and frame-prediction, where straight representations are not favorable for predictions; and nor to other types of robustness. In addition, (3) straighter representations impart temporal stability to class predictions, even for out-of-distribution data. Finally, (4) biologically-inspired elements increase straightness in the early stages of a network, but do not guarantee increased straightness in downstream layers of CNNs. We show that straightness is an easily computed measure of representational robustness and stability, as well as a hallmark of human representations with benefits for computer vision models.

Published as a conference paper at ICLR 2023
EXPLORING PERCEPTUAL STRAIGHTNESS IN LEARNED VISUAL REPRESENTATIONS
Anne Harrington 1.2
Vasha DuTell Ayush Tewari'
Mark Hamilton
Simon Stent
Ruth Rosenholtz!.2
William T. Freeman'
MIT CSAIL MIT Brain and Cognitive Sciences
Woven Planet
fannekh, vasha @mi @mit.edu
ABSTRACT
"straightened" encoding to represent the nat- ural visual world as it evolves in time (Hénaff et al. 2019). In the context discrete video sequences, "straightened" means that changes between frames fol- low a more linear path in representation space at progressively deeper levels processing. While deep convolutional networks are often proposed models of human visual processing, many do not straighten natural videos. In this paper, we explore the relationship between network architecture, differing types robust- ness, looogically-inspiree filtering mechanisms, and representational straight- ness in response to varying input; we identify strengths and limitations straightness as a useful way of evaluating neural network representations. We find that (1) adversarial training leads to straighter representations in both CNN and transformer-basee architectures but (2) this effect is task-dependen not general- izing to tasks such as segmentation and frame-predd crame-prediction, where straight represen- tations
Humans have been shown
favorable for predictions; and nor other types of robustness. In addition, (3) straighter representations impart temporal stability to class predic- tions, even for -ittr--diittibution data. Finally, (4) biologically-inspired elements increase straightness in the early stages of a network, but do not guarantee in- creased straightness in downstream layers of CNNs. We show that straightness an easily computed measure of representational robustness and stability, as well as a hallmark of human representations with benefits for computer vision models.
INTRODUCTION
Visual input from the natural world evolves over time, and this change over time can be thought of as a trajectory in some representation space. For humans, this trajectory has a different representation at varying stages of processing, from input at the retina to brain regions such as V1 and finally to perception (Fig D. We can ask whether there are advantages to representing the natural evolution of the visual input over time with a straighter, less curved, trajectory. If so, might expect that human vision does this. (Hénaff et al., 2019) demonstrated that trajectories are straighter in human perceptual space than pixel space, and suggest that straighter representation may be useful for visual tasks that require extrapolation, such as predicting the future visual state of the world.
Learning a useful visual representation is one of the major goals of computer vision. Properties like temporal stability, robustness to transformations, and generalization - all of which characterize human vision are often desirable in computer vision representations. Yet, many existing computer vision models still fail to capture aspects of human vision, despite achieving high accuracy on vi- sual tasks like recognition (Feather et al. 2019 Hénaff et al., 2019). (Hénaff al. 2019) was found that, while biologically-inppiee pared to the input domain, popular computer vision models such as the original ImageNet-trainee AlexNet (Krizhevsky et al. 2017) do not.
Vi-like transformations yield straighter representations com-
In an effort to achieve favorable numan-vision properties for computer vision models, there has been much work dedicated to incorporating various aspects of human vision into deep neural networks. These include modifying network architectures to mimic aspects of the human visual system pystem/Huang & Rao (201)), for example incorporating filter banks similar to the receptive field properties of visual

---

Published as a conference paper at ICLR 2023
Trajectory Over Time
Representation
Video Frames in Pixel Space
Retina,
V1, 2nd Stage Deep
Perception, Final Stage Deep Network
1st Stage Deep Network
Figure 1: Schematic illustration of the representation of discrete video sequence becoming pro- gressively straighter as information is processed through a visual processing pipeline, starting from the highly nonlinear trajectory of typical video frames in pixel space.
neurons Dapello et al. 2020), as well as enforcing activation properties similar to those seen in visual cortex such sparsity Wen et al. (2066). Another promising avenue has been in modifying training to include adversarial examples. By directly targeting areas of vulnerability, adversarially robust networks show increased representational robustness, more closely aligning their performance with that of their human counterparts (Engstrom et al. 2019b).
To understand if attempts improve neural network models lead to representational straightness, we explore how well models with different architectures, training schemes, and tasks straighten tempo- ral sequences. We evaluate a variety of network architectures, both biologically and oon-biologically inspired, for representational straightness across layers, we then ask whether training for adversarial robustness in both CNN and transformer-bast based architectures may lead to the straighter representa- tions generated by human vision. Because DNNs learn early representation that differs from what known about human vision, we also ask if hard-coding that early representation might lead to trained network with more straightening downstream. We find that straightness is useful tool that can give intuitions about what allows models to adopt representations that mirror the ad advantagcous qualities of human vision, such stability.
2 PREVIOUS WORK
Deep neural networks have been proposed as models of human visual processing, owing to their ability to predict neural response patterns (Yamins DiCarlo 2016 Rajalingham et al. 2015 Kell & McDermott 2019). As such, there has been much effort to improve the alignment of deep net- works to human vision by incorporating known aspects of the human visual system. Some these include: simulating the multi-scale V1 receptive fields of early vision (Dapello et al.) 2020). adding foveation using a texture-like representation in the periphery at CNN's input stage (Deza & Kon- kle 2020), and incorporating activation properties of visual neurons such sparsity Olshausen &
Field 1997 Wen et al. (2016). Predictive coding, often attributed to biological networks Huang &
Rao 2011). has been incorporated into deep networks trained to perform tasks such video frame prediction Lotter et al. (2016), using layers that propagate error-signals.
The desire evaluate the effectiveness of these techniques at creating models of the human visual system, motivated the creation of measures like BrainScore [Schrimpf et al. (2020) that compare models humans using neural and behavior data. addition, number perceptual experiments have been used to compare human and model representations (Berardino et al. [017 Feather et al.) 2019 Harrington & Deza 2022). However, perceptual approaches in particular can require lengthy stimuli synthesis procedures and the of human participants to probe each model's representation. The straighiness/curvature measure, however, is a quick and computationally powerful quantitative measure of how well model aligns with properties of human visual representations Hénaff et al.
2019), particularly in terms of temporal stability.
One important area in understanding how humans and DNNs differ lies in their response to adver- sarial examples (Elsayed et al., 2018 Ilya et al. 2019; Feather et al.) [022 Dapello et al.] [02T). Adversarial examples, which modify images with changes that imperceptible humans,
2

---

Published as a conference paper at ICLR 2023
cause network to misclassify image (Goodfellow al. 2014 Szegedy et al. 2013). Adver- sarial training (adrr et al. 2017) improves the misclassfication problem and has been suggested to help networks learn visual representations that are more perceptually aligned with humans En- gstrom et al. (019b);; Ilyas et al. 2019). If adversarial training leads to models that are more aligned with human perception, perhaps they also learn straight representations like humans do. Adversarial training schemes, however, are not biologically plausible, and recent work has identified mecha-
nisms that are better supported by vision science (Dapello et al. 2020 Guo al. 2022 Dapello et al. 2021
In the context of representation learning in computer vision, adversarially robust models have also been shown to do better at transfer learning than their non-adversarially robust counter- parts (Davchev al., 2019), and adversarially robust features can used directly for tasks like image generation and in-painting (Santurkar et al. 2019). Adversarial training schemes have also been developed for tasks other than classification like semantic segmentation (Xu et al.] 202T). In this paper, we build work around adversarial robustness by evaluating if increasing this robustness leads to straightened representations like those found in human spatiotemporal processing.
There have been various efforts to incorporate constraints that reduce curvature and increase straight- ness in temporal representation learning. Slow feature analysis (Berkes Wiskott 2005) encourages learned representation to change slowly over time, but does not encourage straightness explicitly. Another early attempt to add a straightness constraint to representation learning (Goroshin et al.] 2013) focused video frame prediction for unsupervised learning. We follow in the same vein of this work, evaluating straightness as a metric for encouraging a robust, stable, representation when learning to solve tasks.
3 METHODS: MEASURING STRAIGHTNESS AS Low CURVATURE
1/
P/41
C
Figure 2: Illustration of how the curvature measure is distinct from cosine similarity. (Left) Three points are sampled along a trajectory in time (I;-1, Tt, T++1). The angle between neighboring x samples is their cosine similarity. (Right) Curvature o is the angle between U1 and 2t-1, where t is the difference between Ię and øt-1-
Representational straightness can evaluated as a reduction in curvature. For a temporal sequence, such as a video, curvature is defined as the angle between the vectors representing the difference between consecutive frames. Let x refer to a representation of video length T, with ze being a representation of one frame of video at time step The representation may be at any stage of the processing pipeline, from a vector of raw input pixels from the video frame, to the activations of a network's 's hidden layer. Then, 74 represents the difference between successive frames. We find the curvature at time f by finding the angle between successive i, which we call e (equation The global curvature of video sequence is then simply the mean angle over all time steps. Note there are T - 1 time steps due to curvature being evaluated on frame differences (equation 2).
Up = Fz It-1,
0z
Uj
= arccos (0z iz41)
Global curvature :=
Σ
4
(2)
t=
This is the formulation proposed by (Hénaff et al. 2019). One can compute this global curvature for any representation of a video sequence over time, either on the vector of pixels (likely not very straight), or one can apply it to a representation of that video, e.g. at any layer of a neural network model. In this paper, often report change in curvature with respect the pixel-value input to show if model's representation is more straight at later stages of processing. Change in curvature is the representational curvature minus the pixel curvature.
3

---

Published as a conference paper at ICLR 2023
Note that curvature is distinct from simple cosine similarity in that curvature is calculated frame differences (Uy), whereas cosine similarity depends on the angle between the frame vectors them- selves (z+). Curvature can be thought of as first-order variant of cosine similarity (see Figure and Sec A.I. For a complete description of how curvature was determined for each model, see Sec A.2.2 We also explore the effect of principle components analysis curvature in Sec AA2.4 and provide a note on compute requirements in Section A.2.3
Output. Curvature Change
Non-Robustt Cless
Siv Robust Cless INC Robust Class
Noa-Itobust Sgmt
Video Pred Hybrid Bio
Non-Param
30
Adv Robust Class
Robust Sqmtt
20
10
-30
upr Sondard vogi S.Snndar IN Stapdard feee Reeeeeddn stanntoon
Resheee Seeehheessi DINO by Beshier
By
Crosa Siff-Staddrri wondddrr--anaa bent tNt Grou Croullyy lecation
AlessNos Stannttttttiio and de IN Ros
Shoton All Afiehuntt Rus BusNocch A did Ce
IN
Rosc RosNo Me RRRbbnt 120.5. SC PP
Mesett.frrtteee Atrationn Pettiil tV
A Aaresaa-fftiine Affebust Shiflfabuat th vigu Mell--Shii
Be peggottl conttna Io
Pethi N
SOLG HAPP
MAE SAE-Standand in Ste tal
pap pep Standord NDD
DLJ Standard OfF DL3
Stander of
pap Ap Aftihmnt Ofr
Allahaahii at MilevOo Pag-AAAAteee Soahust--dee DD PSr.
part Atebont ddle Cfr DL AAAaatin
FDC
Di 3-Atobiun out VVC.
DE DL A.Aaobn dade ddlocCf
Prediiet L0 sirratghh ti. KTT
sut Off
Call KITT
W' Pant VChoNee CorNette by
Mouteee--o Extran- (n
Lakkir KITT
1'i Alsoney By
Perm Permi 2e--ii Perppii H
Rosk Ros Midbunt mnd
Vol A Alobani 12-0 ARobust
Prodiiot LO L0-P2-att
alox
P
Rea
Resll Ainii
Ros
Croas Wil AI
Figure 3: Output Representation layer curvature for a variety of network architectures and training schemes. The x-axis ticks describe different models and are formatted as: [ar- chitecture] [training scheme] [training dataset]. The datasets are IN:ImageNet (Deng et al. 2009) RIN:RessreteedImggeeee (Santurkar et al. 20T9).CF:CIFARR VOC.PASCAL VOC 2012 (Everingham et al.) KITTEKITTI (Geiger et al. 2012). Exact model details can be found in the supplemental data. The y-axis shows curvature of each model at the output representation layer relative to the curvature of the input video. For image classification models the output representation is the penultimate layer.
(Krizhevsky et al. 2014), 2010), CrT:Cityscapes (Cordts et al., 2016),
CURVATURE AND ROBUSTNESS IN CONVOLUTIONAL OBJECT RECOGNITION MODELS
We measured curvature for variety of models (Sec A.2. to investigate the relationship between model type and output curvature. As shown in Figure 3 we find non-adversarially trained image recognition models have the highest output curvature. All adversarially trained models have lower curvature than their non-adversarially trained counterparts (Sec A.4, as well as overall, with the majority reducing output curvature below that of the input pixels. Self-supervised DINO (Caron let al. 2021) models have similar output curvature values to their supervised counterparts despite DINO models having been shown to have more semantically meaningful feature correspondences.
4.1 ADVERSARIAL ATTACK TYPE, STRENGTH, AND CURVATURE
Given the increased straightness seen in adversarially robust image recognition / object detection models, we investigated the relationship between the type and strength of adversarial attack and the resulting curvature of the model's output. To evaluate the effect of these attacks on curvature, we compare a set of ResNet50 networks (He et al.) (016)), trained on CIFAR-10 (Krizhevsky et al.) 2014), ImageNet (Deng et al. 2009). and Restricted ImageNet (a subset of ImageNet (Engstrom et al. 2019b Myas et al. 2019). with and without adversarial training and measure the output curvature 3 Left). The adversarially trained models are trained using projected gradient descent with l2 or Lo norms at different perturbation levels (Madry al. 2017; Engstrom al. 2019a). We use ResNet50 adversarially trained with projected gradient descent because that model type has been shown be more aligned with human perception (Engstrom al. 2019h Feather al. 2022
4

---

Published as a conference paper at ICLR 2023
Adversarially Robust Classification Models Layer-Wise Curvature
Curvature Change
40
ResNet50 5ttritdddSaaaardl IN
ResNet50 ResNet50_Standdard__ CF
CrussViT CrussViT_Standardd_ PathO IN
20
0
-20
-40
0.0
0.2
0.6
0.8
1.0
Model Depth
Figure 4: Deep network models vary in curvature for each layer over model depth. Curvature is shown for variety of model. Filled circles indicate non-adversarially trained models, and open circles indicate adversarially-traien below zero, indicating output straightening. All non-adversarial models have output curvature above zero, indicating output representation less straight than the input pixels; All adversarial models shown have output curvature below zero, indicating output representation more straight than input.
robust models. All adversarial models show output curvature
During projected gradient descent, adversarial examples are created to train the model to correctly classify them. The adversarial example is created by adding noise an image. The amount of noise is bounded by a norm (usually l2 loo set to level c). We include some models trained on CIFAR-10 to understand how perturbation level interacts with straightness (CIFAR-10 models are faster to train, so more models are available).
Harrington & Deza 2022).
ResNet Attack vs. Curvature
CIFAR APGD-CE Attack Accuracy
Representation Layer Curvature Change
30
Representation Layer Curvature Change
L2 Attack
L2 APGD-CE Attack
20
Linf Attack
Linf APGD-CE Attack
Non-Adversarial
Non Adv Robust Model
10
0
-10
-20
-5
10-3
10-2
-0
100
10
20
Adversarial Attack Strength
AutoAttack Accuracy (%)
Figure 5: Left: White-box adversarial attacks reduce curvature in the output layer of ResNet50 models. Increased attack strength (e) decreases curvature, with l2 attacks most reducing curvature in the resulting model. Data on symlog x-scale to show show 0, and right of line on log seale. Right: For a variety of robustly trained models, Improved AutoAttack accuracy for APGD-CE is predicted by larger decreases in curvature from the pixel layer for adversarially robust CIFAR-trained ResNet50 models. We plot CIFAR ttrined models trained with l2 norm, 0, 0.25, 0.5, 1.0.
O: data to left of line on linear scale
We find that, as predicted, output curvature is highest for non-adversarially trained networks. loc attacks decrease output curvature, with larger values e leading to decreased curvature. Ia-attacked networks have the straightest output curvatures, however strength of attack does not greatly affect the output curvature. The l2 norm models may achieve straighter representations than the lae models because they allow for greater in training. Among la models, higher values may need to be tested to see a difference curvature among la norm trained models. Overall, we show that adversarial robustness to larger perturbations leads to straighter representations.
5

---

Published as a conference paper at ICLR 2023
While curvature corresponds to the strength of adversarial attack network has been trained with, increased adversarial attack strength does not often translate to improved test set accuracy (Tsipras) et al.. 2018) (See A.5).. In order to evaluate the predictive power of curvature for adversarial ac- curacy, we used the Attokt toolbox (Croce & Hein 2020) to perform APGD-CE attacks on the CIFARRtraieed subset of the Robust ResNet50 models (Fig Right). When comparing the Ac- curacy scores for these models with their curvature, we find that for both lp and (xo attacks, more curvature reduction the representation layer is correlated with higher attack accuracy.
We also explore how curvature relates to other types of robustness by evaluating leaders on the ImageNet-c benchmark (Hendrycks & Dietterich 2019) and anti-aliasing CNNs (Zhagg 2019). We find that none of these other robust models straightened natural videos like the adversarially robust models (See Fig 24 for corruption robust models: MAF, MAE-DAT, YO1-O-d-HAAT; See Fig 25 for a comparison of ResNet50 standard, shift-invariant robust, and adversarially robust; See Fig 27 for anti-aliasing ResNet50, VGG, AlexNet). Even though models trained for other types of robustness behave more like humans, the curvature measure reveals that those types of robustness are not sufficient to create the temporal representational stability observed in humans (Hénaff et al.) Adversarial robustness may stand-out producing straightness because it corresponds to more human- meaningful representations as shown with feature inversion (Engstrom et al. 2019b).
2019).
4.2
CURVATURE AND REPRESENTATIONAL STABILITY
Robust (1= 104.7)
Rohust (i/ 0. 11)
Reqular (u- 111.4)
Regular 0.15)
Count
Count
3000
20/-
300
0.
Curvature (Degrees)
Std. Dev. of True Label Prob.
Figure 6: Left: Curvature distribution for Adversarially Robust (Robust) and Non-Robust (Regular) ResNet when tested on VGGSound dataset. Robust model shows lower curvature over all videos. Right: Standard Deviation of true label prediction for same networks. Robust model shows overall lower variability in probability for the true label.
One potential benefit of lower curvature a stable representation over time. Such a stable represen- tation should show increased stability of class predictions over the frames of a video. To test this, we tested an adversarially robust (ResNet50.Robust.22.3 Robustt_2.33_N and non-robust (ResNet50.Standae model on VGGSound (Chen et al.) 2020). We measured curvature in the final layer for each video, and the standard deviation of the predicted probability of the ground truth labels for the video (Fig The adversarially robust ResNet50 produces overall lower curvature than the standard-traine model over the VGGSound videos. Furthermore, it has an overall lower standard deviation in pre- dicted probability for the true labels. This indicates that adversarially robust models, which have lower curvature, also show more stable predictions over time, despite the tested videos being out- of-distribution from the training set of images. Thus, curvature can used as a computationally cheap proxy for the temporal stability of network representations and predictions.
IN)
5 CURVATURE IN VISUAL TRANSFORMER MODELS FOR OBJECT RECOGNITION
Vision transformers (ViT) have very different architectures than CNNs. They replace convolutional elements with self-attention on visual tokens and achieve state of the art performance on a variety of visual tasks. In addition, these networks more adversarially robust than hundard-traines CNNs, which is attributed to their higeer-level and more generalizable features (Shao et al., 202T). Despite this, we find that standard-trained VIT (ViT Base, patch 16) does not reduce curvature on

---

Published as a conference paper at ICLR 2023
Visual Transformer Models Layer-Wise Curvature
Curvature Change
25
0
-25
WT_DINO_IN
VIT ViT__ Standard_ddl IN
CrossViT Sttnddrdaddddd_aa IN
0.2
0.8
1.0
Model Depth
Figure Visual Transformer models vary in curvature over model blocks. All ViT models (standard and DINO, base-patch-16) show output curvature higher than the pixel input. For DINO and standard VIT models, this increase is consistent throughout the layers. For Cross VIT (18 dagger 408) the representation forks in the middle stage. One path increases in curvature, while the other decreases below input, but after recombination output is curved above pixel input.
the natural video sequences tested (Fig Self-supervised training of ViT (base, patch 16) using the DINO method (Caron al., 2021) also results in a more curved representation than the input space. However, this DINO trained ViT has more straight representation than the baseline VIT. This suggests that self-supervised training may better than supervised for straight representations.
No transformer model except for CiossViT daggar (Chen et al. 2021) reduced curvature in any layer compared the pixel input (Fig(7). The structure of Cross models is unique in that they split into multiple paths, and the reduction in curvature we found interestingly only happens in one pathway of the model and does not persist after the two paths recombine. possible explanation for this is that the ViTssViT is able to straighten in the part of the model that splits because that pathway utilizes a multi-scale representation, a known aspect of human vision representations such as VI which we show in See to induce straightening.
CURVATURE AND ROBUSTNESS IN IMAGE SEGMENTATION AND FRAME PREDICTION MODELS
Segmentation Models Layer-Wise Curvature
Curvature Change
25
0
-25
PSP Standard CIT
M PSP ARobust sat CIT
- PSP ARobust ddc_
0.0
0.2
0.4
0.6
0.8
1.0
Model Depth
Figure 8: Segmentation models vary in curvature for each layer over model depth. Curvature shown for various segmentation model types. Filled circles indicate non-adversarially trained models, and open circles indicate addersarially-trained robust models.
Output straightness over time should favorable, stabilizing property for object detection models, where objects often stay continuously in-frame over many frames of a video. For image segmen- tation models on the other hand, predictions lie close to low-level pixel space and change on frame-by-frame basis. Therefore, for the task of predicting the dynamically changing segmentation map of an input movie, straightness of the output representation would not be desirable (assuming the input movie contains motion). To test this prediction, of 12 image segmentation models from (Xu et al. 202I), consisting of two different architectures, trained on datasets, with non-adversarial and two different adversarial training schemes.
measured output curvature for family

---

Published as a conference paper at ICLR 2023
Unlike object detection models, where adversarial training greatly reduces output curvature, adver- sarially robust segmentation models show only slightly reduced output curvature compared to their non-adversarially robust counterparts (FigB), and all increase curvature compared to the input. In- vestigating curvature change over model layers (Fig[)) reveals that this trend is consistent throughout all layers of the model. This result highlights the importance of task straightness, and supports the idea that curvature reduction is not simply the result of adversarial training; only when temporal stability task-beneficial does adversarial training promote straightness. We also evaluated the final layer for each of these models, finding similar results (Sec A.3)
Another DNN task for which output predictions are low level is next-frame prediction for video sequences. As in image segmentation models, good predictions are expected to vary dynamically frame-to-frame, and representational straightness at and near the output layer is not favorable quality in response to videos containing motion. Rather, for frame-prediction network, a favorable quality would be equal curvature at the input and output. This is because for werl-perforning model, the output prediction over many frames is the same as the input video; two identical videos have identical curvature. To evaluate this, we test the layer-evolution of curvature for three variants of PredNet (Lotter et al., 2016). network inspired by predictive coding, which is trained to predict the next frame in a video sequence (Sec A.7 28g bottom).
Indeed, for all pretrained variants of PredNet tested, the output layer maintains very similar curva- ture value to that of its input frame, with final output curvature closer than all other models to that of the input (Fig B). For all pretrained variants, the model curvatures increase in the representation of the first model block, then re-straighten the representation throughout the rest of the network be- fore returning the original pixel curvature. We also investigate a variant of PredNet fine-tuned to predict multiple frames in the future (PredNet_Extraal (Extra__KTTT] which we find only slightly reduces model curvature, with a similar trajectory over layers. Since PredNet is trained on videos, we can directly evaluate the impact of out-off-istribuue (Fig 28). PredNet models straighten more on in-distribution data, but the effect is small and does not change the overall trend seen in the videos from (Hénaff et al. (019)). We also test adding curvature constraint as a loss during training (see Section (A.7) and find that while the model can straighten, it does not greatly impact the final model output (Figure 29
compared to in an-distribution data on straightness
7 CURVATURE IN BIOLOGICALLY INSPIRED MODELS
We investigated straightness for a variety of both parametric (learedd) and non-parametric biologically-inse -inspired models. Given straightness thought to increase over progressively deeper layers of visual processing, we align these networks along the visual processing areas they are most closely matched to (Fig 9. Some models match to visual areas explicitly in their architec- ture like the non-parametric Henaffbio (Hénaff et al. 2021, which is a two-stage model based on center-surround filters follows by oriented Gabor filters, and PerpheralNet (previously referred to as Brucenet) (Brown et al. 2021). which is based on summary statisties of steerable pyramid. This explicit match to biological vision also holds for the early layers VOneNetCCrnets (Dapello et al.) 2020), which combines VI filter front-end and Cornets neural network that has layers matched to regions of the human brain (Kubilius et 2019)) For the adversarially trained Visual Transformer network Cross ViTRotAdy (Berrios & Deza (0022), these layers are those best matched by BrainScore for V4 and IT layers (Schrimpl et al. 2020).
backbone (Cornets being shallow
For all biologically-nsspeeee -nnsiieed models except for VOneNetCCrnetss, curvature progressively decreases through deeper network layers. For VOneNetCornets, curvature decreases up until the V1 layer, where a noise term capturing neural stochasticity added; curvature then strongly increases, far above the pixel-curvature baseline. To determine if this increase in curvature was due to the added noise, we tested the same model with the noise term set to zero. This reduced the downstream curvature after the VI layer, but did not climinate the curvature increase following the VI layer.
The increase in curvature for VOneNet at later layers suggest that making the front-end of a deep network more like biologically-inspree models does not suffice to straighten representations down- stream in a deep network. This is interesting because VOneNet is reported to be more adversarially robust to white-box attacks than standard CNN. This suggests that adversarial training, not the property of adversarial robustness itself, leads to straightened representations. Our finding supports neural population geometry results showing that adversarial trained models have different signa-
8

---

Published as a conference paper at ICLR 2023
Biological Models Layer-Wise Curvature
Curvature Change
V
VOneNet ConNets
VOneNettConNett NoiseOff IN
PeriphNet_155 1ao__150_BBB0000 BIO
SriphhNet_PSPyr_BO
Henaff_BR
-25
CrossViT CrossviT_ARobust ARobust Rotation_
Pixel Retina/LGN
V1
V2
V4 IT
Model Stage
Figure 9: Biologically-nnspreed network models result in straightened representations com- pared input pixels. Non- Non-parametric multi-stage filter models PeriphNet and Henaffbio progres- sively straighten. The VOneNetCornets model straightens the representation in its V1-block up until its noise layer, where curvature increases dramatically and does not recover in downstream convo- lutional blocks. Adversarially-train Cross ViT model shows progressive straightening.
tures than VOneNet and models trained with neural stochasticity (Dapello al., (021). However, it is puzzling that adversarial training, which is biologically implausible, would lead to straighter representations than biologically inspired mechanisms. More constraints may be needed achieve straight representations with biologically plausible methods.
DISCUSSION
We have shown that one putationally cheap metric for evaluating certain types of robustness, as well as representational sta- bility in both image and video models and across a variety of tasks. We have shown that training with strong white-box adversarial attacks reduces output curvature, and that models that reduce out- put curvature have better adversarial accuracy. However, incorporating simple shift-invariance and robustness to image corruption is not associated with straightness. Moreover, models that reduce the curvature of their input give more stable predictions over time. Furthermore, the relationship between curvature reduction and adversarial robustness depends on task. We find that adversarially robust object recognition models have straightened representations, but semantic segmentation and video frame prediction models do not. Moreover, in evaluating both CNN and transformer-basee based architectures, we have shown that the relationship between curvature reduction and adversarial ro- bustness depends on task and training procedure, not deep neural network architecture alone.
change in model representational curvature as simple and com-
In evaluating curvature for biologically inspired models, we show that biologically inspired mecha- nisms reduce curvature in model's representation, even more than adversarial training. However, simple addition non-parametric biologically inspired filters at the input of a convolutional model is insufficient to maintain output curvature. These results identify representational straight- ness as a common thread between biologically inspired and adversarially robust models, highlighting the benefits and limitations of these techniques in creating temporally-stabe representations.
With the growing need to understand the relationship between the robust properties of human vision and computer vision systems, straightness serves compare learned visual representations. With curvature reduction as a known property of the human perceptual representation, we find that our straightness results match intuitions such as adversarial robustness being important to achieving more human-like representations, and offers new insights such as the failure of self-supervised learning approaches creating temporally stable, human- aligned representations. Curvature can also be computed layer-wise, allowing us to more deeply understand how model architectures support or impede the favorable properties of biologically- inspired representations. While many biologically-inspree networks have useful properties such as robustness, curvature reveals a more nuanced relationship between such networks and this ro- bustness. Furthermore, curvature evaluates temporal stability while still being applicable to static image models. This allows for an expansion current research aligning computer vision with human vision that has historically focused static vision. We believe an essential next step in this research is to incorporate more temporal aspects of vision, and perceptual straightening is an important stepping-stone stone get there.
effective and efficient tool to evaluate and

---

Published as a conference paper at ICLR 2023
9 REPRODUCIBILITY
To reproduce our results, links to all the models analyzed can be found in the supplemental material in the Network Comparison Spreadsheet. Sources for the stimuli are in Sec [A.222 and in the code base linked for the HenafiBio model in Network Comparison Spreadsheet. Straightness can be determined by calculating curvature using Secs and A.1 The models, stimuli, and curvature function together should be sufficient to reproduce our results.
10

---

Published as a conference paper at ICLR 2023
REFERENCES
Research Computing Center at University of Chicago. Chicago motion datahase, 2022. URL
https:// rcc. rcc. uchicago.
Jiawang Bai, Li Yuan, Shu-Tao Xia, Shuicheng Yan, Zhifeng Li, and Wei Liu. Improving vision
transformers by revisiting high-frequency components. arXiv preprint arXiv:2204.00993 2022.
Alexander Berardino, Valero Laparra, Johannes Ballé, and Eero Simoncelli. Eigen-distortions of
hierarchical representations. Advances in neural information processing systems, 30, 2017.
Pietro Berkes and Laurenz Wiskott. Slow feature analysis yields a rich repertoire of complex cell properties. Journal of vision. 5(6):9-9, 2005.
William Berrios and Arturo Deza. Joint rotational invariance and adversarial training of dual-stream transformer yields state the art brain-score for area v4. arXiv preprint arXiv:2203.06649 2022.
Rachel Brown, Vasha DuTell, Bruce Walter, Ruth Rosenholtz, Peter Shirley, Morgan McGuire, and David Luebke. Efficient dataflow modeling of peripheral encoding in the human visual system. arXiv preprint arXiv:2107.11500 2021.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties self-supervised vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.
2021.
Chun-Fu Richard Chen, Quanfu Fan, and Rameswar Panda. Crossvit: Cross-attention multi-scale vision transformer for image classification. In Proceedings of the IEEE/CVF international con-
ference on computer vision, pp. 357-366, 2021.
Honglie Chen, Weidi Xie, Andrea Vedaldi, and Andrew Zisserman. Vggsound: large-scale audio- visual dataset. In ICASSP 2020-2020 IEEF International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 721-725. IEEF, 2020.
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3213-3223, 2016.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free free attacks. In International conference machine learning, pp. 2206- 2216. PMLR, 2020.
Joel Dapello, Tiago Marques, Martin Schrimpf, Franziska Geiger, David Cox, and James J DiCarlo. Simulating primary visual cortex at the front of cnns improves robustness to image perturbations. Advances in Neural Information Processing Systems, 33: 13773 2020.
Joel Dapello, Jenelle Feather, Hang Le, Tiago Marques, David Cox, Josh McDermott, James J DiCarlo, and Sue Yeon Chung. Neural population geometry reveals the role of stochasticity in robust perception. Advances Neural Information Processing Systems, 34:15595-15607, 2021.
Todor Davchev, Timos Korres, Stathi Fotiadis, Nick Antonopoulos, and Subramanian Ramamoor- thy. An empirical evaluation adversarial robustness under transfer learning. arXiv preprint (etp::905002255. 2019.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale scale hi- erarchical image database. In 2009 JEEE conference on computer vision and pattern recognition, pp. 248-255. leee, 2009.
Arturo Deza and Talia Konkle. Emergent properties of foveated perceptual systems. arXiv preprint (0Xi::2006.07991, 2020.
11

---

Published as a conference paper at ICLR 2023
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. /929, 2020.
Gamaleldin Elsayed, Shreya Shankar, Brian Cheung, Nicolas Papernot, Alexey Kurakin, Ian Good- fellow, and Jascha Sohl-Dickstein. Adversarial examples that fool both computer vision and time-limited humans. Advances in neural information processing systems, 31, 2018.
Logan Engstrom, Andrew Ilyas, Hadi Salman, Shibani Santurkar, and Dimitris Tsipras. Robustness (python library), 2019a. URL https::/tt/tttuu.bbb...b.. com wwt.c/omm//////. hatm//MdryLabb/ba robustness
Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Brandon Tran, and Alek- sander Madry. Learning perceptually-aligee representations via adversarial robustness. In ArXiv preprint arXiv:1906..004000 2019b.
M. Everingham, I.. Van Gool, C. K. Williams, J. Winn, and A. Zisserman. The pascal visual object classes (voc) challenge. International Journal of Computer Vision, 88(2):303-338, June 2010.
Jenelle Feather, Alex Durango, Ray Gonzalez, and Josh McDermott. Metamers of neural networks reveal divergence from human perceptual systems. Advances in Neural Information Processing Systems, 32, 2019.
Jenelle Feather, Guillaume Leclere, Aleksander Madry, and Josh H McDermott. Model metamers illuminate divergences between biological and artificial neural networks. bioRxiv, 2022.
Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3364,361, 2012.
doi: 10. (19/CVPP.2012.6248077
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. imagenet-trained cnns are biased towards texture; increasing shape bias im- proves accuracy and robustness. arXiv preprint arXiv:1811.1223/ 2018.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 2014.
Ross Goroshin, Michael F Mathieu, and Yann LeCun. Learning to linearize under uncertainty. Advances in neural information processing systems, 28, 2015.
Chong Guo, Michael J Lee, Guillaume Leclere, Joel Dapello, Yug Rao, Aleksander Madry, and James J DiCarlo. Adversarially trained neural representations may already as robust as corre- sponding biological neural representations. arXiv preprint arXiv:2206. 1/288, 2022.
Anne Harrington and Arturo Deza. Finding biological plausibility for adversarially robust features via metameric tasks. In International Conference Learning Representations, 2022. URL
net you.ee1deeeeeeeeeeeee N
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked au- toencoders scalable vision learners. In Proceedings the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 16000-16009, 2022.
Olivier J Hénaff, Robbe LT Goris, and Eero P Simoncelli. Perceptual straightening of natural videos. Nature neuroscience, 22(6):984-99) 2019.
Olivier J Hénaff, Yoon Bai, Julie A Charlton, Ian Nauhaus, Eero P Simoncelli, and Robbe LT Goris. Primary visual cortex straightens natural video trajectories. Nature communications, 12(1):1-12, 2021.
12

---

Published as a conference paper at ICLR 2023
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common ruptions and perturbations. arXiv preprint 19Xiv:1903.122660 2019.
Yanping Huang and Rajesh PN Rao. Predictive coding. Wiley Interdisciplinary Reviews: Cognitive Science, 2(5):580-593, 2011.
Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. Advances neural information processing systems, 32, 2019.
Alexander JE Kell and Josh McDermott. Deep neural network models of sensory systems: win- dows onto the role of task constraints. Current opinion in neurobiology. 55:121-132, 2019.
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. The cifar-10 dataset. online: http://ww.. cs. loronto. edu/kriz/cifar himl, 55(S), 2014.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. ImageNet classification with deep convo- lutional neural networks. Communications of the ACM, 60(6):84-90, 2017.
Jonas Kubilius, Martin Schrimpf, Kohitij Kar, Rishi Rajalingham, Ha Hong. Najib Majaj. Elias Issa, Pouya Bashivan, Jonathan Prescott-Roy, Kailyn Schmidt, et al. Brain-like object recognition with high-performing shallow recurrent anns. Advances in neural information processing systems, 32, 2019.
Aishan Liu, Xianglong Liu, Hang Yu, Chongzhi Zhang, Qiang Liu, and Dacheng Tao. Training robust deep neural networks via adversarial noise propagation. IEEE Transactions Image Processing, 00:5769-57811, 2021.
William Lotter, Gabriel Kreiman, and David Cox. Deep predictive coding networks for video pre- diction and unsupervised learning. arXiv preprint arXiv: 1605.08 00400 2016.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: 170.00086608 2017.
Xiaofeng Mao, Yuefeng Chen, Ranjie Duan, Yao Zhu, Gege Qi, Shaokai Ye, Xiaodan Li, Rong Zhang, and Hui Xue. Enhance the visual representation via discrete adversarial training. arXiv preprint arXiv:2209.0772 2022.
Bruno A Olshausen and David J Field. Sparse coding with an overcomplete basis set: A strategy employed by v1? Vision research, 37(23))3311-3325, 1997.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems 32, pp. 8024 8035. Curran Associates, Inc., 2019. URL http: httt:::/ppaprr.. www.aapeer...erris......s.ooocee cctpaper/ seeeativvv--llllleee htapyy/eehhhheeeeffommeeeeeeeeee wearninnlerrnngg library...
9015 pdf
Rishi Rajalingham, Kailyn Schmidt, and James DiCarlo. Comparison of object recognition behav- ior in human and monkey. Journal of Neuroscience, 35(35): 2015.
Shibani Santurkar, Dimitris Tsipras, Brandon Tran, Andrew Ilyas, Logan Engstrom, and Aleksander Madry. Computer vision with a single (robust) classifier. In ArXiv preprint arXXiv1906.000 2019.
Martin Schrimpf, Jonas Kubilius, Ha Hong. Najib J Majaj, Rishi Rajalingham, Elias B Issa, Ko- hitij Kar, Pouya Bashivan, Jonathan Prescott-Roy Roy, Franziska Geiger, et al. Brain-score: Which artificial neural network for object recognition is most brain-like? BioRxiv, pp. 407007, 2020.
13

---

Published as a conference paper at ICLR 2023
Kalpana Seshadrinathan, Rajiv Soundararajan, Alan C Bovik, and Lawrence K Cormack. A sub- jective study to evaluate video quality assessment algorithms. Human Vision and Electronic Imaging XV, volume 7527, pp. 128-137. SPIE, 2010a.
Kalpana Seshadrinathan, Rajiv Soundararajan, Alan Conrad Bovik, and Lawrence K Cormack. Study of subjective and objective quality assessment of video. IEEE transactions on Image Pro- cessing, 1944)1142-14411, 2010b.
Rulin Shao, Zhouxing Shi, Jinfeng Yi, Pun-Yu Chen, and Cho-Jui Hsieh. On the adversarial robust- ness of vision transformers. arXiv preprint arXiv:2103. 156700, 2021.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint 13 2013.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be odds with accuracy. arXiv preprint arXiv: (180.111122. 12152, 2018.
Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li. Learning structured sparsity in deep neural networks. Advances in neural information processing systems, 29, 2016.
Ross Wightman. Pytorch image models.
https:///gthaab..com//ww...oooo
2019.
Xiaogang Xu, Hengshuang Zhao, and Jiaya Jia. Dynamic divide-and-conquer adversarial training for robust semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7486-7495, 2021.
Daniel LK Yamins and James J DiCarlo. Using goal-driven deep learning models to understand sensory cortex. Nature neuroscience, 19(3):55-335,, 2016.
Richard Zhang. Making convolutional networks shift-invariant again. In International conference machine learning, pp. 7324-7334. PMLR, 2019.
14

---

Published as a conference paper at ICLR 2023
A APPENDIX
A.1
CURVATURE VS COSINE SIMILARITY
cosine similarity (vectors) == cos (0) = A.B
(3)
|| 3e I1+1
cosine similarity (video frames) := cos (0)
(4)
||ace |3
curvature == arccos (6g (t+1)
(5)
Ur Ur+1
cosine curvature 6r 0z+1
cos (ct)
10 |141
A.2 MODELS
A.2.1 MODEL SOURCES
All deep neural networks we analyzed were pretrained. Exact links code and weights can be found in the supplemental material in the Network Comparison Spreadsheet. The standard ImageNet- trained ResNet (He et al., (016) model was downloaded from PyTorch's model zoo (Paszke al.
2019). Adversarially robust ResNet models were all downloaded from (Engstrom et al. 2019a). The adversarially robust ResNets were trained using projected gradient descent. All VIT (Dosovitskiy et al. 2020) and standard trained CrossViT daggar (Chen et al. (021)) models were downloaded from the timm library (Wightman 2019). All CiT daggar adversarially robust and rotationally invariant models were downloaded from the repository of (Berrios & Deza, 2022). The adversarially robust Cross ViTs were trained with fast gradient sign method as stated in (Berrios Dezal [0022)
DINO models were downloaded from the DINO repository (Caron et al. 2021). while PredNet models were downloaded from (Lotter et al. 2016).
A.2.2 MODEL. ANALYSIS PROCEDURE
We showed each model the same 12 natural videos that were used the psychophysics experiments of (Hénaff et al. 2019). The videos were taken from the Chicago Motion Database (at University of Chicago 2022), the film 'Dogville'. Lions Gate Entertainment (2003), and LIVE Video Quality Database (Seshadrinathan et al. (0100]aa a). The videos were grayscale, consisting of 11 frames each of 512 x 512 pixels, capturing natural motion such as rippling ocean water or a person walking through a crowded street. We resized the video frames to be 224 x 224 for all deep networks and 256 x 256 for bio-models that use steerable pyramids. One limitation of this work is that we did not evaluate models on larger video datasets, but wanted to psychophysically validated stimuli for our analyses. For each model, we recorded its activations at intermediate and final layers for each video. We then found the global curvature for each stage of the model using equation 2 where we used the flattened model activations as the input &, to the curvature procedure.
We compared the global curvature at each layer of the model to the curvature of the video in pixel space. Models that straighten are defined as models that have a lower global curvature at deeper layers. When comparing the curvature of different model layers, we chose not reduce the di- mensionality of each layer activation to be the same across stages. Although principle components analysis (PCA) was sometimes used in (Hénaff al. 2019) when expressing curvature, they did not use it in their analysis of deep networks. Furthermore, while an architecture's 's inherent dimension- ality is likely relevant to a representation's 's curvature, we preferred not to introduce any additional transformations that would influence the measured curvature. We also found that performing PCA did not greatly affect trends like adversarial training leading to more straight representations See Fig- ure T0 for examples of curvature for different numbers of principle components two ResNet50 models.
A.2.3 COMPUTE
Our methods do not require large compute, All individual model analyses can run on CPU. We used a single GPU to speed up getting the features activations at each layer to the order of minutes per model.
15

---

Published as a conference paper at ICLR 2023
A.2.4 EFFECT OF PCA ON MODEL CURVATURE
Standard-Trained ResNet50
Adversarially- Trained ResNet50
gier
Hightr
Besteedo prices Cunetturs FCx2
Adcerarillyy Laperaita wwataentCCo
giar cursiture
Nighar canatare
-25-
candlar
Sightl
Fart
Stande BeskerSO sriike CLivesurs PC-2
Aiverur Rabest Fasteth Loparite Candeeeffet
canoiare
Honal
outor
Folt
Malio
Standard Tasleed Lapon
Adianenally Tota: Soald500 Curnclas (NO
Levatare
Hightr carviaue
Figure 10: Change in curvature for different numbers of principle components. Each subplot shows layer-wise change in curvature. Left is standard ResNet50 trained on ImageNet (Deng et al., 2009) classification. The natural line is the curvature measures on the natural videos from (Hénaff cet al. 2019) that used for all model analyses in the paper. The contrast and artificial sequences were additional video sequences psychophysically tested by (Hénalf et al. 2019) where humans tend to have little change in curvature for contrast but large increase in curvature for artificial. Right an adversarially robust (projected gradient descent with la norm and c = 3) ResNet50 trained on ImageNet. All rows below the top show curvature when taking the first 2, 10, and 100 principle components (PCs) of each representation layer activations. For PCs less than the number of frames in each video (11), we performed PCA on the pixel-level video in addition the model activations when computing the change in curvature. For all PCs, the standard ResNet50 has higher curvature for natural video sequences than the adversarially robust.
16

---

Published as a conference paper at ICLR 2023
A.3 FINAL STAGE CURVATURES
Curvature Change in Final Stage
Non-Robust Closs
Siv Rebust Class
Non-Robust Sgmt Robust Sgmtt
Videoo fred
Non-Paramn Rio
Output Curvature Change
Rebust Class
INC Robust Class
Hybrid
-10
-20
-30
vogii Standard Rest St aaaddddddnun rollleso Standerd dy Reshie S0 Stand Standtend Of Rest Restle 50 OWN iny Gross Sonnaartt rat of Patho CrossVi Rotel Rolatoon Fatho inl Alex Net Stan oo Resd Wel LSO, A ABrbash dint dy He restioo
(TT-0INO-IM
ARC inf of Read (.D. 4 Poouust sinr 2itl Resh MeeSS 550---0 aofooooog
Bees BeshierS ARcobust C.S.Cr
Resch Ree S-- Afeeuur ritho by Gross possiitAARcbuue ARcbuut Rotation
Ril 3 1.er
Crass "AAabuu sa-atte dy Peeiiet Moouut in vigi seon inficbuar di per pesnesio Snilthsuum Aloxhlet
(2-0 or
on
Candardd ih hotE
Het al-aaaaaun till pop Standentt
Di 3. Standard yoC
DL Standard OT
PSR Attottte pontustton sat satvOC NOC
poPt post-beebus coutttddee VDC AesAt
por soapoot-dee del sar
DL D1.-4Robin ARobust sap CTT
gat of
ARtibust duc VDC Di
Dt AR del-
PredNet 225traa prnntn AraaonckiT KITT
KITT
predheen L.D. 4.D. KOTTT
ToneNi Earope selest00 iy
VoneNNet aleetheri iN
Perpit RsPyr BIO
VIT ST Stamto
VOLO 1DD-D---A--
PSP Standard
strenght manghtss
Periphiie Hematt
ProdWet PredWet
DI
Ries
Figure 11: Final stage curvature for a variety of network architecture and training schemes. Model families show very similar curvature that of their output representation layers
The final stage of a network for image classification models is the last fully connected layer. We see similar trends in curvature across model families as in Fig We chose to report the the curvature on the layer just before classification because this is the layer that would be most commonly used as the representational backbone for other tasks.
A.4 MATCHING MODELS CURVATURE COMPARISON PLOTS
ResNet50 Models
10
5
-5
Output
-10
-15
-20
Aov-Adlvesaliill
Adversarial Shift Invariant
-25
ResNet50- BesNet5O-Standardd-t Standard-IN
ResNet50 ARobust-12-3--1
ResNer50-ARoobe ARobust- -in--4-IN
ResNet5o AesbutSo-ARobust-liin ARobust Lin--8-IN
Resnet50 Resnet50-Shiiffe ShiftRouust-N IN
Figure 12: ResNet50 models trained ImageNet
17

---

Published as a conference paper at ICLR 2023
ResNet50 CIFAR Models
30
Non Adversarial
Adversarial
Output Curvature
20
10
ResNee50-SSandard.ct
ResNet50- Soakket5--Aobbustt--------
Restet55-ARRbustt2--0555
ResNet50 ResNet50-ARobust-2-1
ResMet50-ARobust-iin- ---C-
Figure 13: ResNer50 models trained on CIFAR10
ResNet50 Restricted Models
Non-Adversarial
Adversarial
5
Output Curvature
-5
-10
-20
-25
ResMet5D-ARobust-il -ww.--Rotoott-----o--- 12-3-RIN
ResNet50 Saaklet5--Stnndar--aa Stannatdndddiit
Figure 14: ResNet50 models trained on Restricted ImageNet subset of ImageNet)
18

---

Published as a conference paper at ICLR 2023
VIT Models
16
Aoveesverllrial
Adversarial
12
Output Curvature
10
VTTSSandard-IN
VIT-DINO-IN
Figure 15: ViT models trained on ImageNet
CrossVIT Models
Non-Adversariill
15
Adversarial
10
Output Curvature
5
CrossViT-ARobust-NN
CrossviT Rotation-IN CrossVTT-Rtaaiion-NN
CrossViT-Standarde
Cross CrossViT-ARobie Wrr.TTTRRRtsttttttiioo///
Figure 16: ViT models trained on ImageNet
19

---

Published as a conference paper at ICLR 2023
AlexNet Models
16
Aov-rsverllrial
Adversarial
14
Shift Invariant
12
Output Curvature
10
MlexNet-Standarch
Alex AlexNet-ARRbbste -at.-st-obusttt
Alexteet-ShifRRobusttt
Figure 17: AlexNet models trained on ImageNet
VGG Models
Non-Adversarial
Shift Invariant
10
Output Curvature
4-
2-
Vgg11 Vggll-Standard-It St-nnaadnddd-
Vgg11 1 Shifttobbus-ti IN
Figure 18: VGG models trained on ImageNet
20

---

Published as a conference paper at ICLR 2023
Segment DL3 DL3_CIT Models
Non-Adversarial
14
Adversarial
12
Output Curvature
10
4-
2-
DL3 OL3-Standard-CTT
DL3 OL3-ARobust-sat-Ci
D13-ARobust-ddc-CT
Figure 19: DL3 segmentation Models (Cordts et al. 2016
Segment DL3_ DL3_VOC VOC Models
Non-Adversarial Adversarial
Output Curvature
6-
DL3 Standaddddd
DL3 -Robust-tttt
DL3 L.3.ARobust-ddc-VOO
Figure 20: DL3 segmentation Models trained on VOC (Everingham et al. 2010
21

---

Published as a conference paper at ICLR 2023
Segment PSP_CIT Models
Non-Adversarial Adversarial
14
Output Curvature
2-
#SP-Standard-CTT
PSP-ARobust-sat
rsP-ARobust-ddc.c
Figure 21: PSP segmentation models trained on CIT (Cordts et al. 2016
Segment PSP_VOC Models
Non-Adversarial Adversarial
Output Curvature
6-
2
PSP St--taadddd-VOc
PSP S.Pp-Roobst-stt-VOO
PSP Pet--RRobst-dd--ae
Figure 22: PSP segmentation models trained on VOC (Everingham et al. 2010
22

---

Published as a conference paper at ICLR 2023
A.5
ADVERSARIAL ACCURACY AND CURVATURE
ResNet Attack vs. Accuracy
ResNet Accuracy vs. Curvature
100
Representation Layer Curvature Change
30
Inet (Full)
90
20
(Res)
CIFAR
Test Accuracy (%)
10
80
Inet (Full)
Inet (Res)
70
CIFAR
-10
60
-20
50
0
10
10
10
50
60
70
80
90
100
E Adversarial Attack Strength
Test Accuracy (%)
While adversarial attacks with greater strength impart many desirable robustness properties on network, adversarial training does not improve test accuracy, often decreasing test accuracy on the within-domain test set for given model. (Data plotted on symlog scale.) Right: While stronger adversarial attacks decrease curvature, improved test accuracy for a model is not predictive of output curvature reduction. Rather, within given model training/test set, increased test accuracy predicts smaller curvature reduction in the output layer.
Figure 23: Left:
23

---

Published as a conference paper at ICLR 2023
A.6
CURVATURE IN AON-ADVERSAIIALLY ROBUST NETWORKS
ImageNet- C Models Layer-Wiss Curvature
40
Curvature Change
20
-20
+
mee
mee det
-
het
0.0
0.2
0.4
0.6
0.8
1.0
Model Depth
Figure 24: Curvature plots for three of the top five performing models for ImageNet-C (corrupted) dataset benchmark. MAE is masked autoencoder transformer network (He al. 2022), mae dat is the mae model with additional discreet adversarial training (Mao et al. (022) (note this is a mod- ified version of traditional adversarial training for visual models), and hat visual transformer optimized to process high-frequency image components Bai et al. (2022). Although these mod- els are robust to image corruptions, they do not reduce curvature. The mae_dat and hat models also represent the top two performing models on Stylized ImageNet, indicating that robustness to shape/texture transforms like those in Stylized ImageNet (Geirhos et al. 2018) is also not indicative of curvature reduction.
Shift Invariant Models Layer-Wise Curvature
40
Carvature Change
20
S reseeti
uggnet
+
aiemmat
0.0
0.2
0.4
0.0
0.8
1.0
Model Depth
Figure 25: Curvature plots for three model architectures modified to be shift-invariant using the anti-aliasing method from (Zhang 2019). These models robust to location shifting, but do not show significant curvature reduction from input, as seen in adversarially-traine robust models
24

---

Published as a conference paper at ICLR 2023
Shift Invariant Models Layer-Wise Curvature
40
Curcature Change
20
Adverserially
Model Depth
Figure 26: Curvature plots for ResNet50 variants trained: without robust training (Standard ResNet50), with shift-invariance robustness in (Zang 2019) (Shift Invariant ResNet50), and with adversarial training (Adversarially Robust ResNet50) invariant robustness only slightly reduces curvature compared to the standard model, and does not reduce output curvature from the input data, while adversarial robustness strongly reduces curvature reducing it below that of the input data.
Shift Invarlant Models Layer-Wise Curvature
40
Change
20
0-
-20
Standard Aleckit
Invanarr Aleateet
Adversriialy Robust Alechier
0.0
0.2
0.4
D.6
D.8
1.0
Model Depth
Figure 27: Curvature plots for AlexNet variants trained: without robust training (Standard AlexNet), with shift-invariance robustness in (Zhang 2019) (Shift Invariant AlexNet), and with adversarial training (Adversarially Robust AlexNet (Liu et al. 202I)). The small size of AlexNet limits its ability to benefit from adversarial attacks (Liu al. 2021). This may explain why, unlike other adversarially trained models tested, curvature is not reduced significantly from the input. Shift- invariance this model however, does appear to reduce output curvature as compared to the standard model.
25

---

Published as a conference paper at ICLR 2023
A.7 PREDNET STRAIGHTNESS FOR IN AND OUT OF DISTRIBUTION DATA
We measured straightness in PredNet, next-frame video prediction model. In Figure 28 urvature is shown in filled circles for the following standard PredNet models: standard frame future prediction in open circles are two PredNet models trained with straightness terms in the loss function (Pred- Net LO R2 straight KITTI and PredNet LO R3 straight KITTI). All models were trained on the KITTI dataset (Geiger et al., 2012).
5- and all-tayer-loss (PredNet Baas_alllKK]III Shown
Frame Prediction (PredNet) Models Layer-Wise Curvature (KITTI)
40
20
0
Curvature Change
-20
60
PredNet_LP KITTI
PredNet KITTI
-80
PredNet Batrtt__xtt__p_p_e KITTI
PredNet LO R2 [0_straight KITTI
-100
PredNet LO R3 straight KITTI
-120
Pixel
R3
R2
R1
output
Model Layer
Frame Prediction (PredNet) Models Layer-Wise Curvature (Henaff Stimuli)
40
20
Curvature Change
-40
-60
PredNet_LO KITTI LO__TTT_O_
PredNet KITTI
-80
PredNet Extret__xxtpppptee KITTI
-100
PredNet LO, R2 12__strightt_nn KITTI
PredNet R3 stsssigttt_tini KITTI
-120
Pixel
R3
R2
R1
RO
output
Model Layer
Figure 28: PredNet change in curvature across model layers measured in-distribution (top: KITTI dataset) and out-of-distribution (bottom: (Hénaff et al., 2019) stimuli). Generally, all PredNet models have lower curvature values for KITTI, the in-distribution dataset, than the (Hénaff et al. 2019) stimuli, but the curvature follows similar trends across dataset. All PredNet models are trained on KITTI. The blue, orange, and green lines are the baseline PredNet models developed in (Lotter et al. 2016). The baseline models have a lower change in curvature than the (Hénaff et al. 2019) stimuli. However, the change in curvature is generally close to 0 for both datasets indicating little straightening. The red and purple lines show PredNet models that were trained to straighten layers 2 and respectively on the KITTI dataset. Those models show a larger drop in curvature on the KITTI dataset than (Hénaff et al. 2019) stimuli at layers 2 and 3, but the models overall straighten only two layers for both datasets.
the KITTI dataset
26

---

Published as a conference paper at ICLR 2023
The two models in Figure 28 that straighten (red and purple lines) trained with curvature constraint. Our goal is to tested the utility of straightness as a training signal by adding a curvature constraint the first (R3) and second (R2) layers of PredNet during training to create "straight" PredNet models. We evaluate how the constraint affects the model's internal representation and the final frame prediction. For both models, curvature is greatly reduced in R2 and R3 compared the input curvature (Fig 28 bottom). Because of PredNet's 's top-down and bottom-up feedback structure, the constraint layer R3 does affect any other layers of the model. Although we find that is possible to train PredNet to straighten and still get next-frame predictions that has lower mean square error (MSE) than copying the previous frame, we are not able to achieve equivalent or better prediction MSE with the straight models compared to the original on KITTI (Geiger et al., 2012) (original LO PredNet: 0.00687 MSE, straight PredNet: 0.00976 MSE, compared to previous frame: 0.02122 MSE). See Fig 29 for example predictions. While these straightened PredNet variants do not achieve state of the art, we argue that these models may display other advantages such robustness and similarity to human representations
27

---

Published as a conference paper at ICLR 2023
A.8
PREDNET PREDICTIONS
Next Frame
Predicted Actual
PredNet LO
Extrapolate Frames
MSE: 0.0209
PredNet Straight R2
MSE: 0.0235
Figure 29: Example Predictions for Original and Straighten PredNet. Both models trained on the KITTI dataset (Geiger et al. (0022). The straightened PredNet was trained with penalty on its R2 layer if there was high curvature.
28

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
We thank Il Fiete and Nick Watters for helpful discussions and feedback. A.H. is supported by funding from Toyota Research Institute. V.D. is supported by the MIT CSAIL METEOR Fellow- ship. This work was funded in part by NSF/BMBF I6S-1607486 Ruth Rosenholtz and Christoph Zetzsche and by USAFRL and the United States Air Force Artificial Intelligence Accelerator under Cooperative Agreement Number FA8750- 19- 2-1000 to W. Freeman, The authors acknowledge the MIT SuperCloud and Lincoln Laboratory Supercomputing Center for providing (HPC, database, consultation) resources that have contributed to the research results reported within this paper.
NEGATIVE SOCIETAL IMPACTS
We believe there few negative societal impacts of this paper. Our work exploratory and did not introduce any new models. However, we note that the development of machine vision systems, which can operate at above the level of performance of humans at certain tasks, may lead to negative societal impacts such as the loss of jobs and industries as human workers are replaced.
29


Reviews:
[{"review_id": "uG_a3WpQklp", "reviewer": "Reviewer_zqwQ", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper investigates the perceptual straightness of a large set of modern vision models. Authors start from the observation that representation straightness is a known property of biological vision. They then define a metric to capture the average output curvature on a video and evaluate a wide range of models (resnet/vit trained for classification, segmentation models and biologically inspired model).\n\nThe main conclusions of the study are:\n1)\tNon-adversely trained models have the highest output curvature\n2)\tAdversarial training can reduce the model output curvature, when trained with classification tasks but not for all the tasks, i.e. it is not case for segmentation.\n3)\tBiologically inspired mechanism can also reduce the model output curvatures.\n", "summary_of_the_review": "The paper proposes an interesting empirical study. The significance of the paper could be improved by better demonstrating the advantage of representation with a higher level of straightness and showing that the conclusions are robust to the choice of the video dataset used in evaluation.\n\n\n=== After reading rebuttal.\n\nThank you for your responses. The rebuttal addressed most of my original concerns, I updated my score accordingly.\n\n", "strength_and_weaknesses": "Strength:\n-\tThe proposed study appears novel to me and is well motivated.\n-\t Authors perform an extensive evaluation on impressive number of models.\n\nWeaknesses:\n-\tWhile the study is motivated by finding in biological system, it’s unclear why perceptual straightness is a desirable property from a machine learning perspective. It would be nice to identify a set of tasks where improved straightness has a benefit, beyond adversarial robustness.\n-\tWhat is the impact of the dataset distribution on which we compute the output curvature? The evaluation protocol only uses 12 videos to compute this metric. Would the finding remain stable if we were to use different videos?\n-\tVideos are most likely out-of-distribution with respect to the training dataset. Would the finding be similar if we were to reduce the distribution shift between training and evaluation?\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "h-rItKtD31z", "reviewer": "Reviewer_eEJj", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors present an analysis of deep learning models using the change in curvature across frames of videos from pixel space to the representation space. The authors find that training deep networks on classification with adversarial images leads to straighter (like human visual representations, Henaff et al 2019) representations. They also analyze biologically plausible networks to show that there is some (in the early layers) increase in straightness but not consistent across the complete hierarchy. I find the presented analyses to be interesting, but I don't think the authors have clearly conveyed how their contributions impact either machine learning or vision science. ", "summary_of_the_review": "I think the paper presents an interesting set of results linking representational straightness to adversarial training. However, without any evidence on a) how straightness better predicts human visual behavior/neural data or b) how straightness leads to more adversarially or OOD robust deep learning models, I am presented with a partial picture of the significance of the contributions. At this stage, I find the paper to be borderline and would be comfortable recommending acceptance if the authors make the suggested changes mentioned in my review above. Mainly, addressing the following points (quoting from my review) and how the results turn out to be will influence a change in my score:\n\n\"Is representational straightness predictive of improved robustness to adversarial attacks and OOD shift? While it is obvious that adversarial training improves robustness to adversarial attacks, I would like to see the relationship between adversarial robustness and representational straightness. Similarly one could also correlate straightness with model generalization to OOD shift on benchmarks such as ImageNet-c. These evaluations would add more information about how significant the straightness changes are and how they relate to the representation quality.\"", "strength_and_weaknesses": "Strengths:\n1) Clarity of writing and figures. The paper is written with great clarity. Figures such as Fig 1 and Fig 2 are elegant and clear to convey what the authors are studying in this work. \n2) Diverse evaluation. The authors have evaluated the representational straightness in a wide range of diverse deep neural network architectures from the robust and non-robust classes for image recognition and segmentation. They also evaluate multiple biologically plausible models that are of great interest to the visual neuroscience and vision science communities. \n3) Fig 5, $\\epsilon$ vs curvature change. I found this result to be quite interesting that increasing adversarial attack strength also increases straightness. It would be nice to see how much each of these attack strengths contributes to adversarial robustness as well.\n\nWeaknesses:\n1) Significance of the contribution to representation learning. The authors have mentioned in several places how straighter representations bear more advantages than test accuracy, such as robustness, alignment to human perception, and representational stability. However, I don't find any experiment/analysis that quantifies these advantages or even demonstrates them. Is representational straightness predictive of improved robustness to adversarial attacks and OOD shift? While it is obvious that adversarial training improves robustness to adversarial attacks, I would like to see the relationship between adversarial robustness and representational straightness. Similarly one could also correlate straightness with model generalization to OOD shift on benchmarks such as ImageNet-c. These evaluations would add more information about how significant the straightness changes are and how they relate to the representation quality.\n\n2) Non-matching models in robust and non-robust class. Since one of the main takeaways of the paper is that adversarial training improves straightness, I expect to see clearly how the same architecture (and same rnd seed) -- when fixing other factors of variability such as dataset, training set size, training hyperparameters -- varies in their representation straightness. There are some matching models in the robust and non-robust classes in Fig 3. If the above conditions of varying adversarial training in an isolated manner was what was performed for this analysis, I would suggest the authors please make this clear in writing. \n\n3) Hard to understand model specifications from legends. I believe in some figures (e.g Fig 4, 5, 6) it is difficult to know the combination of architecture + training dataset. I would suggest the authors to use a common template to define models e.g. Architecture_NumberOfLayers_Dataset_Task and it would make reading the plots much easier. \n\n4) Discussion section. Currently the discussion section presents a summary of the results in the paper (how curvature changes as a function of adversarial training, optimization task etc.) but does not discuss in detail why these observations are relevant to designing future deep learning models or to human visual representations. This section could be greatly improved to highlight why, in the authors' perspective, the results are of importance to representation learning and human vision.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "6gn8aWn1K2M", "reviewer": "Reviewer_EBgB", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose to leverage a recent finding about human visual perception of movies to evaluate a variety of neural network architectures trained on different tasks. The visual feature is that humans tend to perceive movies with relatively less curvature from frame to frame compared to the actual curvature measured on pixels. We say that visual perception is straightening movies. The proposed work has three contributions :\n(i) adversarial training often leads to straightening in CNN and Vision Transformers,\n(ii) straightening depends on the training task (eg no straightening for segmentation),\n(iii) biologically-inspired piece of architecture always leads to straightening when going deeper and deeper. ", "summary_of_the_review": "For now, I slightly tend to reject the paper. Yet, I will reconsider after reading authors responses and others discussions among reviewers.", "strength_and_weaknesses": "Strength :\n- straightening in neural network hasn't been previously evaluated,\n- the work provides an overview of straightening in many architectures trained on many tasks,\n- the work establishes a link between straightening and robustness,\n- single focus paper with a clear message.\n\nWeaknesses : \n- the claims about robustness are weak because only adversarial training is used while there are other ways to enforce robustness,\n- the authors failed to explain why \"curvature is a useful way of evaluating neural networks representations\". ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster