Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors propose to leverage a recent finding about human visual perception of movies to evaluate a variety of neural network architectures trained on different tasks. The visual feature is that humans tend to perceive movies with relatively less curvature from frame to frame compared to the actual curvature measured on pixels. We say that visual perception is straightening movies. The proposed work has three contributions :
(i) adversarial training often leads to straightening in CNN and Vision Transformers,
(ii) straightening depends on the training task (eg no straightening for segmentation),
(iii) biologically-inspired piece of architecture always leads to straightening when going deeper and deeper. 
strength_and_weaknesses: Strength :
- straightening in neural network hasn't been previously evaluated,
- the work provides an overview of straightening in many architectures trained on many tasks,
- the work establishes a link between straightening and robustness,
- single focus paper with a clear message.

Weaknesses : 
- the claims about robustness are weak because only adversarial training is used while there are other ways to enforce robustness,
- the authors failed to explain why "curvature is a useful way of evaluating neural networks representations". 
clarity,_quality,_novelty_and_reproducibility: Clarity:
The paper is clearly written.
Some citations are not in parentheses while they should be.
The text fontsize in the figures should be almost as large as the main text fontsize.
The figures in the supplementary are not referred to in the main text. As such they are useless. Other supplementary sections should also be referred to in the main text.


Quality:
My opinion is that it is great to have neural network architectures that are aligned with visual perception and neurosciences. Yet, the interest of the paper might be limited to an already acquired audience if the authors do not provide more explanations about why straightening is useful or interesting to have in NNs. 
The performed work is of good quality. Yet, as it is a single focus paper I would expect to find a broader evaluation of robustness and not only adversarial training.

Novelty:
Evaluation of straightening in neural network representation is novel but this is the single focus of this paper.


Reproducibility:
Seems reproducible if the code is made available online. 
summary_of_the_review: For now, I slightly tend to reject the paper. Yet, I will reconsider after reading authors responses and others discussions among reviewers.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper investigates the perceptual straightness of a large set of modern vision models. Authors start from the observation that representation straightness is a known property of biological vision. They then define a metric to capture the average output curvature on a video and evaluate a wide range of models (resnet/vit trained for classification, segmentation models and biologically inspired model).

The main conclusions of the study are:
1)	Non-adversely trained models have the highest output curvature
2)	Adversarial training can reduce the model output curvature, when trained with classification tasks but not for all the tasks, i.e. it is not case for segmentation.
3)	Biologically inspired mechanism can also reduce the model output curvatures.

strength_and_weaknesses: Strength:
-	The proposed study appears novel to me and is well motivated.
-	 Authors perform an extensive evaluation on impressive number of models.

Weaknesses:
-	While the study is motivated by finding in biological system, it’s unclear why perceptual straightness is a desirable property from a machine learning perspective. It would be nice to identify a set of tasks where improved straightness has a benefit, beyond adversarial robustness.
-	What is the impact of the dataset distribution on which we compute the output curvature? The evaluation protocol only uses 12 videos to compute this metric. Would the finding remain stable if we were to use different videos?
-	Videos are most likely out-of-distribution with respect to the training dataset. Would the finding be similar if we were to reduce the distribution shift between training and evaluation?


clarity,_quality,_novelty_and_reproducibility: The paper is easy to follow, and the empirical evaluation appears and novel to me.
summary_of_the_review: The paper proposes an interesting empirical study. The significance of the paper could be improved by better demonstrating the advantage of representation with a higher level of straightness and showing that the conclusions are robust to the choice of the video dataset used in evaluation.


=== After reading rebuttal.

Thank you for your responses. The rebuttal addressed most of my original concerns, I updated my score accordingly.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors present an analysis of deep learning models using the change in curvature across frames of videos from pixel space to the representation space. The authors find that training deep networks on classification with adversarial images leads to straighter (like human visual representations, Henaff et al 2019) representations. They also analyze biologically plausible networks to show that there is some (in the early layers) increase in straightness but not consistent across the complete hierarchy. I find the presented analyses to be interesting, but I don't think the authors have clearly conveyed how their contributions impact either machine learning or vision science. 
strength_and_weaknesses: Strengths:
1) Clarity of writing and figures. The paper is written with great clarity. Figures such as Fig 1 and Fig 2 are elegant and clear to convey what the authors are studying in this work. 
2) Diverse evaluation. The authors have evaluated the representational straightness in a wide range of diverse deep neural network architectures from the robust and non-robust classes for image recognition and segmentation. They also evaluate multiple biologically plausible models that are of great interest to the visual neuroscience and vision science communities. 
3) Fig 5, $\epsilon$ vs curvature change. I found this result to be quite interesting that increasing adversarial attack strength also increases straightness. It would be nice to see how much each of these attack strengths contributes to adversarial robustness as well.

Weaknesses:
1) Significance of the contribution to representation learning. The authors have mentioned in several places how straighter representations bear more advantages than test accuracy, such as robustness, alignment to human perception, and representational stability. However, I don't find any experiment/analysis that quantifies these advantages or even demonstrates them. Is representational straightness predictive of improved robustness to adversarial attacks and OOD shift? While it is obvious that adversarial training improves robustness to adversarial attacks, I would like to see the relationship between adversarial robustness and representational straightness. Similarly one could also correlate straightness with model generalization to OOD shift on benchmarks such as ImageNet-c. These evaluations would add more information about how significant the straightness changes are and how they relate to the representation quality.

2) Non-matching models in robust and non-robust class. Since one of the main takeaways of the paper is that adversarial training improves straightness, I expect to see clearly how the same architecture (and same rnd seed) -- when fixing other factors of variability such as dataset, training set size, training hyperparameters -- varies in their representation straightness. There are some matching models in the robust and non-robust classes in Fig 3. If the above conditions of varying adversarial training in an isolated manner was what was performed for this analysis, I would suggest the authors please make this clear in writing. 

3) Hard to understand model specifications from legends. I believe in some figures (e.g Fig 4, 5, 6) it is difficult to know the combination of architecture + training dataset. I would suggest the authors to use a common template to define models e.g. Architecture_NumberOfLayers_Dataset_Task and it would make reading the plots much easier. 

4) Discussion section. Currently the discussion section presents a summary of the results in the paper (how curvature changes as a function of adversarial training, optimization task etc.) but does not discuss in detail why these observations are relevant to designing future deep learning models or to human visual representations. This section could be greatly improved to highlight why, in the authors' perspective, the results are of importance to representation learning and human vision.
clarity,_quality,_novelty_and_reproducibility: - The presented work is written with great clarity. Figures that explain the authors methods such as Figs 1, 2 were very useful to understand the paper's main theme. Some of the result figure legends however lacked clarity as highlighted in my reviews. 
- This work is a novel extension of prior work analysing representational straightness in the context of adversarial training. I don't find any big reproducibility issues per se, however, the code and weights for segmentation models isn't available in the Network Comparison Spreadsheet in the supplemental data.
summary_of_the_review: I think the paper presents an interesting set of results linking representational straightness to adversarial training. However, without any evidence on a) how straightness better predicts human visual behavior/neural data or b) how straightness leads to more adversarially or OOD robust deep learning models, I am presented with a partial picture of the significance of the contributions. At this stage, I find the paper to be borderline and would be comfortable recommending acceptance if the authors make the suggested changes mentioned in my review above. Mainly, addressing the following points (quoting from my review) and how the results turn out to be will influence a change in my score:

"Is representational straightness predictive of improved robustness to adversarial attacks and OOD shift? While it is obvious that adversarial training improves robustness to adversarial attacks, I would like to see the relationship between adversarial robustness and representational straightness. Similarly one could also correlate straightness with model generalization to OOD shift on benchmarks such as ImageNet-c. These evaluations would add more information about how significant the straightness changes are and how they relate to the representation quality."
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper addresses the problem of "straightness" in perception models. Straightness in the context of videos refers to a smooth path between frames in the representation space. The paper has four main findings: (1) Adversarial training results in more straightened representations in vision architectures. (2) Increase in straightness by adversarial training is task dependent (e.g., not true for segmentation). (3) Straightening makes models more stable against OOD data. (4) Certain classes of biologically inspired models increase straightness in the lower layers of the network.

Strengths:
- The study of straightness is novel and well-motivated.
- The paper includes an extensive set of experiments considering a wide variety of models and tasks.
- The paper is well-written and conveys the message clearly.

Weaknesses:
- Evaluation benchmark only includes 12 videos, so it is not clear how generalizable these findings are. 


justification_for_why_not_higher_score: As mentioned above, better benchmarks and more experiments on those benchmarks are needed to strengthen the impact of the paper. 
justification_for_why_not_lower_score: The paper proposed a valuable study, so the AC and the reviewers believe the community will benefit from such study.
Decision: Accept: poster