Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper presents an interesting and important finding regarding representational straightening of natural movies in artificial neural networks (ANN). The authors showed that representational straightening, which was previously shown in human perception of natural movies and neuronal representations in primary visual cortex, doesn’t necessarily require predictive training of ANNs with movies, and can emerge in ANNs trained with robust supervised learning algorithms (image classification plus adversarial training or random smoothing). 

strength_and_weaknesses: $\textbf{Strengths}$: 

The results of this paper shed light on an important, previously observed phenomenon: namely, natural movie sequences are straightened both in human perception and primate V1 which is beneficial for linear prediction. These empirical observations implied that perception and neuronal representations underlying it are optimized for prediction of natural visual steams. Therefore, both predictive training and natural movies are both necessary for learning straightened representations in ANNs. This paper, interestingly shows that neither of these two factors are necessary. Rather, natural, static images and robust training are sufficient for learning straightened representations in ANNs. In addition to measuring the curvature of latent representations in a trained ANNs, authors use different metrics and experiments to explore this phenomenon from different viewpoints, which make the findings even more convincing. On top of that, the paper presents a computational model for primate V1 that finally shows a significant improvement compared to the previous models. 

$\textbf{Weaknesses}$:

Although the empirical results shown in the paper are quite convincing, the paper doesn’t provide an intuitive understanding of why robust training leads to representational straightening of movies. The ANNs don’t see any natural movies during training, and there is no direct (or indirect) intuitive relationship between noise robustness in pixel space and linearizing sequence of frames in latent space. In the absence of such an explanation in the paper, my own efforts for finding an explanation directed me toward a possible relationship between robust training of natural images and predictive training of natural movies. I elaborate below in my comments.


clarity,_quality,_novelty_and_reproducibility: The majority of the paper is clear. The paper could benefit from more clarity regarding the relationship between robust training and representational straightening (see the summary of the review). The findings of the paper are novel.  
summary_of_the_review: $\textbf{Questions}$:

1- The volume of the hypersphere in the latent space seems to have a relationship with the representational curvature of movies, based on the results presented in the paper (figure 6). Can the authors show a more thourough comparison of curvature and expansion metrics across different models and different layers of the models? It’s possible that the measured curvatures are, in general, smaller in more contracted latent hyperspheres which would also explain the results shown in figure 6.
This would also suggest an explanation for the observed relationship between robust training and straightening: robust training pushes similar images (eg consecutive frames of a movie) close together in the latent space. This could be considered as an implicit predictive training for movies, which is not dissimilar to self-supervised learning algorithms such as SimCLR and contrastive predictive coding. I'm interested to hear what the authors think about this possibility. 

2- The outperformance of the RS model in accounting for V1 responses is very interesting. Given the significant difference between RS and AT in explaining V1 responses, how could we know if the outperformance of RS is caused by robustness, and not specifics of the RS algorithm? 

3- The inverted linear interpolation experiments are very successful in giving a more direct evidence for the straightened representations learned via robust training. However, the SSIM metric used to quantify the similarity of natural movies and synthetic sequences mainly relies on lower-order statistics of the sequences. One could imagine that perceptual straightening relies on more abstract representations, in addition to lower-order statistics. In an ideal situation with enough time for more experiments, human observers would be the best evalutors for sequence similarities. In the absence of that possibility, could the authors provide more generated samples in the appendix. Especially, samples where the first and the last frames are more visually different. Also, which layers of the ANNs were used for the linear interpolations? How would the results look like for the early, intermediate, and late layers (both the appearance of the sequences and the SSIM results)? The most straightened representations seem to be in the intermediate layers. Would we also get the highest SSIM with the intermediate layer interpolations?


$\textbf{Minor Comments}$:

1- I suggest that the authors check the citation formats throughout the manuscript. There are a number of cases where the citation format is not correct, or the reference’s authors names is attached to the word before with no space in between. For example, see the last two lines of page 3. 

2- figure 2 right: why not all AT models are shown in the curvature plots?

3- figure 5 caption: “Positive correlation means the smaller the correlation,…” correlation doesn’t seem to be the right word here.

4- page 7 last paragraph: “Between different RS models tested on different input noise levels, RS L2 stands out…” which RS L2?  Based on table 1, all RS models are L2.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper tackles the question of straightening of natural movies in deep neural networks. It starts from previous observations both in visual psychophysics and neurophysiology that the curvature of the internal representations (abstract perception, V1 neural activity) is lower than the one of the initial movie representations (evolution of pixels in space-time). Such a reduction in curvature is not observed in the feature space (ouputs at different layers) of deep neural network (resnets) with standard training on large image datasets.

The authors shows that robust training (adversarial and random smoothing) is sufficient to obtain a curvature reduction in the feature space. According to the authors, this observation stems from a notion of "invertibility" which is used to describe a neural network in which linear interpolation in the feature space lead to perceptually smooth interpolation in the image space. They show that interpolating between movie frames generate perceptually plausible movies (using SSIM). Finally, the authors also shows that such robustly trained (random smoothing) neural networks are better at explaining the variance of neurophysiological recordings in the primary visual cortex.

strength_and_weaknesses: Strength:
- leverage behavioral and biological vision studies to question deep neural networks,
- relate robustness to input noise to perceptual straightening of movies,
- surprising observation : networks trained robustly on static images reproduces an observed dynamical feature of perception  
- compare two types of robustness (random smoothing, adversarial training). 

Weaknesses:
- incremental work
- results are not sufficiently discussed with the visual perception literature 
clarity,_quality,_novelty_and_reproducibility: Clarity: 
Overall the writing is clear.  

Be careful with the claim of "perceptual plausibility" as it is only supported by SSIM and not by any perceptual data.

In section 3.3 the notion of expansion is introduced. As a reader it is not clear what this notion is intuitively, nor why it is supposed to be an interesting metric to control for later in the result section. It is somehow stated in section 4.2 but this would be great to have it written differently when the notion of expansion is introduced. How does this notion of expansion interact with curvature ?

A lot of citation should be in parenthesis (or use reference numbers ?). There is sometimes a missing space between a citation and the previous word. This is also the case for some references to figures. Equations in 3.3 : cos is a MathOperator in latex + large parentheses must be used.

Figure in the appendix are not referred to in the main text so they are somehow useless for the reader. An additional figure showing more example of interpolation between frames would be great in the appendix.
Text size in the figure should be almost as large as the main text.


Quality: 
All sections are sufficiently good except for the discussion. There are several related questions like : (i) How perceptual movie straightening could affect image segmentation ? (ii) Is perceptual straightening still valid for stationary dynamical textures or are these textures already straight enough ? How perceptual straightening relates to stationarity ? (iii) What are the limitations of the perceptual straightening idea (flash-lag effect ?) ?


Novelty: 
incremental but this is a very interesting step !

Reproducibility: 
The authors do not state if their code will be released online
summary_of_the_review: This a good empirical contribution for ICLR filling a gap between vision studies and artificial neural network. The connection between a static feature of NN (robustness) and a dynamic feature of perception is appealing.

I will recommend acceptance more strongly once the authors have accounted for my remarks (ie improving the discussion section and referring to supplementary figures + fixing typos).
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: [response to author's rebuttal: see https://openreview.net/forum?id=mCmerkTCG2S&noteId=mpYhm0LBTQD]

The paper "Brain-like representational straightening of natural movies in robust feedforward neural networks" investigates to which degree standard ImageNet-trained models and "robust" networks (in terms of adversarial training, training with random smoothing) straighten representational trajectories of short movie sequences, which is an effect that is known from neuroscience. They find that robust networks indeed straighten representational trajectories. A few other experiments investigate links to fitting brain data and inverting representations.
strength_and_weaknesses: Strengths:
+ establishes a connection between adversarial training/random smoothing and perceptual straightening, which is novel and very interesting
+ consistent color scheme, (mostly) clear figures, especially Figure 2 conveys the main point of the paper

Weaknesses:
- Brain-score comparison may be impacted by different baseline accuracies between adversarially trained and RS models (had to tell, since I couldn't find the RS model's baseline accuracy in the paper)
- some claims / wording not backed up by data ("brain-like representational straightening")
- multiple issues regarding clarity, see suggestions below
- poor reproducibility

clarity,_quality,_novelty_and_reproducibility: ## **Clarity:**
Overall, the paper uses a consistent color scheme which is very helpful in understanding and relating the different figures. Especially Figure 2 is really cool and very clear.

**Major:** Suggesting to tone down language regarding "brain-like" representations. None of the plots show the ceiling for e.g. predicting neural data, so a relative comparison between computational models (e.g. model A better fits neural data than model B) must not be mistaken with stating that model A is brain-like / has a brain-like representation. I would suggest to give the paper a careful read and whenever the term "brain-like" is used, to either contextualize - e.g. stating that model A is *more* brain-like than model B - or to actually back the statement up by showing that the model is on par with neural systems. For instance, the title states "brain-like representational straightening", but this claim is never investigated in the paper and would involve comparing to the estimated neural curvature of neural systems when viewing exactly the same stimuli.

**Other points:**
- references throughout the paper are cited in the wrong citation style. If the sentence does not refer to the authors specifically as in Authors (2022), then the citation should be in brackets (Authors, 2022). Depending on the package, this can be achieved via \cite{} vs. \citep{} etc. Currently sentences with citations are often hard to read.
- overall sloppiness & typos, e.g. "BarlowtwinsZbontar", "front-endDapello", often related to the above point; also Figure 4 caption "is more brain-like representations"
- contributions point (1.) is descriptive without answering what the contribution is, the use of past tense ("gave rise") is suggestive of this being known from prior work whereas what the authors likely intend to convey is that this is their first contribution. Suggesting to re-phrase it as e.g. "We show that ..."
- Table 1 formatting is different from template (double columns on left/right)
- Figure 3 left: it's hard to see anything in the movie frames, I would suggest to use an example where one can see more. Currently I have a hard time understanding what's happening even in the natural movie row.
- Figure 3: cutting the y axis (instead of starting at zero) is misleading and contradicts good practice in data visualization. I would strongly suggest to start the y axis from zero so that the bar height is proportional to the effect.
- "EXPLAINING BRAIN-LIKE REPRESENTATIONS" (section heading): that's a very broad goal, can the authors be a bit more specific here in terms of which definition of "explaining" they are using? Is "explanation" really the best term, or would "fitting neural data" be more suitable perhaps? Different people have very different expectations when thinking of what an explanation would consistute.
- minor: page 8 "confirmed in a recent study Guo et al. (2022)" -> perhaps "as investigated in a recent study by Guo et al. (2022)?" I don't think the study answered the question to the degree that one would like to think of it as completely confirmed.
- why do the curves in Figure 2 not start from zero curvature difference for the input space (before layer 1, i.e. at x=0)?
- term "neural behaviour": can you clarify? I thought data is usually either neural data or behavioural data
- Given that the action recognition network and VOneResNet were introduced in the methods section, why are they not shown in the main figures of the paper?


## **Quality:**
**Major:** Figure 4 claims that RS fits brain data better than adversarial training: do the baseline ImageNet accuracies of the RS and the adversarially trained models match? If not, can you provide an accuracy-matched comparison? Given that the ImageNet accuracy of the chosen adversarially trained models is very low, it is not surprising that they're also bad at fitting brain data, so this would be a caveat in the comparision.

**Minor:** Movie interpolation example (Figure 3 left): I would suggest to show more examples in the appendix, ideally e.g. 10 examples that are randomly selected (rather than chosen by the authors).


## **Novelty:**
To the best of my knowledge, establishing a connection between adversarial training and perceptual straightening (Figure 2) is a novel and important contribution.

The investigation of the invertability is less novel, as the authors mention this is known from robust models and the RS model performs worse than the adversarially trained models in this regard.

In terms of the neural fits (e.g. Figure 4), it would help to explain where data is simply taken from brain-score.net and where it is computed/evaluated by the authors.

## **Reproducibility**:
Very poor - no code submitted, not enough details. For instance: How is the RS model trained? What is its ImageNet accuracy? Which hyperparameters, data augmentation, or even which deep learning library were used during training? I would strongly suggest to improve the reproducibility by providing more details in the paper itself, alongside submitting code if at all possible, and if this is not possible to explain why.
summary_of_the_review: I am leaning towards acceptance since investigating the link between adversarial training / noise robustness and representational straightening is a clear, novel and important finding. At the same time, the paper currently has a number of issues that should be improved (see suggestions above). I would be happy to raise my score if my concerns are sufficiently addressed.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper investigates the degree to which adversarial training "straightens" visual representations (a concept borrowed from biological vision). A connection is drawn between adversarial training/random smoothing and perceptual straightening, which the reviewers found to be quite interesting. At the same time, it was also noted that the novelty was somewhat limited because the work builds heavily on concepts already established in the literature. It was also noted that the paper fails to provide any kind of theoretical justification for the found connection between adversarial training/random smoothing and perceptual straightening. 



justification_for_why_not_higher_score: Although the established connection between adversarial training/random smoothing and perceptual straightening appears to be novel and interesting this remains the main contribution of this paper which is purely experimental as it lacks any kinds of theoretical justification for the observed results. 
justification_for_why_not_lower_score: There is unanimous support for this paper to be accepted.
Decision: Accept: poster