Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper presents a study of trained neural networks in the frequency domain. Two methodological contributions are proposed:
 1) an extension to multiclass classification of the technique in Rahaman et al for probing the ability of a model to learn signals of varying frequency 
 2) a method for estimating the Fourier components of trained networks along linear interpolation paths between examples
The empirical contributions of this paper are:
 3) the spectral bias also applies to CIFAR10/Imagenet ResNets (whereas in Rahaman et al. only simpler models were studied)
 4) trained networks are on average lower frequency on within-class paths than on between-class paths
     1) higher disparity between within-class and between-class examples correlates with better generalization performance
     2) a correlation between more diverse classes (measured by examples with low mean C-scores) and high within-class frequency and vice versa.
 5) a relationship between image frequency and function frequency where are low image frequency components are learned faster than high image frequency components
strengths_and_weaknesses: **Originality**: This paper is in the line of other papers studying the so-called *spectral bias*: the tendency of neural networks to learn low-frequency functions first. All points listed in the summary are  original as far as I can tell.

**Quality**: The paper lacks some details in experiments and notation, and I think (but I might be proven wrong in the rebuttal) that there are some technical flaws in 1/, 2/ and 4/. 

**Clarity**: Due to (potential) technical flaws, and missing details regarding experiments, I found it difficult to really judge on the significance of the experiments and the validity of the claimed results.

**Significance**: I feel that the methodological contributions are only minor improvements upon existing techniques (Rahaman et al for 1/ and Zhang et al. for 2/). Empirical observation 3/ is an extension of the findings of Rahaman et al. which can also be considered a minor improvement. I have the feeling that points 4/, 4.1/ and 4.2/ have a technical flaw/are trivial. I did not fully grasp empirical result 5/ so I am unable to judge of its significance.
questions: **Technical flow in 1/** (l129): "retain a valid probability distribution". Assume that S(X) = -1 (as can be the case when using the sine function mentioned l146), then for the true class, \bar{y} = 2-1/M is greater than 1, which does not define a valid probability distribution. How are you then able to train a model using these labels ? Are you using the cross-entropy loss or any other loss function?

**Technical flaw in 2/ thus 4/, 4.1/ and 4/2** (figure 3 right) If I understand the experiment correctly then within class I want to learn a function y(lambda=0) = y(lambda=1) = onehot(true class) and between class a function y(lambda=0) = onehot(class of example 0) \neq y(lambda=1) = onehot(class of example 1). Then I am not surprised that within class the Fourier coefficients are equally spread (in fact, they probably converge to 0 as you increase the number of examples), whereas between class you basically learn a function that grows continuously from 0 to 1. As you average over many examples, I would not be surprised if this resembled a sigmoid function. Can you maybe directly plot the (averaged) function ? If I am correct this then renders your findings somewhat trivial, but I might be proven wrong.

What exactly are you plotting in all barplots with low/high frequency components? What threshold are you using for splitting between low and high frequencies?

In figure 2 left: why is ||y_lambda - y_0|| > 0 when lambda = 0? when lambda = 1 (images come from the same class)?

**Other major comments**
 - l183: Can you make more precise what "model output" means? From the context I think it is the probability, but it could also be the logit output.
 - in figure 5, can you motivate the use of frequency=0.038? How does the experiment vary when using other values?

**Minor comments**
 - l170-175: I find the choice of a linear path and l2 (not precised in the text?) distance between images questionable. As a failure case, imagine 2 images sharing the same background B1 (e.g. grass) but with 2 different foreground objects C1 and C2, then the l2 distance between this images will likely be smaller than two images with the same foreground object class C1 but 2 different backgrounds B1 and B2.
 - figure 3 right: can you make precise whether this plot uses training set examples or test set examples?

limitations: I don't think that such a paper should discuss negative societal impact.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This work seeks to evaluate how training design decisions affects the spectral bias of neural networks. Specifically, they first extend the label noise procedure of Rahaman et. al to the multi-class setting and analyze when the deep learning model fits the noise. They also propose a second method which uses linear interpolation between two images, and use this to distinguish between the spectral content for "within classes" or "between classes". The authors consider results on CIFAR10 and on ImageNet, and consider factors such as accuracy, model size, training set size, and data augmentation. 
strengths_and_weaknesses: ## Strengths
*Quality*: I thought that the experiments were quite thorough, and was particularly impressed that they covered so many design decisions (such as data augmentation, training set size, etc). 

*Clarity*: I thought the paper was relatively easy to follow. 

*Originality/Significance*: This paper seems like an important step for making analysis of spectral biases practical. The paper is thus relevant to the community.

## Weaknesses

*Significance:* Please include some intuition on what these spectral biases translate to for real applications. What kind of spectral bias is desirable (and why)? Should we prioritize design decisions that are higher frequency between classes and lower frequency within classes (or does this just happen to be true for well-generalizing models)?  A final impact section, discussing what the concrete implications of this work are, would be helpful. 

*Clarity*: The section on the relationship between the interpolation and label smoothing classes could be more clearly written. 

questions: - Are there better ways of interpolating between two images (for example, optimal transport methods) rather than simply in pixel space. I could imagine that interpolating in a naive way could cause images to be relatively out of distribution.

- What do the spectral biases look like for non-convolutional networks (such as ViTs). Do they look similar to CNNs?

- In Section 5.2, you found a difference in spectral bias for diverse vs homogeneous classes? Do you see any differences between "hard" (i.e., low confidence) or easy examples. 
limitations: The authors discuss limitations of the label smoothing method in Section 3.1. It would be useful to move up the discussion of the interpolation method from appendix A.2. The authors do not discuss any negative societal impacts of their work.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper aims to improve how the spectral bias of neural networks is measured. To this end, the authors introduce two techniques: first based on label smoothing, and the second based on probing the network outputs along a linear path between two data samples. The end result is a number of tricks that make spectral bias a more useful diagnostic tool for analyzing neural nets (pre-trained or otherwise), as empirically demonstrated in the experiments. 
strengths_and_weaknesses: ### Positive
+ The use of label smoothing (instead of adding sinusoids to binary targets as done in Rahaman et al) is sound. The purpose of adding sinusoids to the target in Rahaman et al. was to expose the model to spatially correlated noise, which this method gracefully extends to the multiclass setting.
+ The idea of using the vector V to locally probe spectral bias is neat. I imagine that setting V to the principle component vectors of the data can also be insightful.
+ The idea of using projections of the function also makes sense, and I appreciate the sanity check in Figure 2 right.
+ The experiments are well crafted do a good job at teasing out interesting aspects of spectral bias.

-------
### Not Positive
+ The presentation can be improved. For instance, it would make for a more pleasant reading if the authors were to clearly (mathematically or pictorially) express what they mean by “noise fitting” and “min noise fitting” (even when the caption of Figure 1 explains what they are). The y-axis of Figure 2 left can also be confusing. In general, I feel the paper would read better if the experiments are bundled with their description (instead of being split in sections 3 and 4), but this is a matter of taste.
questions: - Regarding the experiment described in Figure 5: do the authors have an intuition as to what would happen if the analysis were to be performed on earlier layers of the network?
- Can the technique described in Section 3.2 be related to the Projection-slice Theorem?
- How exactly is “high-frequency fraction” defined? Is it the sum of the magnitudes of the DFT for f larger than some threshold? If so, what is this threshold?
limitations: The authors have been upfront about the limitations.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper investigates the generalization of neural networks by investigating the spectra of the learned functions. While existing literature focuses on simple architectures like 2-layer MLPs, this work studies spectral bias in modern image classification networks. Two methodologies/metrics are developed for analyzing the spectral bias: i) label noise to enable measuring spectral bias in multi-class classification via label smoothing and ii) measuring the smoothness of a learned function via linear interpolation between test examples.The key finding is that high-accuracy models learn a function that is high frequency in regions between different image classes but low-frequency within each class. This finding is demonstrated to be valid under different settings, e.g., for different model architectures and training strategies. 
strengths_and_weaknesses: Strengths:

- The paper is overall well-written and easy to follow. 
- The proposed metrics/tools for analyzing spectral bias in more complex classification architectures are well-motivated and ablated. While the proposed metrics appear not completely novel (see weaknesses), applying these metrics to quantify spectral bias in neural networks is a valuable contribution and might be useful tools for future works in this research area.
- Generally, I found the paper insightful and interesting to read.


Weaknesses:
- The paper aims to explain the generalization ability of neural networks by investigating their spectral bias. However, there are no experiments in the paper that actually demonstrate that the proposed metrics are indeed linked to better generalization of the network (i.e. there is no metric to report generalization ability in the experiments).\
- The proposed label smoothing appears very similar to the label-smoothing regularization proposed in [37] and it is not clear enough how it differs.
- The proposed linear interpolation approach appears related to the Perceptual Path Length (PPL) regularization proposed in [1]. Please add this to the related work and discuss how PPL differs from the proposed linear interpolation.
- The terminology of ‘noise’ and ‘smoothing’ is confusing. E.g, L.136 the *noisy* validation loss corresponding to the *smoothed* labels., L.179 “label smoothing noise”. L.181, increasing the ‘label smoothing frequency’ corresponds to adding higher frequencies, i.e. noise.
- The key insight, that the learned function is high-frequency between classes and lower frequency within classes, is not very surprising considering that it is measured by linearly interpolating between samples of different, randomly chosen classes. For randomly chosen classes, a path very likely traverses multiple class boundaries which would explain higher frequencies. I suggest rather formulating this more conservatively, i.e. as a ‘proof of concept’ by confirming an intuitional result, as opposed to ‘a key insight’. Nonetheless, the proposed tools allow to quantify and support this intuition.

Missing related work:
- [1] for PPL, mentioned above.
- Model sensitivity to image frequency: [2] finds that discriminative models are not sensitive to the frequency range but rather to the magnitude of the frequencies. 
Please also consider adding a brief discussion on how their finding aligns with your findings on image frequency. Sensitivity to magnitude might also provide another explanation for the higher sensitivity to natural image directions (4.3). 

[1] T. Karrs, S. Laine, M. Aittala, J. Hellsten, J. Lethinen, T. Aila. Analyzing and Improving the Image Quality of StyleGAN. CVPR 2020

[2] K. Schwarz, Y. Liao, A. Geiger. On the Frequency Bias of Generative Models. Neurips 2021


Misc:
- In many references the journal/conference is missing.
- Consider renaming S from ‘target function’ to ‘noise function’ to i) be consistent with Rahaman et al and ii) make the name more intuitive.

questions: - It seems that noise fitting happens when the validation loss starts to plateau (Figure 1). A simple baseline would hence be to simply determine the noise fitting point via the slope of the validation loss. It would be interesting to see if this is an equally good measure to justify the complexity of perturbing labels and training models from scratch.
- Why does sampling along paths between images rather than in regions surrounding each image allow for a more global measurement of spectral bias (L.93)? Could the methodology by Zhang et al. also be used as a metric in this setting? If not, please explain why?
- L.171 why is the distance between adjacent images kept constant as opposed to the number of samples?
- How do the results from section 4.2 relate to the results for MLPs of Rahaman et al (On the spectral bias of neural networks)?
- The findings of Basri et al., that the learned function will be higher frequency in regions with denser samples (L.80), seem to contradict the finding of this work that the learned function of high-accuracy models is low-frequency within each class. Could you clarify how these findings can be aligned?
- Could you give an intuition for the formula for label smoothing in L.129?

limitations: The limitations were addressed adequately.
review_title: null

Meta-review: metareview: The authors have largely convinced the reviewers (and definitely myself) of the merits of the paper after extensive and detailed rebuttal and discussion. I am happy to recommend acceptance.
confidence: Certain
Decision: Accept