On the universality of neural encodings in CNNs

Authors: florentin  guth
brice  ménard
Venue: ICLR_cc_2024_Workshop_Re-Align
Published: 2024-02-08 18:18:51

Abstract: We explore the universality of neural encodings in convolutional neural networks trained on image classification tasks. We develop a procedure to directly compare the learned weights rather than their representations. It is based on a factorization of spatial and channel dimensions and measures the similarity of aligned weight covariances. We show that, for a range of layers of VGG-type networks, the learned eigenvectors appear to be universal across different natural image datasets. Our results suggest the existence of a universal neural encoding for natural images. They explain, at a more fundamental level, the success of transfer learning. Our work shows that, instead of aiming at maximizing the performance of neural networks, one can alternatively attempt to maximize the universality of the learned encoding, in order to build a principled foundation model.

Main paper: 
 introduction: 
 Deep neural networks reliably achieve high performance on visual tasks such as image classification, with remarkable robustness to the exact details of the architecture, initialization, and training procedure. Furthermore, transfer learning results show that, in some cases, networks trained on one task can perform well on other tasks by simply retraining the last few layers. In the context of computer vision tasks, this raises the question: to what extent do networks share a universal encoding of images, irrespective of their architecture and training dataset? What does this encoding look like? Can one identify generic learned features across datasets and tasks?Deep networks can be studied through two main points of view: one can compare their representations (how an input image is expressed through neural activations) or their encodings (how a training dataset is encoded in network weights). The dominant approach to studying the properties of neural networks has been through representations. It has been shown that representations learned by networks trained from different initializations appear to be similar over many layers (Raghu et al., 2017; Kornblith et al., 2019) . Similar observations have been made in the context of human neural representations by studying fMRI response patterns in visual cortex (Haxby et al., 2011) , as well as between neural network representations and IT spiking responses (Yamins et al., 2014) . Here, we ask whether this similarity at the level of network representations arises from a more fundamental similarity between their learned weights, i.e., in the functions implemented by different networks rather than their outputs. In addition, we explore which aspects of the encoding are similar.Neural encodings (or network weights) are less easily prone to analysis than hidden representations and have thus been less studied. The weights of convolutional neural networks (CNNs) learn to exploit simultaneous correlations across channels and between neighboring pixels to solve numerous tasks. The first layer of CNNs can be directly visualized and the learned spatial filters are known to be Gabor-like filters (Krizhevsky et al., 2012) for a wide range of settings. Recent work by Cazenavette et al. (2022) pointed out that, in CNNs, most of the learning deals with channel mixing: the spatial filters can be kept frozen from their random initializations, and the remaining learning of channel mixing leads to similar performance. Exploring the structure of all weights, including channel-mixing dimensions, is more challenging. While the input basis of the first layer is fixed (for images, it is aligned with the pixel coordinate system), the other layers are expressed in random bases set by the neurons of the previous layers. This random shuffling has been a challenge to explore the properties of encodings and, to a great extent, is at the origin of the black box qualification of neural networks.Recently, Guth et al. (2023) have shown that this random shuffling can be canceled by aligning the hidden representations to a common reference.In this paper, we explore the statistical properties of CNN weight tensors following the approach of Guth et al. (2023) . We first find that the learned eigenvectors of spatial filters are surprisingly simple. They are low-dimensional and, to first order, they do not depend on the filter size, dataset or task over a wide range of settings. In other words, a universal set of spatial filter eigenvectors emerges. It is therefore possible to factorize CNN weight tensors to separate the information processing between space and channels. We do so using a frozen set of such spatial filters and only learn weights along channels. It then becomes possible to compare learned channel eigenvectors between networks and we show that, in the context of natural images, the learned channel-mixing features also show signatures of universality: they are similar across datasets well beyond the first few layers.Our main contribution is to characterize the joint processing of CNNs along spatial and channel dimensions and show that they typically use a preferred encoding strategy:• We develop a procedure to compare the weights of networks rather than their representations.This can be applied to measure similarities between neural encodings of different datasets and thus, indirectly, measure similarities between datasets.• We show the emergence of a seemingly universal encoding of natural images, through both spatial filters and to a lesser extent channel weights. We find encoding similarity across datasets and tasks over an appreciable range of layers. It explains at a more fundamental level the success of transfer learning, self-supervised learning, and foundation models. 
spatial filters in cnns: 
 At each layer, a convolutional neural network simultaneously mixes information across the spatial and channel dimensions of a representation by learning a weight tensor W containing C out × C in × k × k trainable parameters. In general, the respective importance of spatial and channel mixing is unclear as the two are entangled in a single linear operation. How does the training process use this multidimensional capacity? How much and what kind of information emerges in the spatial and channel domains?The first layer of CNNs, which can be directly visualized, is known to learn Gabor-like filters (Krizhevsky et al., 2012) in a wide range of settings. However, the properties of learned spatial filters at deeper layers have not yet been exposed clearly. To gain some insight, we first train a VGG-11 (Simonyan & Zisserman, 2015) (which has L = 8 convolutional layers) on the ImageNet dataset (Russakovsky et al., 2015) . We first set the filter size k to 7 to ease interpretation. We slightly simplify the architecture by removing biases and using a linear classifier, see Appendix B for details. A visual inspection of the spatial filters of each layer reveals a set of rather smooth and redundant spatial filters, suggesting a low-dimensional statistical description. A subset is shown in Appendix A.To perform a statistical comparison, we compute eigenvectors of spatial filters at all layers for different filter sizes. More precisely, we diagonalize the k 2 × k 2 "spatial" covariance obtained by averaging over the C out × C in channel dimensions as in Trockman et al. (2023) . The resulting leading eigenvectors are shown in panel (a) of Figure 1 . Interestingly, for layers l ≥ 2, we observe very consistent sets of eigenvectors, dominated by a low-dimensional subset, which was not apparent on visual inspection of the individual filters. This means that the main difference between the covariances of the layers (as seen in Appendix A) originates from the eigenvalues rather than the eigenvectors. We also note a slight dilation of the filters affecting all eigenvectors with depth. On occasions, subsequent ranks appear in flipped order. This happens when consecutive eigenvalues are similar. Panel (a) also shows the eigenvectors of filters learned with smaller filter sizes k = 3 and 5. Although the resulting filters are in different spaces R 3×3 , R 5×5 , R 7×7 , their visual correspondence is clear. It suggests that these eigenvectors are determined by a mechanism that does not depend on layer depth nor filter size. Pushing the exploration further, we show in panel (c) similar measurements for different datasets (CIFAR10, CIFAR100, and ImageNet) on standard classification tasks and resized to the same 32 × 32 resolution. In all cases, the eigenvectors show again the same canonical set. It is slightly noisier than the version presented above due to the lower resolution of the images. We also note that the effective spatial support of the filters is slightly smaller for lower-resolution images, as can be expected. Finally, we explore the effect of the training task: we train our network on ImageNet (at 224 × 224 resolution) on random labels (Zhang et al., 2021) , which can be considered as a self-supervised training task leading to a compressed representation of the data. The corresponding eigenvectors, shown in panel (b), reveal that even with random labels, the same set of eigenvectors emerges again (Maennel et al., 2020) . In summary, this canonical set is observed for different filter sizes, datasets and tasks: it shows signatures of universality.Several researchers have proposed that certain aspects of the weights of CNNs may be determined by symmetry groups of the data (Cohen & Welling, 2016; Kondor & Trivedi, 2018; Marchetti et al., 2023) . Both Trockman et al. ( 2023) with artificial networks and Pandey et al. (2022) in the context of receptive fields of biological sensory neurons have shown that the covariance of learned spatial filters can be mathematically modeled. In summary, the existence of a canonical set of spatial filters which can be defined mathematically implies that most of the learning ends up taking place along channel dimensions.Motivated by the existence of a canonical set of spatial filters, we introduce a simplified architecture: we factorize the standard 2D convolution operation into a depthwise and pointwise (1×1) convolution so that each layer operates only across spatial or channel dimensions, similar to separable architectures (Sifre & Mallat, 2013; Chollet, 2017) . The depthwise convolution convolves each channel with a fixed set of frozen (non-learned) filters, which we set to be the universal spatial eigenvectors observed in the previous section. The non-linearity is applied after the depthwise convolution so that there is no linear combination of these spatial filters across channels. It is followed by the pointwise convolution, which is simply a 1 × 1 convolution, i.e., a linear operation along channels applied at each spatial location. Interestingly, this simplified architecture can recover most of the network performance, similarly to Cazenavette et al. (2022); Trockman et al. (2023) . See Appendix B for more details on this architecture. 
exploring learned channel weights: 
 Our goal is now to investigate what is being learned in the channel domain, using the factorized architecture introduced in the previous section. We explain in Section 3.1 how this can be reduced to comparing covariances of channel weights through an alignment procedure. We then define in Section 3.2 measures of dimensionality and similarity of these learned covariances. Finally, we use this approach to characterize neural encodings of different datasets and evaluate their universality in Section 3.3.How does one meaningfully compare (channel) weights of two trained deep networks? Here, we briefly review the approach introduced by Guth et al. ( 2023) and present the main ideas. The analysis is more challenging than for the spatial filters: the latter are expressed in fixed spatially-aligned axes, while channel weights are expressed in random bases set by the (randomly initialized) neurons of the previous layer. This requires an alignment procedure between hidden representations before comparing the weights. These concepts are illustrated in Figure 2 .Aligning hidden layers and weights. We begin by comparing two hidden representations ϕ(x) and ϕ ′ (x) learned by two different networks. In general, ϕ(x) and ϕ ′ (x) are not directly comparable (they might even have different dimensionalities). Representational similarity analysis (Kriegeskorte et al., 2008) instead compares their similarity structures (or kernels) ⟨ϕ(x), ϕ(y)⟩ and ⟨ϕ ′ (x), ϕ ′ (y)⟩, which have empirically been found to be close in various settings (Raghu et al., 2017; Kornblith et al., 2019) . This implies that the variability in the representation between ϕ and ϕ ′ must preserve this similarity structure, and is thus limited to an orthogonal transform. In other words, when ⟨ϕ(x), ϕ(y)⟩ ≈ ⟨ϕ ′ (x), ϕ ′ (y)⟩, there exists an orthogonal alignment matrix A such that ϕ ′ (x) ≈ Aϕ(x) (Guth et al., 2023) . It is defined by minimizing the mean-squared errorEQUATIONover orthogonal matrices A. This is also known as the Procrustes problem (Hurley & Cattell, 1962) and can be solved in closed form (Schönemann, 1966) . It corresponds to a so-called "shape metric" (Williams et al., 2021) on the kernels defined by the representations (Harvey et al., 2023) .Now consider two neurons w and w ′ in the next layer of the two different networks. What does it mean for w and w ′ to be equivalent? It seems natural to ask that the two neurons compute similar outputs: ⟨w, ϕ(x)⟩ ≈ ⟨w ′ , ϕ ′ (x)⟩. Because ϕ(x) ̸ = ϕ ′ (x), this condition is not equivalent to w ≈ w ′ . Rather, using the fact that ϕ ′ (x) ≈ Aϕ(x), we haveEQUATIONso that the two neurons compute similar outputs when w ≈ A T w ′ , or equivalently, when w ′ ≈ Aw.Just like the alignment A maps representations in the first network to representations in the second network, it maps next-layer neurons in the first network to equivalent neurons in the second network.Comparing hidden neurons from different networks thus requires aligning their hidden representations and taking this alignment into account in the comparison.Comparing weight distributions through covariances. Comparing individual neurons in two different networks amounts to searching for a one-to-one mapping between them. If the two networks had the same neurons, but possibly in a different order, then their representations would differ by a permutation (Entezari et al., 2022; Benzing et al., 2022; Ainsworth et al., 2022) . The use of rotations when aligning representations suggests that more variability might be present. Rather than comparing individual neurons from two different networks, we search for similarities between the neural populations at a global level, i.e., whether they have the same statistics. This corresponds to testing whether the neurons in both networks can be modeled as samples from the same distribution, as done in so-called "mean-field" analyses of neural networks.Which statistics of the neural populations should we then measure and compare? Guth et al. (2023) have shown that the covariance of neuron weights captures most of the encoding properties, as knowledge of the weight covariances can be sufficient to generate new networks with similar performance.In particular, the covariances at the end of training are the sum of a "learned" component and a "random" component, the latter mostly resulting from the initialization. The learned component is spanned by the leading eigenvectors and is rather low-dimensional, as low-rank approximations of the learned weights result in negligible performance loss (see Figure 2 ).Summary. Though the overall behavior of the network only depends on the collective properties of the neurons, their weights are expressed in a basis defined by the individual neurons of the previous layer. To compare weights between two networks, for each layer, we thus compute the alignment matrix A between the input representations and use it to align the neuron weights of both networks. The learned encodings can then be characterized by the leading eigenvectors (and eigenvalues) of the weight covariances.Similarly to our approach for spatial covariances in Section 2.2, we can compare the eigenvectors of two channel covariance matrices after alignment. These eigenvectors however cannot be easily visualized and interpreted, though can be compared by measuring cosine similarities between them.Another challenge is that channel covariance matrices are significantly higher-dimensional than spatial covariance matrices, and the number of neurons (which here correspond to samples) is comparable to or even smaller than the dimension. This results in inconsistent estimations of the covariance matrices: in particular, their eigenvalues and eigenvectors deviate significantly from their "ground-truth" values (if the number of neurons were infinite). Assessing whether two networks have the same channel covariances thus requires some care. The dimensionality and similarity measures of weight covariances we now introduce thus rely on a shrinkage of the covariance eigenvalues.Eigenvalue shrinkage. The decomposition of the weight covariance in "learned" and "random" components corresponds to the so-called "spiked" covariance model (Johnstone, 2001) , under which only eigenvectors with sufficiently large eigenvalues can be estimated (Baik et al., 2005) . The learned component of the covariance is thus optimally estimated by shrinking the eigenvalues of the empirical covariance matrix (Donoho et al., 2018) . More precisely, for a weight matrix of size C out × C in , let γ = C in /C out , which measures the "sampling noise". The eigenvalues of the C in × C in weight covariance matrix are shrunk according to the formulaEQUATIONwith a normalization so that the initialization variance is equal to 1. This removes most of the noise resulting from the limited number of neurons. We can thus meaningfully compare aligned weight covariances through their estimated "learned" components.A first, indirect way to compare the resulting learned covariances is by comparing their dimensionality. We define the effective rank r eff of a matrix as its eigenvalueweighted mean rank. More precisely, if the covariance eigenvalues are λ 1 ≥ • • • ≥ λ d after shrinking, we haveEQUATION)We will see that this dimensionality measure is correlated with the difficulty of task (e.g., the number of classes).A more direct and fine-grained comparison can be done by comparing individual eigenvectors, as in Section 2.2. However, this suffers from instabilities, e.g. when individual eigenvalues are too close. A more quantitative, summarized measure of similarity may be desired, which can also more easily reveal trends as a function of hyper-parameters.First, we wish to quantify the level of similarity between two covariance matrices C 1 and C 2 after eigenvalue shrinkage. A relevant metric is the Bures-Wasserstein distance (Bhatia et al., 2019) , which corresponds to the optimal transport distance between two centered Gaussian distributions of respective covariances C 1 and C 2 (Peyré & Cuturi, 2019, Remark 2.31) . It thus measures the displacement of individual neuron weights that is needed to change their global covariance from C 1 to C 2 . It allows us to compare neural encodings by accounting for their statistical shapes and overlap. This distance is defined asd 2 (C 1 , C 2 ) = tr C 1 + tr C 2 -2∥C 1/2 1 C 1/22 ∥ 1 , where ∥•∥ 1 is the nuclear norm (the sum of the singular values). It can be turned into a cosine similarity withEQUATION)This metric allows quantifying similarities at the level of covariances rather than individual eigenvectors and is therefore better suited at revealing correlations potentially diluted over ranks. For meaningful comparisons across varying layer dimensionalities, we normalize the resulting value. Its zero point is defined as the cosine similarity cos θ(C 1 , O C 1 O T ) between C 1 and a random rotation O ∈ O(R d ) of it, leading to non-zero but minimal eigenvector correlations. The upper bound is defined as the cosine similarity cos θ(C 1 , C 1 ) after a "resampling" of the covariance C 1 . We define C 1 as the estimated (shrunk) covariance obtained from C out random Gaussian vectors of covariance C 1 + σ 2 Id, where σ 2 Id is the initialization variance. This leads to the normalized cosine similarityEQUATION)This normalization enables comparisons of similarity levels at different layers, and can thus be used to reveal trends across depth.We now compare the properties of neural encodings for networks trained on different datasets and tasks. To do so, we measure the absolute value of the cosine similarity between eigenvectors of the (aligned) weight covariances for the same factorized VGG architecture with frozen spatial filters but different training contexts. We present our results in Figure 3 . Each panel shows encoding similarities of a given convolutional layer as a function of rank. We consider different training datasets and use subsets of CIFAR10, CIFAR100, and ImageNet all downsampled to 32 × 32 resolution. We use different tasks: true labels or random labels. The suffix (bis) refers to a different random initialization.Using the same image resolution and architecture for all datasets ensures that all networks have the same number of layers and receptive field sizes, allowing meaningful comparisons. For each trained network, the effective rank r eff is indicated by an arrow overlaid on the rank axis.We begin our exploration by considering two networks trained on ImageNet but with different random initializations. The encoding similarities are shown in the top row of the figure. First, we can observe that, as depth increases, layer-based encodings become higher-dimensional. In addition, the eigenvector similarities shown as a function of ranks show that the learned channel eigenvectors of the two networks are very similar. This similarity will be quantified below. This comparison, for which only the random initialization changed, provides a reference for our next experiments. This is the maximum level of similarity one can expect.We now consider networks trained on different datasets: CIFAR5 is the subset of CIFAR10 composed of the first 5 classes and ImageNet100a/100b are two randomly chosen but disjoint subsets of ImageNet classes. We present the encoding similarities in the bottom block of the figure. Interestingly, within the relevant ranks, we find a high level of weight eigenvector similarity for all pairs of training sets and over an appreciable range of layers. We can observe that datasets with more classes or more diversity (such as ImageNet100 compared to CIFAR100) lead to encodings with more relevant eigenvectors. Deeper into the networks, these similarities gradually vanish. This is expected as the encodings have to become task-specific toward the final classifier. This overview shows that, over a range of layers, networks learn a universal and relatively low-dimensional encoding of natural image datasets. This implies that, in principle, such encodings do not need to be learned and could be originally preset in a given architecture. Only the deeper task-dependent layers encodings would have to be learned. Figure 3 suggests that, for a range of layers, different trained networks provide us with different samplings of what could be a universal encoding of natural image datasets. These results reveal the origin of transfer learning capabilities.We then consider networks trained on different classification tasks: using true or random labels with ImageNet downsampled to 32 × 32 resolution. For these experiments, we use data augmentation with random labels: it thus encourages the network to be invariant to horizontal flips and small spatial shifts while compressing the training dataset. The results are presented in the top block of the figure.Interestingly, for our VGG-type architecture, the weak correlation between the encodings of true and random label classification tasks reveals that the corresponding encoding strategies are significantly different. However, when comparing different realizations of random labels ("a" and "b"), one finds a common encoding scheme over a range of layers.This overview of encoding similarities is summarized in Figure 4 . It shows the normalized cosine similarity between the covariances of two trained networks (eq. 6), allowing a more quantitative comparison. In the left panel, we start by showing the encoding similarity for two networks trained on ImageNet but with different random initializations (blue curve). We then show the encoding similarities in the case of random labels (red and green curves). Similarly, we observe a high level of similarity up to the layer ℓ = 5, showing that learning on different sets of random labels leads to similar encodings. In contrast, as mentioned above, when we compare what is learned with true and random labels, we find a much lower correlation: the two encodings are mainly different. We detect a subset of shared eigenvectors leading to a correlation amplitude of about 0.2, showing the existence of some channel eigenvectors systematically learned in these two tasks. In the right panel, we show the encoding similarity for all of the pairs of datasets considered above. It reveals a continuous trend with depth and shows the existence of a shared encoding at most layers. In summary, our exploration indicates the existence of some level of universality and two rather distinct encoding strategies which are shared across disjoint sets of tasks, when true or random labels are considered. 
discussion: 
 Does a universal encoding of natural images emerge in networks trained on different datasets? What does it look like? Answering these questions at the most fundamental level requires an investigation of the network weights. To do so, we have developed a procedure to compare the weights of networks rather than their representations. It extends the work of Guth et al. (2023) by including a spacechannel weight factorization of standard CNN architectures, as well as dimensionality and similarity metrics which rely on optimal shrinkage of covariance eigenvalues. Our approach can be used to measure similarities between learned network weights and, equivalently, similarities between different datasets through their neural encodings. It can be used to define universality classes of datasets.We have found that space and channel dimensions can be considered separately. In both cases, the relevant learned features are encoded through weight eigenvectors. Along spatial dimensions, we have evidenced a compact set of spatial filter eigenvectors which appears to be universal. It does not depend on filter size, layer depth, or training dataset over a wide range of settings. The same spatial filters also emerge when a network is trained on random labels. The learned channel eigenvectors also show signatures of universality. We show that they display a high level of correlation between datasets and tasks over an appreciable range of layers. Interestingly, we also show that when trained on random labels, VGG-type networks use a different encoding strategy. This second encoding however does not depend on the realization of the random labels, hinting that universality classes only depend on the amount of label noise. We do not attempt to characterize channel eigenvectors individually. In addition, assessing whether they are similar across layers is more challenging than along space and beyond the scope of this work. We do not yet know if similar functions are implemented at different layers. If so, one could develop architectures with weight-sharing across layers, greatly simplifying the learning process.Our similarity metric can be used to identify trained networks with different or unexpected properties. It is now possible to measure the effect of architecture and training recipes on what is learned. For instance, it would be interesting for future work to look at the effects of adversarial training on the learned weight covariances as well as the type of task (classification versus generation). The metrics we introduce can also be computed at various points during training, and can thus be used to compare learning trajectories instead of just endpoints. Our results explain at a more fundamental level the success of transfer learning, self-supervised learning and foundation models. Instead of aiming at maximizing performance when training, one can also attempt to maximize the universality of the learned encoding and get closer to a foundation model. What are the architectural components that lead to increased universality of the learned features? Furthermore, an ensemble of trained networks can be viewed as a set of partial projections of such a foundation model. What would be the best strategy to define this universal encoding? K C in to C out and is applied over channels only. It is thus a C out × KC in matrix, and corresponds to setting the kernel size of the convolution layer to 1.Network weights are initialized with i.i.d. samples from a uniform distribution (Glorot & Bengio, 2010) with so-called Kaiming variance scaling (He et al., 2015) , which is the default in the PyTorch library (Paszke et al., 2019) . Networks are trained for 90 epochs, with an initial learning rate of 0.005 that is divided by 10 every 30 epochs. We use the SGD optimizer with a momentum of 0.9 and no weight decay. We use classical data augmentations: horizontal flips and random crops for CIFAR10 and CIFAR100, and random resized crops of size 224 and horizontal flips for ImageNet and subsets.For the random label experiments in Figure 1 (b), following Zhang et al. (2021) , we disabled data augmentation, removed batch normalization layers, and doubled training time (with learning decays being performed every 60 epochs). We also increased the initial learning rate to 0.02. For the random label experiments in Figures 3 and 4 , we kept data augmentation and batch normalization layers, and quadrupled the training time with an initial learning rate of 0.005. 
 
 Appendix:  



Reviews: review: title: title: Interesting paper, strong inclusion to workshop
main_review: main_review: This paper introduces novel experiments with takeaways that will be relevant to the community. 

## Strengths
- The authors introduce relevant questions (r.e. weight alignment between networks) and some relevant early results showing that the weights of vision models are often relatively aligned.
- The experiment described in section 3.2 and in Figure 5 is particularly interesting. The description and motivation for the alignment is likewise very clear.

## Weaknesses
- The experiments could use significantly better baselines/strengthening. For example, do the suggested results hold for models trained on data that lack relevant properties, e.g. non-images or for random initializations (and other relevant baselines)? The authors claim that the results explain transfer learning, what actually happens for models under transfer learning / finetuning? Do these results hold for models trained under different experimental settings, e.g. different optimizers, different datasets, architectures, and so on?
- Some of the claims are a bit strong. 
   - In particular, in the abstract that the results "explain, at a more fundamental level, the success of transfer learning" does not seem clearly substantiated. The results in the paper give evidence, but the authors do not e.g. exhaustively show that transfer learning is not possible without this structure.
   - Likewise, the citations to claims in (Guth 2023) around universality and the relationship between task performance and effective rank are perhaps strong. For example, while the experiments might explain the models considered, they might not explain other crucial generalization properties of deep learning models [1].
   - I am unsure about the relevance to grokking. Under the definitions of grokking I have seen, the network abruptly finds a *generalizing* solution. 
- Missing related work, e.g. [3] and references therein.

### Question
- How would things change in the results given in Fig. 5 if instead of rotations you used permutations, e.g. the permutation weight alignments used in [2]?
- The authors compare the weights of networks. How do such comparisons (e.g. in Fig 6) compare to related representation comparisons?

[1] Vyas, Nikhil, Yamini Bansal, and Preetum Nakkiran. "Empirical limitations of the NTK for understanding scaling laws in deep learning." Transactions on Machine Learning Research (2023).

[2] Ainsworth, Samuel K., Jonathan Hayase, and Siddhartha Srinivasa. "Git re-basin: Merging models modulo permutation symmetries." arXiv preprint arXiv:2209.04836 (2022).

[3] Huh, Minyoung, et al. "The low-rank simplicity bias in deep networks." arXiv preprint arXiv:2103.10427 (2021).
strength_weakness: strength_weakness: See strengths above.
title: title: A good paper, recommended for acceptance
main_review: main_review: The paper motivates to give methods for comparing the weights between different models, rather than the conventional approach of comparing representations, where the authors employ the correlation coefficient between spatial covariance as the quantitative measure of similarity. They show that universal encoding pattern emerges in regards to both the spatial filters and channel weights over various datasets and tasks. 

The paper is easy to follow and provides good presentations. I find the paper is quite interesting. Directly comparing weights between different models is challenging since no prior will be introduced during the comparison. And It is apparently beneficial to study the similarity metric between models. Additionally, this paper gives insights in regards to what could the model learn universally in different datasets.

Meanwhile, some concerns may also be raised. Firstly, as an empirically-focused study, the experiments could be further improved to better support the results. Secondly, I think the author may give also comparisons between models with same initialization. Also, the author may show the impact of techniques on this similarity. Thirdly, it is easy to come up with this interesting question: could we exchange the filters if they are quantitative similar? If so, it could be a solid evidence for the proposed metric.
strength_weakness: strength_weakness: This paper proposes a novel metric for comparing similarity between filters in CNNs. And this paper provides interesting insights regarding the universal encoding pattern between models and tasks. The authors present valuable findings, which can deepen our understanding towards the deep learning.
title: title: A pA paper that presents some interesting tools for comparing network weights in CNNs along with experiments which support their value
main_review: main_review: Strengths: 
- This paper is mostly well-written. The diagrams communicate the experimental results well and are visually appealing.
- The reviewer agrees with the paper that the analysis of network weights has been understudied relative to the representations of data. This is a challenging problem and this work contributes what appear to be two useful analytical tools to this effort.
- The experiments are well chosen and interesting. In a maintrack version of the paper more experiments will be needed (see Weaknesses section below), but this is a good start.

Weaknesses:
- The one area for improvement is the mathematical descriptions. Trying to describe variants of convolutions in words is challenging and adding symbolic descriptions would be useful. For instance, this reviewer had to go back to Trockman, et. al. (2023) to confirm their understanding of the details in the spatial eigenvectors section. A symbolic description (with spatial and channel indices) would have made this unnecessary. The same observation holds for Section 3.1.
- The reviewer found the experiments on universal features to be compelling. To make the paper stronger, these should be further explored to better understand the extent to which they hold for different models, a broader swathe of datasets, and different training approaches. For instance, it would be very interesting to understand to what extent the universal features learned are impacted by details of the architecture of the network. Similarly, this reviewer was curious about the impact of dataset diversity vs dataset (image) resolution. ImageNet has higher resolution and is also arguably more diverse than CIFAR10, so it was hard to disentangle this from the present experiments.
- The weight analysis methods that are described in this paper all make sense. However, measuring and reasoning about complex and high-dimensional data like network weights is always challenging and the potential for pitfalls is ever present so it would probably make sense to include more sanity checking experiments in a future iteration of the paper. Specifically, 'are these experiments measuring what we think?'


Nitpicks: 
- What are ‘IT spiking responses’?
- In Section 3.3, rank is displayed as a ‘value is indicated by an arrow overlaid on the rank axis.’ It took me a long time to figure out where this was. It might be good to have this better labeled in the figure.

Questions:
- The covariance values in Figure 3 vary from .9 to .99999. The reviewer wonders how meaningful differences in similarity on the order of .01 and smaller are. Is it likely that this is measuring something meaningful?
strength_weakness: strength_weakness: This is an important research direction, and this reviewer feels that the proposed tools have the potential to shed light on representation learning in deep neural networks. The initial results on universal features are interesting and would likely be of interest to the community.