Correcting Biased Centered Kernel Alignment Measures in Biological and Artificial Neural Networks

Authors: alex  murphy
joel  zylberberg
alona  fyshe
Venue: ICLR_cc_2024_Workshop_Re-Align
Published: 2024-02-09 04:51:27

Abstract: Centred Kernel Alignment (CKA) has recently emerged as a popular metric to compare activations from biological and artificial neural networks (ANNs) in order to quantify the alignment between internal representations derived from stimuli sets (e.g. images, text, video) that are presented to both systems (Sucholutsky et al., 2023; Han et al., 2023). In this paper we highlight issues that the community
should take into account if using CKA as an alignment metric with neural data. Neural data are in the low-data high-dimensionality domain, which is one of the cases where (biased) CKA results in high similarity scores even for pairs of random matrices. Using fMRI and MEG data from the THINGS project (Hebart et al., 2023), we show that if biased CKA is applied to representations of different sizes in the low-data high-dimensionality domain, they are not directly comparable due to biased CKA’s sensitivity to differing feature-sample ratios and not stimuli-driven responses. This situation can arise both when comparing a preselected area of interest (e.g. ROI) to multiple ANN layers, as well as when determining to which ANN layer multiple regions of interest (ROIs) / sensor groups of different dimensionality are most similar. We show that biased CKA can be artificially driven to its maximum value when using independent random data of different sample-feature ratios. We further show that shuffling sample-feature pairs of real neural data does not drastically alter biased CKA similarity in comparison to unshuffled data, indicating an undesirable lack of sensitivity to stimuli-driven neural responses. Positive alignment of true stimuli-driven responses is only achieved by using debiased CKA. Lastly, we report findings that suggest biased CKA is sensitive to the inherent structure of neural data, only differing from shuffled data when debiased CKA detects stimuli-driven alignment.

Main paper: 
 introduction: 
 The goal of measuring (or inducing) alignment between biological and artificial neural networks relies on the selection of a metric that can correctly quantify representational alignment between two systems (Sucholutsky et al., 2023) , while also being invariant to transformations that can cause similar representations to appear unrelated. CKA has recently gained popularity as a metric to study alignment between brain responses and deep learning models (Sucholutsky et al., 2023; Han et al., 2023) . Furthermore, recent work to create more brain-like ANNs has used CKA in a loss function to force models to exhibit more brain-like responses that maximize CKA-alignment to held-out neural data, resulting in models that are more robust to adversarial attacks (Dapello et al., 2023) .In the context of modern deep learning, it is common to have dataset sizes multiple orders of magnitude higher than what can be expected for neural data (Smucny et al., 2022) . Therefore, when studying the alignment between ANNs and brains, similarity measures are applied to matrices which contain many more columns (features) than rows (samples). Previous research has noted that this situation is where standard ("biased") CKA becomes an unreliable alignment metric that can potentially lead to erroneous conclusions (Kleemann et al., 2007; Smilde et al., 2008) . This work demonstrates this unreliability in a practical set of experiments, and promotes the adoption of a debiasing step put forward in Kornblith et al. (2019) , which we show is robust to the aforementioned weakness. Though Kornblith et al. (2019) published their work in 2019, biased CKA has still appeared in publications. We further highlight the importance of using both random and shuffled controls when quantifying alignment between neural networks and brains.We first examine the sensitivity of biased CKA to alignment of two random matrices, where one is of a fixed shape and we systematically increase the feature dimension of the other. We compare three alignment metrics: (i) biased CKA, (ii) debiased CKA and (iii) the RV2 Coefficient. We then examine how varying feature dimensionality alone can result in potentially misleading conclusions by contrasting biased CKA and debiased CKA across two CNN models (ResNet18 & CORnet-S), two brain data modalities (fMRI & MEG) over the same image stimuli set. We compare across three experimental manipulations: (i) using the correct data order (where samples and features are matched correctly), (ii) shuffled neural data and (iii) random data. The final section outlines interesting findings regarding biased CKA's insensitivity to shuffled data, in which we show that only in the debiased version do we see sensitivity to true sample-feature pairs. Additionally, we suggest that biased CKA is sensitive to the inherent structure of neural data responses and can thus be decomposed into such a component, in addition to true stimuli-driven responses detectable using debiased CKA.The primary contributions of this work are:1. We provide a systematic investigation of CKA for the alignment of biological and artificial neural networks, arguing for the importance of using random and shuffled controls and of applying the debiasing step to avoid erroneous conclusions. 2. We show that without debiasing, CKA alignment of data matrices that have a larger number of columns than rows is an unreliable quantification of similarity. This situation is a common occurrence when comparing ANN models with neural data when the number of parameters between ANN layers or neural data set sizes are not equal. 3. We show how biased CKA can give the incorrect impression that fMRI / MEG have similar alignment scores across multiple ANN layers, which the debiased version of CKA successfully detects as erroneous. 4. Our experiments support the idea that biased CKA is decomposable into at least two components: one that is sensitive to the generalized structure of neural data (e.g. prototypical fMRI responses to image viewing) and one that is sensitive to true stimuli-driven responses, while debiased CKA is only sensitive to the latter component. 
mathematical formulation & properties: 
 Centered Kernel Alignment (Cortes et al., 2012; Kornblith et al., 2019) is a metric used to quantify the similarity of representational structure between two matrices, each of which must be matched for the number of samples but can differ in the dimensionality of each feature space, i.e. X ∈ R N ×P 1 , Y ∈ R N ×P 2 , but P 1 ̸ = P 2. Unlike correlation/cosine-based measures of matrix similarity (e.g. Representational Similarity Analysis (Kriegeskorte et al., 2008) ), CKA quantifies similarity using the dot-product operation. A generalization of dot-product similarity for centered matrices is the Hilbert-Schmidt Independence Criterion (HSIC), which is a kernel-based operation. Namely, if K and L represent the output of two kernel operations on the original data, the empirical estimator for HSIC is given by the following, where H is a centering matrix:HSIC(K, L) = 1 (n -1) 2 tr(KHLH) (1)CKA is the extension of HSIC through the application of a normalization step in order to introduce invariance to isotropic scaling:EQUATIONRegarding the selection of an appropriate kernel in CKA, it is virtually ubiquitous to use CKA with a linear kernel since initial experiments demonstrated that it performed comparatively well to nonlinear kernels (Han et al., 2023; Davari et al., 2022a) . We only consider the linear variant in this work.The invariance properties of CKA are important to consider with regard to the other similarity metrics to which it has previously been compared. CKA is invariant to orthogonal transformations and isotropic scaling, but not to invertible linear transformations, unlike other methods such as Canonical Correlation Analysis (CCA). Kornblith et al. (2019) showed that several common similarity metrics were unable to detect similarity between corresponding layers of neural networks trained using the same architecture and hyperparameters, but different random seeds. In contrast, CKA was able to detect this similarity. Operations such as batch normalization apply an invertible linear transformation that changes ANN behavior, and it's not clear that a similarity metric should be invariant to such operations (Thompson et al., 2019) . Other interesting findings using CKA to provide insights into the representational structure in ANNs were that wide networks learn similar representations and that saturation of feature channels is more commonly a feature of higher layers rather than lower layers (Nguyen et al., 2021) . We next examine some of the criticisms of CKA and the attempts to address identified weaknesses.2.2 CRITICISMS Davari et al. (2022a; b; 2023) show that CKA is sensitive to subset translations, and that targeted attacks on the metric can artificially and arbitrarily modulate CKA similarity values, without changing the functional behaviour of a neural network. Thompson et al. (2019) point out that the standard implementation of CKA described above is highly sensitive to dataset size discrepancies, i.e. as the feature-sample ratio increases, CKA will tend towards 1.0 even for random, unrelated matrices. This is a situation that is common when dealing with neural data, which often results in large feature dimensionalities (e.g. fMRI) from costly experiments, which restricts the amount of data that can be collected. This results in a comparatively lower number of samples that can be concurrently presented to both biological and artificial neural networks. Yao et al. (2020) reported that this issue was also present in biological analysis, e.g. transcriptomics studies, where feature dimensionality of gene expression profiles is much larger than the number of collected samples. Linear CKA is equivalent to a metric introduced in Robert & Escoufier (2018) called the RV Coefficient (Thompson et al., 2019; Klabunde et al., 2023) , which Smilde et al. (2008) reformulated in terms of random matrices and showed that this metric can be approximated by the following formula, where N denotes the number of samples and P 1 , P 2 denote the feature dimensionality of each matrix:EQUATIONBy subtracting the diagonal elements off the Gram matrix, namely AA T -diag(AA T ), we eliminate the dependence of the metric value on the dimensionalites of P 1 and P 2 , which are contained within the diagonal of the matrix. This results in the modified RV Coefficient (RV2) that is no longer sensitive to sample-feature ratio discrepancies.In an updated version of their paper, Kornblith et al. (2019) point out that earlier work (Song et al., 2007) derived an unbiased estimator of HSIC by recognizing that HSIC can be formulated as a U-Statistic. They substituted a more numerically stable formation of this estimator (See A.5)) from Székely & Rizzo (2014) into the kernel-centering step of CKA, resulting in a debiased version, which fixed the aforementioned issues. The range of the debiased version of CKA is [-1, 1], unlike biased CKA which is in [0, 1].Linear CKA is similar to Representational Similarity Analysis (RSA), a common tool in studying neural representations between different biological / artificial systems (Kriegeskorte et al., 2008) .Whereas CKA implements dot-product similarity, RSA often implements correlation/cosine-based similarity. The primary consequence of this difference manifests itself in RSA's sensitivity to rotations of the data (Mehrer et al., 2020) , which could be important in some scenarios, such as differing head positions in M/EEG acquisition. Kornblith et al. (2019) showed the inability of common alignment metrics to detect similar representations among identical ANN layers trained in identical scenarios but over different random seeds. This highlighted an issue that has important implications in studying model-brain alignment. The mammalian visual processing hierarchy, like ANNs, takes in representations from previous processing stages and applies computations on those representations which are further passed to downstream layers. This raised the question as to whether earlier alignment metrics might have also been insensitive to true similarities among different visual processing systems. CCA, for example, is invariant to invertible linear transformations and might not detect truly similar representations, which CKA would be able to detect. The increased sensitivity led to the consideration of CKA as a promising metric to study alignment in biological and artificial neural networks.  
brain representations: 
 In order to test the effect of bias correction on CKA used as an alignment metric to quantify similarity of neural network responses with human neural data, we used images, fMRI and MEG responses from the THINGS dataset (Hebart et al., 2023) . This is a public repository of neural responses to a suite of naturalistic object images on relatively clear backgrounds, designed to represent solely the specified class label with as little confounding background information as possible. The fMRI data consist of three participants, while the MEG data consist of four participants. We used data from all three fMRI participants and selected participants one, two and four from the MEG data (randomly chosen). Participants in the fMRI experiment saw a set of 720 image classes (12 images in each class) from a larger image dataset consisting of 1,854 image classes. MEG participants viewed all images in the larger image dataset. The fMRI data were preprocessed using a variant of GLMSingle (Prince et al., 2022) , resulting in voxelwise beta regressors per image and required no further preprocessing. Further details relating to the acquisition protocol are available in Hebart et al. (2023) . Details relating to preprocessing steps we applied to the raw MEG data are given in Appendix A.2. We extracted fMRI responses across the following ROIs: V1, V2, V3, hV4, VO1, TO1, lFFA, rFFA, lPPA, rPPA, lEBA and rEBA (functional ROIs prefixed with l-/r-designate left/right brain hemispheres, while retinotopic visual regions implicitly encompass both). For the MEG data, we restrict our analysis to occipital electrodes (N = 39). Our choice of sensors / ROIs is motivated by capturing neural responses along the visual processing hierarchy. Further information relating to per-participant neural data dimensions is given in Appendix A.4.We employed two convolutional neural network (CNN) models in order to extract layer-wise activations to a subset of the THINGS stimuli set mentioned in Section 3.1. Specifically, we used ResNet18 (He et al., 2015) and CORnet-S (Kubilius et al., 2019) . ResNet models have been widely used to study alignment between ANN representations and neural data (Bennett & Baldassano, 2023; Han et al., 2023) . We specifically selected ResNet18 due to its computational tractability, prominence in prior work and for its similarity in layer count to the CORnet model family, which was specifically designed to mimic properties of the mammalian visual processing hierarchy during Core Object Recognition and has also been widely used in brain-model alignment research (Federer et al., 2020; Han et al., 2023) . We selected the CORnet-S variant because it had the highest score on the Brain-Score benchmark (Schrimpf et al., 2018; 2020) . CORnet-S is subdivided into block-like structures: V1, V2, V4 & IT (analogous to the notion of stages in ResNet models). Both models were trained using the ImageNet dataset (Deng et al., 2009) . Layer activations were extracted and saved using the Python toolbox THINGSvision (Muttenthaler & Hebart, 2021) for a single image from each of the 720 classes used in the fMRI experiment and a single image from each of the 1,854 classes used in the MEG experiment. 
cka sensitivity to random data: 
 As mentioned in Section 2.2, one undesirable property is the fact standard CKA is sensitive to large discrepancies between the dimensionality of the feature spaces being compared. We use a reference matrix of random values A ∈ R 1024×1024 and compute the similarity with matrices B ∈ R 1024×P where P runs from [10, 250k] on a log scale (geometric progression). We calculate similarities using: (i) biased CKA (red), debiased CKA (green) and RV2 (black). As the number of features increases, we move from the high-data low-dimensionality regime (Figure 1A ) up to a commonlyoccurring scenario in neural / biological data analysis where feature dimensionality vastly outnumbers the samples (Figure 1C ).As shown in Figure 1 , the RV2 and debiased CKA metrics are not sensitive to large feature-sample ratio differences and accurately detect that there is no similarity among the matrix representations, since the dependence on the sample-feature ratio is removed. The debiasing step in CKA is more widely applicable to any choice of kernel, while RV2 is strictly limited to the linear case, so we do not perform any further comparisons using the RV2 metric.Neural measurements can exhibit low signal-to-noise ratios resulting from, for example, loose sensors, misaligned voxel locations due to excessive head movement or just background activity not connected to any experimental stimuli, which has the potential to resemble random data. This fact suggests that biased CKA is not an ideal metric to directly compare among representational spaces if there is a large difference between their cardinalities. There are other instances in which inflated similarity results can arise when using biased CKA: (i) when fixing neural data (e.g. ROIs in fMRI or sensor groups / temporal windows in M/EEG) and comparing the alignment with each layer in an ANN and (ii) when fixing an ANN layer and comparing alignment across ROIs, temporal windows or sensor groups. In these scenarios the ratio of the dimensionality of the data is not fixed, which could cause issues when using biased CKA to quantify representational alignment. We examine this in the next section.  
cka sensitivity to stimuli-driven neural responses: 
 The previous section showed how biased CKA is sensitive to changes in the ratio of sample-feature dimensionality for random matrices. We now show how this issue can lead to erroneous conclusions in a standard alignment experiment between biological and artificial neural networks across two types of brain recordings, matched on the same stimuli set and two CNNs trained on the same data (see Methods for full description). We selected primary visual cortex (V1) and occipital sensors for each of the three participants in the fMRI / MEG data, respectively. ANN responses were calculated across layers of ResNet18 and CORnet-S while keeping neural data fixed (Figure 2a-b ). This was also done when fixing a CNN layer and calculating similarity scores across varying neural data sizes (fMRI ROIs in Figure 2e and increasing temporal windows of occipital MEG responses in Figure 2f ). In each plot, we contrast biased CKA with debiased CKA across three data manipulations: (i) random data (See Appendix A.1), (ii) shuffling the rows of the neural data such that there is no correspondence between CNN representations and image-level neural responses and (iii) with the original data order, where sample-feature pairs are matched.We first see that CNN representations across both ResNet18 and CORnet-S result in high alignment values to random ("neural") matrices, where the CNN representations are unchanged. This inflated result could lead to the erroneous conclusion with noisy / near-random neural data that both CNN models extract interesting representations that align well with the associated neural responses. When a CNN layer is fixed (Figure 2e -f) and we sample random data to match the size of ROIs or sensor groups, we show how conclusions based on biased CKA results alone could be misinterpreted as meaningful alignment when this effect is driven by discrepancies in feature dimensions.For true neural responses (both in the original order and shuffled) we see increasing biased CKA alignment over layers of both CNNs. For fMRI, we see a slightly elevated value over the corresponding shuffled version, but for MEG data these values are equal. For debiased CKA, it correctly reports zero alignment with random matrices and shuffled data. For fMRI, we do see small positive alignment values for debiased CKA in the original data order, while this is absent from the MEG data. Biased CKA without a shuffled-data control would result in a potential conclusion that both ResNet18 and CORnet-S extract similar information, however this would not be due to stimulidriven responses. Only in the case of debiased CKA do we see a positive alignment value in this case (for fMRI). We now examine the effects of biased and debiased CKA on shuffled and unshuffled data.6 CKA SENSITIVITY TO SHUFFLED DATAIn Figure 2 we observed that biased CKA consistently resulted in similar alignment scores for both shuffled / unshuffled data orderings across both fMRI / MEG and two CNN models (ResNet18 / CORnet-S), while debiased CKA only reported positive alignment scores for unshuffled data, i.e. the true stimuli-driven responses in the fMRI results. In this section we examine this finding in more detail and suggest an explanation for the observed results.The Manifold Hypothesis states that real-world data are expected to concentrate in the vicinity of a manifold of lower dimensionality, embedded in a higher-dimensional space (Bengio et al., 2013) .Deep learning models aim to learn a parameterization of the manifold of the data they are trained on (Lei et al., 2020) . If a set of input data is presented to experimental participants, whose brain responses are recorded, it follows that the neural data will also lie on a lower-dimensional manifold within a higher dimensional space. In the context of alignment between biological and artificial neural networks, this can be seen when permuting neural responses and measuring the alignment with layer activations from deep learning models using a selected metric of interest. When both systems have seen the same input data, it remains an open question how much of the detectable alignment is sensitive to true stimuli-driven neural responses, as well as to a more generalized response that detects the presence of structured neural data, e.g. prototypical fMRI responses during image viewing. Figure 3 shows a comparison between biased and debiased CKA across ResNet18 (CORnet-S results reported in Appendix A.6) for a subset of fMRI ROIs, categorized into early (V1, V2, V3), intermediate (hV4, VO1, TO1) and late (lFFA, lPPA, lEBA) groups (Witthoft et al., 2013) . For the original data order, the reported value is the mean across the three fMRI participants (N = 3), while for the shuffled order, the reported value is the mean across five random shuffles per participant (N = 3 × 5 = 15). For all ROIs and ResNet18 layers, biased CKA indicates an alignment for the original data order which is only slightly larger than for shuffled data. This indicates that biased CKA is not primarily sensitive to stimuli-driven responses. Debiased CKA does not suffer from this undesirable side-effect and returns a positive value only for stimuli-driven neural responses. It is only via debiased CKA that we recover previously-reported observations that early visual regions are more aligned with lower layers of CNNs, while later regions are more aligned with the upper layers (Yamins et al., 2014; Yamins & DiCarlo, 2016) . Though this pattern is clear, the values we report are small in magnitude compared to results obtained with biased CKA, which we expect might be associated with the limited dataset of 720 images. We performed an additional analysis (See Appendix A.7) between fMRI areas V1 & left PPA (lPPA) and the first three layers of ResNet18.We gradually varied the amount of shuffling and see that only in the case of positive stimuli-driven alignment scores is there sensitivity to the amount of shuffling (in V1). For lPPA, biased and debiased CKA are insensitive to varying amounts of shuffling. This supports our interpretation that biased CKA is not sensitive to stimuli-driven activity.An interesting observation in Figure 3 is that with respect to the original data order, biased CKA alignment exhibits an increased value over the corresponding shuffled version in line with the magnitude of the debiased CKA score. This is predominantly seen in the early ROIs (V1, V2 & V3), while when debiased CKA does not detect alignment (e.g. lPPA), biased CKA is near-identical for both the original and shuffled data orders. This indicates that when stimuli-driven alignment exists between biological and ANNs, biased CKA is able to detect it as roughly the additional alignment given over and above that which is given for shuffled data. We have posited a hypothesis to explain what biased CKA might be additionally sensitive to, in light of the similarity to shuffled samplefeature pairs. As data exhibit more of an unstructured random profile, biased CKA scores rapidly increase (See Figure 1 ). The similarity between shuffled and unshuffled data pairs points to a sensitivity based on the structure of the neural data itself, which might be characterized by structure that looks like fMRI data, for example. Even when using random data sampled from Normal distributions where voxel-wise / sensor-wise means and standard deviations are used to parameterize these distributions (See Appendix A.1) in order to closer approximate structured neural data, biased CKA results are indistinguishable from random results reported in Figure 2 . The fact that shuffled responses are treated differently lends further support to our idea that biased CKA sensitivity is driven by a generalized neural response pattern. 
conclusion: 
 Using fMRI and MEG data paired with stimuli from the THINGS dataset, we showed that measuring alignment with biased CKA has several weaknesses that are corrected by implementing a debiasing step. Biased CKA can be increased arbitrarily by varying the feature ratio w.r.t. a fixed dataset size (e.g. examining fMRI ROIs w.r.t. to a single CNN layer). This prevents direct comparisons across varying feature dimensionalities which often arises in comparisons between neural data and ANNs (e.g. different ROI sizes / sensor groups, temporal windows, parameter counts). We postulate that biased CKA is sensitive in part to the inherent structure of neural data since biased CKA reveals similar alignment values between shuffled and unshuffled data when debiased CKA results are close to zero. We show that alignment based on the true stimuli-driven responses is only obtained when using the debiased version of CKA. Our analysis did not show debiased CKA alignment with MEG data. We chose one method of preprocessing, but other methods that boost the signal-to-noise ratio (e.g. more data, different settings) will likely increase our ability to detect stimuli-driven alignment with ANN representations. It will be informative to quantify the same analysis across other common alignment metrics, which we leave to future work. As biological and ANN alignment research is an emerging topic in representation learning, methods like CKA are being used not only to quantify alignment but also to induce brain-like representations in neural networks. We have demonstrated some issues when using CKA as an alignment metric in the hope that the community can sidestep erroneous conclusions and spot potential issues when assessing reported alignment metrics in their work and the work of others. (conv1, maxpool & layer1.0.conv1) . We find that the amount of shuffling has no effect on ANN-ROI pairs that originally showed no stimuli-driven alignment. We hypothesize that this is due to biased CKA's sensitivity to a generalized neural response. For early layers of ResNet18 and V1 in Figure 3 , we saw stimuli-driven alignment scores, which are indeed gradually reduced as the percentage of neural data shuffling is increased. 
 
 Appendix:  



Reviews: review: title: title: Nice work
main_review: main_review: The authors in this work argue for debiasing the centered kernel alignment (CKA) measure. Specifically, the authors consider a setting where researchers use CKA to compare the representational similarity between biological and artificial neural networks. The authors use convincing results on simulated, MEG, and fMRI data to make their point, and show the reader that results without debiasing CKA can completely obfuscate the actual representational similarity between biological and artificial neural networks.

**Clarity** \
The authors do a great job of convincing the reader how important it is to use debiased CKA through a variety of experiments. To improve the paper for a potential full conference or journal submission, I would recommend a few things to improve the clarity:
- On Page 4, the sentence before the Methods section reads: “The above approaches that use CKA to quantify/induce alignment between biological and artificial networks specify a form that suggest the standard (biased) implementation was used in those works.”. Although I generally agree it is good to point out potential biases in other people’s works, I do think this statement is a little too implied for the actual experiments you have performed. I understand you reference these works to explain why it is important to bring awareness to debiased CKA, since these works likely still (accidentally) use a biased version of CKA. This can certainly be true, but I believe it would be more constructive in a full conference or journal paper to reproduce their experiments with debiased CKA to verify whether their claims hold up, and how the new results differ. Otherwise I would reword/remove this sentence because it can be read as questioning their conclusions, which the experiments in this paper do not directly tackle.
- On Page 1 “Previous research has noted that this situation is where standard (“biased”) CKA becomes an unreliable alignment metric that can potentially lead to erroneous conclusions.”. It would be helpful to cite this research.
- Figure 2, I think it would help the reader if the y-axes for a and b are log-scale, and the y-axes are aligned between the ResNet-18 and CORnet-S. Currently, for the MEG results, the axes differ between a and b, which makes it harder to compare the results between the two models.

**Correctness** \
All the analyses support the main conclusion in the paper, and seem to be conducted in a clear and reproducible manner. Especially if the authors manage to publish the code upon acceptance, which they promise to do in the appendix.

**Novelty** \
Although the importance and idea of debiasing CKA were introduced in Kornblith et al. 2019, which the authors themselves cite, the analyses in that paper mostly compare representations between layers in the same network. The authors in this work specifically evaluate representational similarity between biological and artificial neural networks. Especially to draw more attention towards explicitly debiased CKA for biological - artificial neural network representational similariy, this work is important. To improve the paper for a potential full conference or journal submission, I would recommend a few things to improve the novelty:
- Page 9, Apart from the first point I mentioned under clarity, I believe the authors can unpack their last sentence before the conclusion more in a full conference or journal paper: “The fact that shuffled responses are treated differently lends further support to our idea that biased CKA sensitivity is driven by a generalized neural response pattern.”. I believe showing this more rigorously through experiments will add to the novelty of the paper. Additionally, if the authors decide to pursue a direction like this, I would recommend moving Figure 3 to the Appendix to reclaim some space. Although it is an interesting result, especially for a workshop paper, I believe it is too similar to Figure 2 to add much to the novelty of the work in a full conference or journal paper.

**Interest to the community** \
I think this work is a perfect fit for the workshop.

**Grammatical/spelling errors (these may be personal preference so feel free to ignore them)** \
- P4, “… found models that were better aligned to human behaviour were more robust” -> human behaviour and were
- P5, “Both models were both trained” -> Both models were trained
- P5, “As the number of features increases, we move from high-data low-dimensionality regime…” -> from the high-data low-dimensionality regime
- P5 “…where feature dimensionality vastly outnumbers the number of samples…” -> vastly outnumbers the samples
strength_weakness: strength_weakness: Although I believe there are some limitations to the paper, including a slight lack of novelty, I believe the importance of presenting this work at a workshop on representational alignment outweighs the negatives. Furthermore, many of my critiques are under the assumption that the authors want feedback on their work for a future archival publication :-)
title: title: Review for "Correcting Biased Centered Kernel Alignment Measures in Biological and Artificial Neural Networks"
main_review: main_review: Summary: This paper presents a methodical evaluation of CKA for alignment of representations between biological (fMRI and MEG) and artificial neural networks. The evaluation of biased and debiased CKA was presented in a well-motivated fashion and well-situated in the representational alignment literature. The authors' motivation for using debiased CKA over the ordinary [biased] CKA was well-supported by their analyses. Overall, I found this paper to be interesting and clearly addressed the points the authors set forth; I have some recommendations for improvement that I outline below.

Strengths:
- A well-suited and motivated dataset choice (THINGS fMRI and MEG) for the problem the authors sought to address
- Reasonable ANN choices and demonstrated generalization across ANNs
- Clear motivation for why biased CKA needs addressing
- Shuffled data and random data were adequate control comparisons
- Clearly written and comprehensive paper

Areas for improvement:
- It was not entirely obvious how the debiased CKA is formulated. Since it is the backbone of the paper, I think the authors should include a formulation of the metric in section 2.2. 
- The point about linear kernels was a bit confusing. In the final paragraph of section 2.2, the authors note that the debiasing step added to CKA is not restricted to linear kernels, which makes debiased CKA more useful than the RV2 coefficient, but in section 2.1 the authors note that "it is virtually ubiquitous to use CKA with a linear kernel." Does debiased CKA with a linear kernel afford anything beyond RV2? If they are not identical, I would want to see a comparison with RV2 at least in Figure 2. 
- Figures 2 (A,B,E,F), 3, and 4 are very difficult to read given the colors and overlays. If the authors think that the best way to visualize the data are to have them as overlaid line/bar plots with common axes, and I understand this given the point the results are trying to drive home, I would highly recommend reconsidering the colors / saturations used for each comparison  (original CKA/debiased CKA and shuffled CKA/debiased CKA). As it currently is, it's impossible to see the CKA debiased results. 
- RSA would be an easy additional benchmark to throw into these analyses, and I think the first necessary additional comparison. 
- A more detailed addressing of the MEG results would be nice. MEG preprocessing pipelines and signal processing is far less developed than fMRI, so it is unshocking to me that the results are less conclusive. However, a more thorough discussion of this point would be valuable, perhaps proposing what specific preprocessing could be changed, why those choices would be relevant, how the sensors were chosen for the current analysis. Could it be that MEG just does not show positive debiased CKA alignment with ANNs, regardless of the preprocessing? These would be valuable discussion points.
strength_weakness: strength_weakness: The work is well-situated in the literature and addresses an important question in how we evaluate the alignment of representations.
title: title: A potentially highly relevant paper.
main_review: main_review: ## Summary
This paper demonstrates the importance of using debiased CKA instead of standard CKA in the context of representational alignment between neural data and ANN activations. The authors point out that neural data is typically in the “low-data, high-dimensionality”-domain, in which standard CKA is systematically biased, thus overestimating alignment. They demonstrate this effect empirically with random matrices of varying feature-sample ratios, showing that CKA converges to a high alignment value as the ratio grows, while debiased CKA correctly maintains its low value. Next, the authors examine CKA values between real fMRI / MEG measurements and a ResNet18 / CORnet-S, showing how CKA drastically overestimates the alignment even for random and shuffled “neural” values, while debiased CKA only takes non-zero values for the real data.

## Strengths
The paper makes a potentially highly relevant contribution to the field of representational alignment, demonstrating the practical relevance and utility of debiased CKA. Furthermore, the paper is clearly written, well-structured and technically sound, as far as I can tell. I appreciate the informative overview of CKA instead of a typical related work section. While the paper does not present a new method or collect new data, it does open a new perspective on existing data and raises a relevant point.

## Weaknesses
Section 5 is very descriptive of figure 2, but I’m actually very intrigued by the implications of findings in figure 2. If the value range of debiased CKA is [0,1], then these results effectively imply that there is very little alignment between ANN activations and fMRI data, and almost no alignment between ANN activations and MEG data - even though previously, one might have believed this to be the case, based on the high standard CKA values. This goes far beyond just noting that debiased CKA is less sensitive to the failure modes of shuffled and random data. Essentially, a bit more context for these results would be nice.

## Recommendation
As somebody not very familiar with CKA, I don’t know whether the community has effectively already made the shift to debiased CKA, or if the practice of using standard CKA is still common. Especially in the latter case, the paper constitutes an important piece of information for practitioners, possibly shedding new light on earlier results. I propose to accept.

## Questions
- What is the value range of (debiased) CKA? Does it yield values in [0,1]? Explaining this would be very helpful to put the findings in figures 3 and 4 into perspective.
- Are there other works which report high standard CKA values and therefore draw conclusions that may now turn out to be erroneous? I am wondering about the practical impact of this finding (e.g. if nobody uses standard CKA, for this reason or others, this finding seems less relevant than if multiple other findings are now called into question).
- On page 8 you write that the small absolute values obtained with debiased CKA could be explained by the small size of the dataset, but as far as I understand, the biased CKA values were obtained using the same set of 720 images, right? So why should the number of images explain this effect, vs. it being a “true” effect, that maybe brains and ANNs are not as aligned as biased CKA leads us to believe? Maybe it would be interesting to quantify how many images one needs to get reliable debiased CKA values, or get some other measure of uncertainty. 

## Additional Feedback
- It is a bit hard to distinguish lines in figure 2, it took me a while to realize that debiased CKA for random data was not actually omitted, but just sits at 0. Maybe plotting with less alpha would help, or using other linestyles. Also, the color scheme is not ideal for colorblind folks.
- What would the debiased CKA for fMRI-data of different participants be? The idea being to establish a noise ceiling - if brain-to-brain alignment is also only 0.2, this puts the absolute debiased CKA values into perspective and says something about the signal-to-noise ratio of fMRI in general. 
- “Previous research has noted that this situation is where standard (”biased”) CKA becomes an unreliable alignment metric that can potentially lead to erroneous conclusions.” $\rightarrow$ maybe cite that other work here
- “Kornblith et al. (2019) showed that neural networks trained using the same architecture and hyperparameters, but across different random seeds, were unable to detect similarity between corresponding layers in several common similarity metrics, but not CKA.” $\rightarrow$ this sentence is muddled, do you mean: “Kornblith et al. (2019) showed that several common similarity metrics were unable to detect similarity between corresponding layers of neural networks trained using the same architecture and hyperparameters, but different random seeds. In contrast, CKA was able to detect this similarity.”?
- “The above approaches that use CKA to quantify / induce alignment between biological and artificial neural networks specify a form that suggests the standard (biased) implementation was used in those works.” $\rightarrow$ “specify a form” sounds a bit unclear here, are you just saying that they probably used the biased implementation, but you’re not sure? It seems pretty relevant to me whether people already know about this issue and are already using debiased CKA or not. You could just ask them what they did.
- “It will also be informative to quantify the same analysis across a host of other common alignment metrics and, which we leave to future work.” $\rightarrow$ remove “and”
- “ANN responses were calculated across layers of ResNet18 and CORnet-S while keeping neural data fixed (Figure 2a-b) and also when fixing a CNN layer and calculating similarity scores across varying neural data sizes (fMRI ROIs in Figure 2e and increasing temporal windows of occipital MEG responses in Figure 2f).” $\rightarrow$ maybe break this into two sentences to increase clarity.
- “CCA, for example, is invariant to invertible linear transformations and might not detect truly similar representations, which CKA would be able to detect.” $\rightarrow$ isn’t CCA’s invariance a desirable property here? If CCA is invariant, that means it would detect that representations are truly similar. 
- On page 7, you refer to figure 4 but mean figure 3, probably the labels of the two figures (ROIs for Resnet18 / CorNet-S) got mixed up?
strength_weakness: strength_weakness: The paper demonstrates a serious issue with standard CKA. Using debiased CKA instead seems to be something that other researchers should know about. While the method itself is not new, it appears to me that this convincing demonstration of the need to actually use the debiased version could shed new light on other results.