SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Wei Chen†, Zichen Miao†, Qiang Qiu
{chen2732, miao z, qqiu} @purdue.edu
Analyzing representational similarity among neural networks (NNs) is essential for interpreting or transferring deep models. In application scenarios where numerous NN models are learned, it becomes crucial to assess model similarities in computationally efficient ways. In this paper, we propose a new paradigm for reducing NN representational similarity to filter subspace distance. Specifically, when convolutional filters are decomposed as a linear combination of a set of filter subspace elements, denoted as filter atoms, and have those decomposed atom coefficients shared across networks, NN representational similarity can be significantly simplified as calculating the cosine distance among respective filter atoms, to achieve millions of times computation reduction over popular probing-based methods. We provide both theoretical and empirical evidence that such simplified filter subspace-based similarity preserves a strong linear correlation with other popular probing-based metrics, while being significantly more efficient to obtain and robust to probing data. We further validate the effectiveness of the proposed method in various application scenarios where numerous models exist, such as federated and continual learning as well as analyzing training dynamics. We hope our findings can help further explorations of real-time large-scale representational similarity analysis in neural networks.
Deep neural networks (NNs) have shown unprecedented performance in a large variety of tasks [24, 46]. In many scenarios, numerous models are learned and their relations can be beneficial to exploit. For example, as illustrated in Figure 1(a), to aggregate knowledge across space, federated learning (FL) trains models over a large number of clients while keeping data localized. To preserve knowledge across time while learning new ones, continual learning (CL) can be addressed by training a large group of models, one for each timestep. Finding the relations among models is the cornerstone to boosting performance in these scenarios, such as improving personalization for FL [53] or providing knowledge retrieval for CL [40]. Considering the large number of NNs potentially allowed in those scenarios, e.g., to model growing spatial/temporal coverage, it becomes crucial to have a highly computationally efficient way to assess NN model similarity.
We are inspired by one recent state-of-the-art CL framework in [35], where each convolutional filter is represented as a linear combination of a set of filter subspace elements, denoted as filter atoms. It is easy to notice that each convolutional layer now becomes two convolutional layers, a filter atom layer followed by an atom coefficient layer with  $1 \times 1$  filters. Then, motivated by the literature on task subspace modeling [10, 25, 33, 45, 64] that tasks can be modeled as a set of latent basis tasks and their linear combinations, a group of tasks are sequentially modeled using NNs by learning for each task a different set of filter atoms, while sharing common atom coefficients across
(b)
Figure 1: (a) The illustration of scenarios where numerous models exist, such as federated learning (FL), continual learning (CL), and model training process. The relations among models are usually critical, and the computational cost to assess the model relation can be a major bottleneck. (b) Comparison between our method and probing-based methods. (left) Feature space similarity metrics, e.g., CCA, rely on probing data, and calculate the correlation between large groups of features generated by the forward pass of probing data through NNs. (right) In comparison, our filter subspace-based method decomposes convolutional filters  $\mathbf{W}$  as filter atoms  $\mathbf{D}$  (filter subspace elements) and atom coefficients  $\alpha$ ,  $\mathbf{W} = \alpha \times \mathbf{D}$ , and only calculates the filter subspace similarity between a small portion of parameters, i.e., filter atoms, which is independent from probing data and computation efficient. The proposed filter subspace-based method can achieve millions of times computation reduction than popular probing-based methods.
tasks. [35] has in detail analyzed and validated this framework in the CL context. This learning framework with individually modeled filter subspaces but shared rules of linear combinations is generally applicable to many multi-model scenarios, especially CL and FL settings, where numerous NNs with the same architecture are learned. For example, FL learns a single global model to fit the data on all clients, and the majority of aggregating methods in FL require NNs to maintain the same network structure [28, 29, 34].
In the above setting, it is easy to observe that the representation variations across different NNs now become dominated by respective filter atoms. Thus, [35] adopts in experiments filter subspace distance to assess task relevancy, however, without formal justification. In this paper, we formally explore NN representational similarity using filter subspace distance, with detailed theoretical and empirical justifications. We first simplify the filter subspace distance to the cosine distance of two sets of filter atoms, to eliminate the computation of singular value decomposition in calculating principal angles. Then, we show both theoretically and empirically that the obtained filter subspace similarity preserves a strong linear correlation with other popular probing-based similarity measures such as CCA [43], which require external probing data as input stimuli. Our representational similarity is also immune to inappropriate choices of probing data, while probing-based metrics can be perturbed drastically.
Previous works [37, 43] measure representational similarity directly relying on deep representations revealed by input data. These approaches introduce heavy computation from both the forward pass of numerous probing data and the calculation of high-dimensional covariance matrices. As these similarity metrics are probing-dependent, their quality can potentially deteriorate when probing data are inappropriately chosen, scarce or unavailable. Such properties make the popular probing-based approaches less appropriate for our target scenarios where a large number of NN models are present.
The proposed filter subspace similarity shows extreme efficiency in both memory and computation. Since our similarity computation does not involve network forward pass, no GPU memory access is required, whereas other probing-based measures consume the same amount of GPU memory as regular inference. On the other hand, the proposed method involves only inner product calculations on filter atoms, which takes negligible time for similarity evaluation. The evaluation time of probing-based measures includes the time of both the forward pass of probing data and the calculation of high-dimensional covariance matrices. We report later the dramatically improved evaluation time of the proposed method against other popular probing-based methods, e.g., CKA [21]. These unique
properties make our method highly desirable for exploring NN similarity under scenarios with a large number of NN models.
We further validate our filter subspace similarity for knowledge transfer with various CL and FL tasks, as sample examples to exploit NN model relations. In both settings, we fix the atom coefficients, learn the filter atoms for each task, and finally conduct knowledge transfer among tasks by recalling the most similar models for the ensemble. Compared with probing-based similarity metrics, the proposed measure achieves competitive performance with millions of times reduction in the computational cost.
We summarize our contributions as follows,
- We formally explore NN representational similarity measure using filter subspace distance.
- We show both theoretically and empirically that the proposed filter subspace-based measure preserves a strong linear correlation with other popular probing-based measures, while being significantly more robust and efficient in both memory and computation.
- We demonstrate the effectiveness of the proposed similarity measure using several simple examples, such as federated and continual learning as well as analyzing training dynamics.
In this section, we first review probing-based representational similarities and show their limitations. Then, we provide a filter subspace formulation for NNs, and propose a NN similarity metric based on a simplified filter subspace distance. We further demonstrate that under certain assumptions, the proposed measure shows a strong linear relationship with popular probing-based measures, while exhibiting dramatic improvement in computational efficiency and data robustness. These unique characteristics of the proposed measure can potentially enable real-time large-scale NN similarity assessment, e.g., helping fast knowledge retrieval across a large number of NN models.
2.1 Revisiting Representational Similarity in Feature Space
Intuitively, the NN representational similarity can be directly assessed via features generated from different neural networks. As shown in Figure 1(b), it usually includes three steps to evaluate probing-based representational similarity between two NNs  $\mathcal{F}_u$  and  $\mathcal{F}_v$ : (1) Collect an appropriate and sufficient amount of external probing data  $\mathbf{X}_p \in \mathbb{R}^{n \times c' \times h' \times w'}$  that can represent the whole data distribution. (2) Generate the feature  $\mathbf{Z}_u$  and  $\mathbf{Z}_v$  ( $\mathbf{Z}_u, \mathbf{Z}_v \in \mathbb{R}^{n \times c \times h \times w}$ ) by the forward pass of probing data through different neural networks,  $\mathbf{Z}_u = \mathcal{F}_u(\mathbf{X}_p, \theta_u)$  and  $\mathbf{Z}_v = \mathcal{F}_v(\mathbf{X}_p, \theta_v)$ , where  $\theta_u, \theta_v$  denote parameters of two NNs. (3) Choose a probing-based metric to assess the model similarity. Several popular probing-based methods can be adopted in step (3), and we will give a brief introduction below.
CCA. [43] proposes to analyze the NN representational similarity by conducting canonical correlation analysis on  $\mathbf{Z}_u, \mathbf{Z}_v$ , which is a recursive process of finding projection directions for two matrices that their correlation is maximized. Specifically, let  $Q_u, Q_v$  denote the orthonormal bases of  $\mathbf{Z}_u, \mathbf{Z}_v$ , the CCA can be denoted as,
$$
\mathcal {S} _ {C C A} \left(\mathcal {F} _ {u}, \mathcal {F} _ {v}\right) = \sqrt {\frac {1}{c} \sum_ {l = 1} ^ {c} \sigma_ {l} ^ {2}}, \tag {1}
$$
where  $\sigma_{l}$  denotes the  $l$ -th eigenvalue of  $\Lambda_{u,v} = Q_u^{\mathrm{T}}Q_v$
CKA. [21] proposes another way to assess the NN similarity based on Centered Kernel Alignment (CKA). Let  $K_{u} = \mathbf{Z}_{u}\mathbf{Z}_{u}^{\intercal}$ ,  $K_{v} = \mathbf{Z}_{v}\mathbf{Z}_{v}^{\intercal}$  denote the Gram matrices of two feature space, the CKA is computed by,
$$
\mathcal {S} _ {C K A} \left(\mathcal {F} _ {u}, \mathcal {F} _ {v}\right) = \frac {\operatorname {H S I C} \left(K _ {u} , K _ {v}\right)}{\sqrt {\operatorname {H S I C} \left(K _ {u} , K _ {v}\right) \operatorname {H S I C} \left(K _ {u} , K _ {v}\right)}}, \tag {2}
$$
where HSIC is the Hilbert-Schmidt Independence Criterion [11].
However, in addition to the forward pass, all the aforementioned approaches further introduce significant computational costs while performing evaluation in the representation space. Nevertheless,
(b)
Figure 2: (a) Correlation between Grassmann similarity and filter subspace similarity; (b) Correlation between CCA and filter subspace similarity. (Table) Correlation between filter subspace similarity and other approaches.
Correlation
CCA [43]	0.9327
CKA [21]	0.9550
Grassmann Distance [35]	0.9469
their qualities rely heavily on the mindful choice of probing data  $\mathbf{X}_p$ , which undermines their robustness.
2.2 Representational Similarity in Filter Subspace
Filter subspace. As in [41], the convolutional filter  $\mathbf{W} \in \mathbb{R}^{c' \times c \times k \times k}$  ( $c'$  and  $c$  are the number of input and output channels,  $k$  is the kernel size) can be decomposed over  $m$  filter atoms (filter subspace elements)  $\mathbf{D}[i] \in \mathbb{R}^{k \times k}(i = 1, \dots, m)$ , linearly combined by atom coefficients  $\alpha \in \mathbb{R}^{m \times c' \times c}$  as  $\mathbf{W} = \alpha \times \mathbf{D}$ . Note that each convolutional layer now becomes two convolutional layers, a filter atom layer followed by an atom coefficient layer with  $1 \times 1$  filters. The filter subspace is then expressed as  $\mathcal{V} = \operatorname{Span}\{\mathbf{D}[1], \dots, \mathbf{D}[m]\}$ . With this formulation, we consider a paradigm where filter subspaces are model-specific, and subspace linear combination rules, i.e., atom coefficients, are shared across different networks. The intuition and detailed validation of this learning paradigm can be found in [35], where state-of-the-art performance in the continual learning context is reported.
In this setting, we dive deep into the relationship between filter subspaces and representations. For simplicity, let  $c = c' = 1$ , and the argument extends. Given an input image  $\mathbf{X}(b)$  ( $b \in \mathcal{B}, \mathcal{B} \subset \mathbb{Z}^2$ ), define the local input norm  $\| \mathbf{X} \|_{F,N_b} \coloneqq (\sum_{b' \in N_b} \mathbf{X}(b - b')^2)^{1/2}$  and the convolution  $\langle \mathbf{X}, w \rangle_{N_b} \coloneqq \sum_{b' \in N_b} \mathbf{X}(b - b') w(b')$ , where  $N_b \subset \mathcal{B}$  is a local Euclidean grid centered at  $b$ . Then the decomposed convolution can be written as  $\mathbf{Z}(b) = \sum_{i=1}^{m} \alpha_i \langle \mathbf{X}, \mathbf{D}_i \rangle_{N_b}$ , where  $\mathbf{D}[i]$  denotes the  $i$ -th atom,  $\alpha_i$  is the corresponded  $i$ -th coefficient.
Proposition 2.1. Suppose  $\mathbf{D}_u$  and  $\mathbf{D}_v$  are two different sets of filter atoms for a convolutional layer with the common atom coefficients  $\alpha$ , we can upper bound the changes in the corresponding features  $\mathbf{Z}_u, \mathbf{Z}_v$  with atom changes,
$$
\left| \left| \mathbf {Z} _ {u} - \mathbf {Z} _ {v} \right| \right| _ {F} \leq \left(\left| \left| \boldsymbol {\alpha} \right| \right| _ {F} \lambda\right) \sqrt {| \mathcal {B} |} \cdot \left| \left| \mathbf {D} _ {u} - \mathbf {D} _ {v} \right| \right| _ {F}, \quad w i t h \lambda = \sup  _ {b \in \mathcal {B}} \left| \left| \mathbf {X} \right| \right| _ {F, N _ {b}}. \tag {3}
$$
The proof is provided in Appendix A.1. We further empirically validate this relationship in Section A.3.
Filter subspace similarity The above theorem suggests the possibility to measure the representational similarity of two NNs by simply measuring the distance of their filter subspaces. As proposed in [35], the representational similarity of two NNs with different filter subspaces  $\mathcal{V}_u, \mathcal{V}_v$  can be assessed by the similarity based on Grassmann distance between  $\mathcal{V}_u, \mathcal{V}_v$  as,
$$
\mathcal {S} _ {\text {G r a s}} \left(\mathcal {F} _ {u}, \mathcal {F} _ {v}\right) = d \left(\mathcal {V} _ {u}, \mathcal {V} _ {v}\right) = \frac {1}{m} \sum_ {i} \cos \theta_ {i}, \tag {4}
$$
where  $\theta_{i}$  is the  $i$ -th principal angle between  $\nu_{u}$  and  $\nu_{v}$ .
However, the above metric requires costly singular value decomposition. Note that filter atoms in different NNs are intrinsically aligned under shared atom coefficients, which allows us to approximate the filter subspace similarity using the cosine similarity of the corresponding filter atoms. To this end, as shown in Figure 1(b), we propose a significantly simplified representational similarity measure with filter atom similarity.
Definition 2.2. Suppose two convolution neural networks  $\mathcal{F}_u, \mathcal{F}_v$  share atom coefficients layer-wise, and their model-specific filter atoms are  $\mathbf{D}_u, \mathbf{D}_v$ , then the filter subspace representational similarity is simplified as,
$$
\mathcal {S} _ {\text {A t o m}} \left(\mathcal {F} _ {u}, \mathcal {F} _ {v}\right) = \cos \left(\mathbf {D} _ {u}, \mathbf {D} _ {v}\right) = \frac {<   v e c \left(\mathbf {D} _ {u}\right) , v e c \left(\mathbf {D} _ {v}\right) >}{\left| \left| v e c \left(\mathbf {D} _ {u}\right) \right| \right| \cdot \left| \left| v e c \left(\mathbf {D} _ {v}\right) \right| \right|}. \tag {5}
$$
The above definition is a layer-wise similarity, allowing us to compare the similarity of different networks per layer, and we simply average layer-wise similarities for the network-wise similarity.
Remark 2.3. The filter subspace similarity measure becomes a proper metric after taking the arccosine, i.e.,  $\operatorname{arccos}(\mathcal{S}_{\text{Atom}}(\mathcal{F}_u, \mathcal{F}_v))$  is a proper metric.
We further show that  $S_{Atom}$  and  $S_{Gras}$  are equivalent under certain assumption.
Proposition 2.4. Assume  $\mathbf{D}_u, \mathbf{D}_v \in \mathbb{R}^{k^2 \times m}$  are orthogonal matrices, then  $S_{\text{Gras}} = S_{\text{Atom}}$ .
The proof is provided in Appendix A.1. We empirically show in Figure 2(a) that the above simplified filter subspace similarity has still a strong linear correlation with the Grassmann subspace similarity even without imposing the above orthogonality over atoms.
Note that our filter subspace similarity measure only involves linear operations of vectorized atoms of around hundreds of dimensions, which requires negligible computation. Additionally, the proposed method depends solely on models themselves and eliminates the reliance on external probing data, equipping our similarity with robustness to inappropriate choice of probing data.
2.3 Algorithm Complexity Analysis
Here, we provide a detailed comparison of computation complexity between the proposed filter subspace similarity and probing-based similarities. Consider one convolutional layer with filter  $\mathbf{W} \in \mathbb{R}^{c' \times c \times k \times k}$  ( $\mathbf{W} = \boldsymbol{\alpha} \times \mathbf{D}$ ,  $\mathbf{D} \in \mathbb{R}^{m \times k \times k}$ ) which transforms the input  $\mathbf{X}_p \in \mathbb{R}^{n \times c' \times h' \times w'}$  to output  $\mathbf{Z} \in \mathbb{R}^{n \times c \times h \times w}$ . The complexity of our method is dominated by inner product of two tiny filter atoms,  $\mathcal{O}(m \cdot k^2)$ , e.g.,  $m = 9$ ,  $k = 3$  in a typical setting.
In contrast, probing-based similarity measure first forward feeds  $n$  probing samples with a complexity of  $\mathcal{O}(n \cdot h'w' \cdot k^2 \cdot cc')$ , then calculates covariance matrix with the complexity of  $\mathcal{O}(n^2 \cdot hw \cdot c)$ . In total, the time complexity of CCA is  $\mathcal{O}(n \cdot h'w' \cdot k^2 \cdot cc' + n^2 \cdot hw \cdot c)$ . Our method is at least  $\frac{n \cdot h'w' \cdot k^2 \cdot cc' + n^2 \cdot hw \cdot c}{m \cdot k^2}$  times more efficient than probing-based similarity measures. As  $h \gg k$ ,  $cc' \gg m$ , the computational cost of our method is negligible. For example, with 10k probing datapoints, the CCA calculation requires  $1.14 \times 10^7$  times more FLOPs than the proposed method.
2.4 Relationship with Probing-based Similarities
The proposed filter subspace similarity not only shows extreme efficiency but also exhibits a strong linear relationship with other popular probing-based similarities. Here, we analyze the proposed filter subspace similarity  $S_{Atom}$  with CCA,  $S_{CCA}$  [43]. Suppose forward passes of decomposed convolutional layer for  $\mathcal{F}_u$  and  $\mathcal{F}_v$  are  $\mathbf{Z}_u = \alpha \mathbf{X}_p\mathbf{D}_u$ ,  $\mathbf{Z}_v = \alpha \mathbf{X}_p\mathbf{D}_v$ , respectively. To start with, we show that the  $S_{CCA}$  is upper bounded by the proposed  $S_{Atom}$ .
Theorem 2.5. Let  $\mathcal{T} = \operatorname{Tr}(\mathbf{X}_p^\intercal \boldsymbol{\alpha}^\intercal \boldsymbol{\alpha}\mathbf{X}_p)$ ,  $\mathcal{C} = \sigma_{min}(\mathbf{X}_p^\intercal \boldsymbol{\alpha}^\intercal \boldsymbol{\alpha}\mathbf{X}_p)$ . Assume  $\mathcal{K}(\mathbf{Z}_u^\intercal\mathbf{Z}_u)$ ,  $\mathcal{K}(\mathbf{Z}_v^\intercal\mathbf{Z}_v) \leq \gamma$ . Then  $\mathcal{S}_{CCA}(\mathcal{F}_u, \mathcal{F}_v)$  is upper bounded by  $\mathcal{S}_{Atom}(\mathcal{F}_u, \mathcal{F}_v)$ ,
$$
\frac {\mathcal {C}}{\gamma c ^ {\frac {3}{2}} \mathcal {T}} \cdot \mathcal {S} _ {C C A} \left(\mathcal {F} _ {u}, \mathcal {F} _ {v}\right) \leq \mathcal {S} _ {A t o m} \left(\mathcal {F} _ {u}, \mathcal {F} _ {v}\right), \tag {6}
$$
where  $\mathbf{Tr}(\cdot)$  denotes trace of a matrix,  $\sigma_{min}$  indicates the minimum eigenvalue,  $\mathcal{K}(A)$  denotes the condition number of matrix  $A$ . We provide the proof in Appendix A.1.
${}^{ \dagger  }$  Specifically,the formulation writes as  ${\mathbf{Z}}_{u} = \alpha {\mathbf{X}}_{p} \star  {\mathbf{D}}_{u}$  ,where  $\star$  is the convolutional operation. By converting convolutional kernel  ${\mathbf{D}}_{u}$  into a Toeplitz matrix,we can replace the convolution operation  ${\mathbf{X}}_{p} \star  {\mathbf{D}}_{u}$  with matrix multiplication  ${\mathbf{X}}_{p}{\mathbf{D}}_{u}$  . We also modify  $\alpha$  by  ${I}_{hw} \otimes  \alpha$  ,where  $\otimes$  is Kronecker product,to enable the matrix multiplication  $\alpha {\mathbf{X}}_{p}{\mathbf{D}}_{u}$  .
Figure 3: (a) The ratio of the computational cost savings of our filter subspace similarity over probing-based similarities. (b) The performance of probing-based similarities can be compromised by poorly selected probing data. For models trained on CIFAR-100, they have high CCA and CKA similarities with probing from CIFAR-100 but low similarities with probing from other datasets. In contrast, our filter subspace similarity does not rely on probing data and shows a high similarity between the networks, aligning with our expectations.
Since  $S_{CCA}$  is probing-dependent, the calculated value varies depending on the choice of probing data, and the value range shows bounded by our filter subspace similarity, as in the theorem above.
With additional assumptions imposed, we can further show a near-linear relationship between CCA and our filter subspace similarity.
Assumption 2.6. Suppose the diagonal elements of  $\mathbf{Z}_u^\top \mathbf{Z}_u$ ,  $\mathbf{Z}_u^\top \mathbf{Z}_v$  and  $\mathbf{Z}_v^\top \mathbf{Z}_v$  are larger than non-diagonal elements, i.e.,  $(\mathbf{Z}_u^\top \mathbf{Z}_u)_{ii} \gg (\mathbf{Z}_u^\top \mathbf{Z}_u)_{ij}$ .
The Assumption 2.6 suggests different channels of feature  $\mathbf{Z}$  have a low correlation. Reducing channel-wise dependencies has been studied in [63] and has been shown to benefit model stability. We provide the empirical verification of the assumption in Appendix A.3.
Theorem 2.7. If Assumption 2.6 holds,  $\mathcal{S}_{CCA}(\mathcal{F}_u,\mathcal{F}_v)$  is approximately linear to filter subspace similarity,
$$
\frac {\sqrt {c}}{\gamma_ {1} \gamma_ {2} \gamma_ {3}} \cdot \mathcal {S} _ {C C A} \left(\mathcal {F} _ {u}, \mathcal {F} _ {v}\right) = \mathcal {S} _ {A t o m} \left(\mathcal {F} _ {u}, \mathcal {F} _ {v}\right), \tag {7}
$$
where  $\gamma_{1},\gamma_{2}$  and  $\gamma_{3}$  contain higher order of features, which can be found in detail with the proof in Appendix A.1. Specifically, we have  $\gamma_{2} = \sqrt{1 - \frac{\Delta}{\gamma_{1}^{2}\gamma_{3}^{2}}\frac{1}{\cos^{2}(\mathbf{D}_{u},\mathbf{D}_{v})}}$  and since  $\Delta$  are small, with Taylor expansion,  $\gamma_{2}\approx 1 - \frac{1}{2}\frac{\Delta}{\gamma_{1}^{2}\gamma_{3}^{2}}\frac{1}{\cos^{2}(\mathbf{D}_{u},\mathbf{D}_{v})}$ . The term  $\frac{1}{\cos^2(\mathbf{D}_u,\mathbf{D}_v)}$  causes non-linearity in the relation between CCA and filter subspace similarity.
As in Figure 2, we empirically observe the linear correlation between CCA and filter subspace similarity, which agrees with our theoretical findings. In addition, we find that the proposed similarity also shows a strong correlation with CKA.
In this section, we first validate our theorems with several validation experiments and then demonstrate simple example applications of the proposed filter subspace similarity in efficiently analyzing training dynamics as well as in federated and continual learning scenarios.