SlowGuess/NeurIPS_2023_OCR huggingface dataset:
<sup>8</sup>We also conduct experiments on CNNs. For a Conv layer, the forward propagation will be denoted as  $\mathbf{W} \mathbf{H}$  similar to a linear layer.
Figure 14: Comparison between  $\mathbb{E}_{\mathcal{D}}[1 - \mathrm{cosine}_{\alpha}(\pmb{x}_i)]$  and  $\mathbb{E}_{\mathcal{D}}[1 - \mathrm{cosine}_{A,B}(\pmb{x}_i)]$  and demonstration of  $\mathbb{E}_{\mathcal{D}}[1 - \mathrm{coef}_{\alpha}(\pmb{x}_i)]$ . The Straight-Through Estimator (STE) [1] are used to obtain two linearly connected modes  $\theta_{A}$  and  $\theta_{B}$ . Results are presented for different layers of MLP on both MNIST and CIFAR-10 dataset, with  $\alpha \in \{0.25, 0.5, 0.75\}$ . Standard deviations across the dataset are reported by error bars.
Figure 15: Comparison between  $\mathbb{E}_{\mathcal{D}}[1 - \mathrm{cosine}_{\alpha}(\pmb{x}_i)]$  and  $\mathbb{E}_{\mathcal{D}}[1 - \mathrm{cosine}_{A,B}(\pmb{x}_i)]$  and demonstration of  $\mathbb{E}_{\mathcal{D}}[1 - \mathrm{coef}_{\alpha}(\pmb{x}_i)]$ . The weight matching is used to obtain two linearly connected modes  $\theta_{A}$  and  $\theta_{B}$ . Results are presented for different layers of ResNet-20 (32x) on the CIFAR-10 dataset, with  $\alpha \in \{0.25, 0.5, 0.75\}$ . Standard deviations across the dataset are reported by error bars.
Figure 16: Comparison of  $\mathrm{Dist}_{\mathrm{com}}$ ,  $\mathrm{Dist}_W$ , and  $\mathrm{Dist}_H$ . The spawning method is used to obtain two modes that satisfy LLFC,  $\theta_A$  and  $\theta_B$ . The results are presented for different layers of VGG-16 on the CIFAR-10 dataset.
Figure 17: Comparison of  $\mathrm{Dist}_{\mathrm{com}}$ ,  $\mathrm{Dist}_W$ , and  $\mathrm{Dist}_H$ . The activation matching is used to obtain two modes that satisfy LLFC,  $\theta_A$  and  $\theta_B$ . The results are presented for different layers of MLP on the MNIST dataset.
Figure 18: Comparison of  $\mathrm{Dist}_{\mathrm{com}}$ ,  $\mathrm{Dist}_W$ , and  $\mathrm{Dist}_H$ . Both the activation matching and weight matching are used to obtain two modes that satisfy LLFC,  $\theta_A$  and  $\theta_B$ . The results are presented for different layers of MLP on the CIFAR10 dataset.
Figure 19: Comparison between  $\mathrm{Dist}_{\mathrm{com},LMC}$  and  $\mathrm{Dist}_{\mathrm{com,not}LMC}$ . Both the spawning and permutation methods are used to obtain two linearly connected modes.
Layer l	FC 1	FC 2	FC 3
ErrD(B>εA≤l)	2.69	2.11	1.92
Table 1: Error rates  $(\%)$  of stitched MLP on the MNIST test set. The model stitching is employed in different layers. The spawning method is used to obtain two neural networks that satisfy LLFC, i.e.,  $A$  and  $B$ . Error rates  $(\%)$  of  $A$  and  $B$  are 1.9 and 1.77, respectively.
Notably, the experiments are not conducted on the first Conv/Linear layer of the model because the commutativity condition is naturally satisfied for the first layer where  $H_A^{(0)} = H_B^{(0)} = X$  where  $X$  is the input data matrix.
B.4 Experiments on Model Stitching
Model stitching [19, 3] is commonly employed to analyze neural networks' internal representations. Let  $A$  and  $B$  represent neural networks with identical architectures. Given a loss function  $\mathcal{L}$ , model stitching involves finding a stitching layer  $s$  (e.g., a linear  $1 \times 1$  convolutional layer) such that the minimization of  $\mathcal{L}(B_{> \ell} \circ s \circ A_{\leq \ell})$  is achieved. Here,  $B_{> \ell}$  denotes the mapping from the activations of the  $\ell$ -th layer of network  $B$  to the final output,  $A_{\leq \ell}$  denotes the mapping from the input to the activations of the  $\ell$ -th layer of network  $A$ , and  $\circ$  represents function composition.
In this section, we explore a stronger form of model stitching. Specifically, given two neural networks  $A$  and  $B$  that satisfy LLFC, we evaluate the arruacy of  $B_{> \ell} \circ A_{\leq \ell}$  over the test set  $\mathcal{D}$  without finding a stitching layer, i.e.,  $\mathrm{Err}_{\mathcal{D}(B_{> \ell} \circ A_{< \ell})}$ . As shown in Tables 1 to 3, we include experimental results for MLP on the MNIST dataset, VGG-16 on CIFAR-10 the dataset and ResNet-20 on the CIFAR-10 dataset. Only the spawning method is utilized to find modes that satisfy LLFC. The results depicted
Layer ℓ	Conv 1-1	Conv 1-2	Conv 2-1	Conv 2-2	Conv 3-1
ErrD(B>ℓ○A≤ℓ)	7.2	8.43	8.39	9.91	11.84
Layer ℓ	Conv 3-2	Conv 3-3	Conv 4-1	Conv 4-2	Conv 4-3
ErrD(B>ℓ○A≤ℓ)	9.55	8.22	7.61	6.99	7.05
Layer ℓ	Conv 5-1	Conv 5-2	Conv 5-3	FC 1	FC 2
ErrD(B>ℓ○A≤ℓ)	6.91	6.88	6.88	7.07	6.92
Table 2: Error rates (\%) of stitched VGG-16 on the CIFAR-10 test set. The model stitching is employed in different layers. The spawning method is used to obtain two neural networks that satisfy LLFC, i.e.,  $A$  and  $B$ . Error rates (\%) of  $A$  and  $B$  are 6.87 and 7.1, respectively.
Layer ℓ	Block 1-1	Block 1-2	Block 1-3	Block 2-1	Block 2-2	Block 2-3	Block 3-1	Block 3-2	Block 3-3
ErrD(B>ℓ∘A≤ℓ)	10.88	10.57	13.35	10.64	10.74	10.55	12.27	11.8	8.99
Table 3: Error rates (\%) of stitched ResNet-20 on the CIFAR-10 test set. The model stitching is employed in different layers. The spawning method is used to obtain two neural networks that satisfy LLFC, i.e.,  $A$  and  $B$ . Error rates (\%) of  $A$  and  $B$  are 8.69 and 8.58, respectively.
Figure 20: Singular values of weight matrix  $W^{(\ell)}$  of  $\ell$ -th layer of  $\theta$  in a descending order. Here,  $\theta$  can be used to achieve LMC with weight matching. The results are presented for different layers of various model architectures and datasets.
in Tables 1 to 3 demonstrate that the error rates of the stitched model on the test set closely resemble the error rates of the original models  $A$  and  $B$ , regardless of the dataset or model architecture. This observation suggests that models that satisfy LLFC encode similar information, which can be decoded across different models. Subsequently, the experiments of model stitching provides new insights towards the commutativity property, i.e.,  $\forall \ell \in [L]$ ,  $W_B^{(\ell)}H_A^{(\ell -1)}\approx W_B^{(\ell)}H_B^{(\ell -1)}$ .
B.5 Discussion on Git Re-basin [1]
In this section, we investigate the ability of permutation methods to achieve LMC. While we have interpreted the activation matching and weight matching methods proposed by Ainsworth et al. [1] as guaranteeing the commutativity property, we have yet to address why permutation methods can ensure the satisfaction of this property. Thus, in order to delve into the capability of permutation methods, we must address the question of why these methods are capable of ensuring the satisfaction of the commutativity property.
Low-rank model weights and activations contribute to ensure the commutativity property. We now consider a stronger form of the commutativity property, where given two modes  $\theta_{A}$  and  $\theta_{B}$  and a
MLP on CIFAR10
FC1
Figure 21: Singular values of post-activations  $H^{(\ell)}$  of  $\ell$ -th layer of  $\theta$  over the whole test set  $\mathcal{D}$  in a descending order. Here,  $\theta$  can be used to achieve LMC with activation matching. The results are presented for different layers of MLP on the CIFAR-10 dataset.
FC3
ResNet20 (32 x) on CIFAR10
Figure 22: Comparison between the stable rank  $\operatorname{srank}(\mathbf{W}_T^{(\ell)})$  and  $\operatorname{srank}(\mathbf{W}_1^{(\ell)})$ . Here,  $\mathbf{W}_T^{(\ell)}$  denotes the weight matrix of the  $\ell$ -th layer of the model  $\theta_T$  in the terminal phase of training. Similarly,  $\mathbf{W}_1^{(\ell)}$  denotes the weight matrix of the  $\ell$ -th layer of the model  $\theta_1$  in the early stage of training (1 epoch indeed). Also, the stable rank can be calculated as  $\operatorname{srank}(\mathbf{W}) = \frac{\|\mathbf{W}\|_F^2}{\|\mathbf{W}\|_2^2}$ . The results are presented for different layers of ResNet-20 (32x) on the CIFAR-10 dataset.
dataset  $\mathcal{D}$  , we have:
$$
\forall \ell \in [ L ], \boldsymbol {W} _ {A} ^ {(\ell)} \boldsymbol {H} _ {A} ^ {(\ell - 1)} = \boldsymbol {W} _ {A} ^ {(\ell)} \boldsymbol {H} _ {B} ^ {(\ell - 1)} \wedge \boldsymbol {W} _ {B} ^ {(\ell)} \boldsymbol {H} _ {B} ^ {(\ell - 1)} = \boldsymbol {W} _ {B} ^ {(\ell)} \boldsymbol {H} _ {A} ^ {(\ell - 1)}.
$$
Thus, to satisfy the commutativity property for a given layer  $\ell$ , we can employ the permutation method to find a permutation matrix  $P^{(\ell -1)}$  such that:
$$
\boldsymbol {W} _ {A} ^ {(\ell)} \left(\boldsymbol {H} _ {A} ^ {(\ell - 1)} - \boldsymbol {P} ^ {(\ell - 1)} \boldsymbol {H} _ {B} ^ {(\ell - 1)}\right) = 0 \wedge \boldsymbol {P} ^ {(\ell)} \boldsymbol {W} _ {B} ^ {(\ell)} \left(\boldsymbol {H} _ {B} ^ {(\ell - 1)} - \boldsymbol {P} ^ {(\ell - 1) ^ {\top}} \boldsymbol {H} _ {A} ^ {(\ell - 1)}\right) = 0.
$$
In a homogeneous linear system  $\mathbf{W} \mathbf{X} = 0$ , a low-rank matrix  $\mathbf{W}$  allows for a larger solution space for  $\mathbf{X}$ . Therefore, if the ranks of  $\mathbf{W}_A^{(\ell)}$  and  $\mathbf{W}_B^{(\ell)}$  are low, it becomes easier to find a permutation matrix  $\mathbf{P}^{(\ell -1)}$  that satisfies the commutativity property. Similarly, if we consider another form of commutativity property:
$$
\forall \ell \in [ L ], \boldsymbol {W} _ {A} ^ {(\ell)} \boldsymbol {H} _ {A} ^ {(\ell - 1)} = \boldsymbol {W} _ {B} ^ {(\ell)} \boldsymbol {H} _ {A} ^ {(\ell - 1)} \wedge \boldsymbol {W} _ {B} ^ {(\ell)} \boldsymbol {H} _ {B} ^ {(\ell - 1)} = \boldsymbol {W} _ {A} ^ {(\ell)} \boldsymbol {H} _ {B} ^ {(\ell - 1)}.
$$
Then, to ensure the commutativity property, we need to find  $P^{(\ell - 1)}$  and  $P^{(\ell)}$  such that
$$
\left(\boldsymbol {W} _ {A} ^ {(\ell)} - \boldsymbol {P} ^ {(\ell)} \boldsymbol {W} _ {B} ^ {(\ell)} \boldsymbol {P} ^ {(\ell - 1) ^ {\top}}\right) \boldsymbol {H} _ {A} ^ {(\ell - 1)} = 0 \wedge \left(\boldsymbol {P} ^ {(\ell)} \boldsymbol {W} _ {B} ^ {(\ell)} \boldsymbol {P} ^ {(\ell - 1) ^ {\top}} - \boldsymbol {W} _ {A} ^ {(\ell)}\right) \boldsymbol {P} ^ {(\ell - 1)} \boldsymbol {H} _ {B} ^ {(\ell - 1)} = 0.
$$
Then, if the ranks of  $\pmb{H}_A^{(\ell -1)}$  and  $\pmb{H}_B^{(\ell -1)}$  are low, it is easier to find the permutation matrices to satisfy the condition. In real scenarios, both model weights (see Figure 20) and activations (see Figure 21) are approximately low-rank, which helps the permutation methods satisfy the commutativity property.
Additionally, Ainsworth et al. [1] mentioned two instances where permutation methods can fail: models with insufficient width and models in the early stages of training. In both cases, the model weights often fail to satisfy the low-rank model weight condition. In the first scenario, when the model lacks sufficient width, meaning that the dimension of the weight matrix approaches the rank of the weight matrix, the low-rank condition may not be met. For example, compared the singular values of ResNet-20 (32x) (see Figure 20) with singular values of ResNet-20 (1x) (see ??), it is evident that in the wider architecture, the proportion of salient singular values is smaller. In the second scenario, during the initial stages of training, the weight matrices resemble random matrices and may not
exhibit low-rank characteristics. For example, as shown in Figure 22, the stable ranks of weight matrices of the model after convergence are significantly smaller than those of the model in the early stage of training. Consequently, permutation methods may struggle to find suitable permutations that fulfill the commutativity property, resulting in the inability to obtain modes that satisfy LMC.