Going Beyond Neural Network Feature Similarity: The Network Feature Complexity and Its Interpretation Using Category Theory
Authors: Yiting Chen
Zhanpeng Zhou
Junchi Yan
Venue: ICLR 2024 poster (2024)
Keywords: Category Theory; Neural Networks Feature Complexity
Primary area: visualization or interpretation of learned representations

Abstract: The behavior of neural networks still remains opaque, and a recently widely noted phenomenon is that networks often achieve similar performance when initialized with different random parameters. This phenomenon has attracted significant attention in measuring the similarity between features learned by distinct networks. However, feature similarity could be vague in describing the same feature since equivalent features hardly exist. In this paper, we expand the concept of equivalent feature and provide the definition of what we call *functionally equivalent features*. These features produce equivalent output under certain transformations. 
Using this definition, we aim to derive a more intrinsic metric for the so-called *feature complexity* regarding the redundancy of features learned by a neural network at each layer. We offer a formal interpretation of our approach through the lens of category theory, a well-developed area in mathematics. To quantify the feature complexity, we further propose an efficient algorithm named Iterative Feature Merging. Our experimental results validate our ideas and theories from various perspectives. We empirically demonstrate that the functionally equivalence widely exists among different features learned by the same neural network and we could reduce the number of parameters of the network without affecting the performance. We have also drawn several interesting empirical findings, including: 
1) the larger the network, the more redundant features it learns; 2) in particular, we show how to prune the networks based on our finding using direct equivalent feature merging, without fine-tuning which is often needed in peer network pruning methods; 3) same structured networks with higher feature complexity achieve better performance; 4) through the layers of a neural network, the feature complexity first increase then decrease; 5) for the image classification task, a group of functionally equivalent features may correspond to a specific semantic meaning. Source code will be made publicly available.

Published as a conference paper at ICLR 2024
GOING BEYOND NEURAL NETWORK FEATURE SIMI- LARITY: THE NETWORK FEATURE COMPLEXITY AND ITS INTERPRETATION USING CATEGORY THEORY
Yiting Chen, Zhanpeng Zhou, Junchi Yan"
Department of Computer Science and Engineering, Shanghai Jiao Tong University {sjtucyt, zzp1012, yanjucahi | jesjtu.ee edu. on
https: //gi thub. wwm./o////////// (8 /ettettttii veetiiie--eetrree Mergi
ABSTRACT
The behavior of neural networks remains opaque, and a recently widely noted phenomenon is that networks often achieve similar performance when initialized with different random parameters. This phenomenon has attracted significant at- tention in measuring the similarity between features learned by distinct networks. However, feature similarity could be vague in describing the same feature while similar features could yield different outputs. In this paper, we expand the con- cept of equivalent feature and provide the definition of what we call functionally equivalent features. These features produce equivalent output under certain trans- formations. Using this definition, we aim to derive a more intrinsic metric for the layer-wise feature complexity regarding the redundancy of features learned by a neural network at each layer. We offer formal interpretation of our approach through the lens of category theory, a well-developed area in mathematics. To quantify the feature complexity, we further propose an efficient algorithm named Iterative Feature Merging (IFM). Our experimental results validate our ideas and theories from various perspectives. We empirically show that the functionally equivalence widely exists among different features learned by the same network and could reduce network size almost without affecting the performance.
INTRODUCTION
Deep neural networks (DNNs) have achieved significant success across diverse fields, including vision, texts and other areas. However, DNNs are often regarded as "black box" models associated with high dimensional feature maps and numerous parameters. Recently, many studies report an interesting phenomenon that neural networks with different random initialization often converge to solutions with similar performance Meanwhile, methods such as pruning (Wen et al., 2016; Ye et al., 2018; Peng et al., 2019) and knowledge distillation (Hinton et al., 2015) have been proposed to reduce the complexity of the neural network structure, leading to a more compact DNN achieving similar performance to their dense/large counterparts. These phenomena further draw increasing studies on the features' learned by neural networks and popular treatment is measuring the so-called feature similarity². Various feature similarity measures (Li al., 2016; Morcos al., 2018) have been devised to quantify the distance between two features, and some works have further empirically found that similar features can be learned from either between different networks (Li et al., 2016; Wang et al., 2018) or within a (wide) network (Nguyen et al., 2021). In literature, feature similarity measures are often designed to be invariant to certain transformations under certain transformations e.g. permutation (Li et al., 2016), isotropic scaling (Barannikov al., 2022), invertible linear transformation (Wang et al.,
the test set (Dauphin et al., 2014; Frankle et al., 2020).
Correspondencce author. This work was part supported by NSFC (92370201, 62222607) and SITU Trans-med Awards Research (STAR) 20210106.
I 'More precisely speaking, the feature here is the layer-wise feature map corresponding to data point. Note that most previous works, egg. measuring feature similarity, are hased on a set features corresponding to a specific datasct, while our definition and method are data-agnostic and may he more fundamental. Thhe feature similarity also referred representational similarity.

---

Published as a conference paper at ICLR 2024
Functionally Eguivolent Foetume
Feature Complecity
Flowchart
Takee dleer losk 11.2
Identity: must
(8.
(dentdyittyy indicata medunbbngy in
Merge weights minimum
You
Mo
Mode Jo
Mode fo
For I and anx
The feature compledty 2f
18..1 (0...
Egou (8,_00
reoresentatoon 2?
Feature complexity
An illustration of our proposed functionally equivalent features and feature complexity. Two different models learn functionally equivalent features if the features and outputs of the two models are equivalent under certain invertible linear transformations. Feature complexity (layer- wise) is the dimensionality of the most compact representation of the feature at certain layer. In order to retrieve the most compact version, we propose iterative feature merging (IFM).
Figure I:
2018), affine transformation (Raghu et al., 2017; Williams et al., 2021) etc. However, previous work (Ding et al., 2021) has shown similar features could also yield different outputs. It shows that previous feature similarity measures often vague in exactly describing the inherent structure of the neural networks (e.g. the redundancy of features learned by a neural network or the same feature learned by different neural networks) and can hardly provide actionable insights.
In this paper, we expand the concept of feature equivalence and formally define a more general form, the so-called functional equivalence. Features considered functionally equivalent if they produce outputs that are equivalent under certain transformations. Based on the functionally equiv- alent features, one can derive metric to reflect the inherent layer-wise feature complexity of the network regarding the redundancy of features learned by neural networks. In another word, the feature complexity corresponds to the most compact representation of the feature at certain layer.
For clearer and more formal description of our approach, we provide an abstract view of neural networks and interpret our approach using the language of category theory, a branch of mathematics, which is widely applied in various fields, including mathematics, physics, and computer science. In general, a category a graph with objects and arrows (or "morphisms") between objects. The map- pings between categories are called functors that preserves the structure and the mappings between functors are called natural transformations (for a formal introduction, see Sec. 2.2). In the context of neural networks, we abstractly represent the network structure category and certain neural network as a functor that maps this structure to specific parameters. Through the lens of category theory, prove that functional equivalence between features be elegantly represented as the existence natural transformation between two functors defined above.
To empirically measure the feature complexity, we propose algorithm called Iterative Feature Merging (IFM) to merge the weights corresponding to redundant features. By iteratively matching the weights corresponding perimental results show that DNNs learns lot of functionally equivalent features which could be merged removed with little impact on the neural network performance. We also provide many valuable insights that may inspire future works. Our major contributions are as follows:
different features, we merge functionally equivalent features. Ex-
We provide definition for functionally equivalent features (in Sec. 2.3) and the layer-wise feature complexity (in Sec. 2.4). This differs largely from existing works dwelling on feature similarity.
We provide a category theory perspective on neural network structure and training. From the category theory perspective, we further offer insights into phenomena like linear mode connectivity.
We propose algorithm called Iterative Feature Merging (IFM) (in See. 3) measure the feature complexity. Experimental results show its efficiency and potential as an effective pruning method. Experiments with IFM also yield valuable insights.
2

---

Published as a conference paper at ICLR 2024
2 DEFINITION OF NEURAL NETWORK FEATURE COMPLEXITY
In this section, we define the feature complexity of a neural network from category theory perspec- tive. We first introduce an abstract view of neural networks using the language of category theory (in Sec. 2.2) and further define functionally equivalent feature from the category theory view (in Sec. 2.3). Finally, we provide the definition feature complexity in Sec. 2.4.
2.1 NOTATIONS
Consider an I--layr neural network -():)) where A is the parameter. We use f'(O',) to denote the 1-th layer. For the feature map (6, x), have (0, x) f'((', (0,x)) and Z (0, x) ('(O,x), When x is not of concern, we abbreviate Z (0) for Z x0.
To take closer look, we proceed with an L-layer MLP for its ease of presentation, despite the fact that our definition of feature complexity is architecture agnostic and our proposed method could be applied to arbitrary architectures. For a MLP. the feature map (0) ER contains di features. We use (0), E [1, di] to denote the i-the feature. The O' corresponds to a weight matrix W E Rd xd:-1 and a bias vector b' E Rd and we have (0) (W) bb),, where a corresponds the activation function. For permutation on 0. we have (0 Piz for each / { L). where P1 E X4xx is permutation matrix. The weight is also permuted as W P.W' Pi_1 and by P/b', which make sure that Vx, f(0,, x) xx).
2.2 CATEGORY THEORY VIEW OF NEURAL NETWORKS
We introduce basic and necessary concepts in category theory which is a branch of mathematics. For more comprehensive introduction please refer to (Mac Lane, 2013; Adámek et al., 1990).
Category: A category C is a graph with set of objects and arrows (or "morphisms"), For an arrow f E A from object to b, it is written as f a + b where a is the domain of f written as domf and b the codomain of f written as coof Identity: For each object there exists an arrow id,, + a. Composition: For each pair of arrows with dong- = codf. there is an arrow o f domʃ - cody called their composite.
b. category also has two additional operations:
< f.g g >
For the set of arrows from b to in category C, it is written as H Home(a, b) called "hom-set".
Functor: functor is the morphism between categories. For two category and B, a functor T : C - B consists two suitably related functions: the object function, which assigns each object e of C' an object Te of and the arrow function which assigns each arrow f : c + d' C' an arrow Tf : Te Té' of B, in such way that:
T(ide) = idre: T(g o f) Tgo
It means the mappings defined by the functor T preserves the structure of the category C.
Natural transformation: Given two functor S, T C - B, a natrual transformation : S T a function assigns each object e of C' an arrow Te : Sc - Tc of B, in such way that every arrow f c of C' yields commutative graph:
Te Te
Sc
Sf Sc'
fTf
Tcl
Tel
(2)
Here commutative graph means that different paths between two objects yield, by composition, an equal arrow between the two objects, such that I'f (Te(Sc)) = To (Sf(Sc)).
For neural network structure .(. it could be abstracted into category FF where objects are the shape of feature (feature map) and arrows are the type of transformation applied to the features, e.g. linear transformation, convolution, attention etc.. For an identity arrow, it simply applies no transformation on the feature while the composite of arrows is the composite of the corresponding transformations. Take a simple L-layer neural network structure for instance, it could be abstracted as the category depicted in Eq. 3, where each object corresponds to the shape of the input, feature maps and output while each arrow corresponds to the type of transformation applied the feature (we omit identity circle and composites of arrows for simplicity).
1. z1 2
14
f(;
(3)
3

---

Published as a conference paper at ICLR 2024
Terms in category theory
Corresponding specific concept in our definition
Category
Neural network structure.
Shape of feature maps including input and output
Object
(e.g. tensor of shape x 224 x 224).
Type of layers (transformations applied on the feature maps)
Arrow (Morphism)
(e.g. linear layer, convolution layer, etc.)
Parameterization of neural network
Functor
Mapping each arrow (type of layer) to parameterized layer.
There exist natural transformation iff
Natural transformation
two neural network learn Functionally Equivalent Features.
Table I: Terms in category theory and their embodiment in our definitions about neural networks.
In the context of neural network structure, the training process aims to discover suitable parameters. In other words, it seeks to find a functor T' FF + P, where P represents a category describing the shapes of features, and its arrows encompass all possible transformations with specific parameters. For example, consider a network -().-) with parameter 6, the corresponding functor To maps each arrow ('(-.-) to f(O', with specific parameters 0'_ To paint a clearer picture, as an analogy, we could think of the category F as class in object-oriented programming language and think of the functor as creating an object instantiating the corresponding class. In Table 1, we list the terms in category theory that used in our definition and the corresponding concepts about neural networks.
2.3 DEFINITION OF FUNCTIONALLY EQUIVALENT FEATURE FROM THE CATEGORY VIEW
Through the lens of category theory, as introduced in Sec. 2.2, we further define functionally equiv- alent feature using natural transformation. Consider a natural isomorphism T between two functors To.. To, F - P satisfying that for each object E F, the transform invertible linear transformation. Natural transformations require naturality, ensuring that each arrow in category F results in commutative graph in category P, as described in Eq. 2. Therefore, natural isomorphisms between parameters are non-trivial.
Too - To, is an
Definition 2.1. [Functionally Equivalent Features] For a L-layer neural network f(::-) and two different parameter On and 0z, if there is a natural isomorphism between functor Te, and To, then model / (00,,) -) and model f(0p.)) have functionally equivalent features such that
Vx E D, E [1, L-
1], -1,
T2441 (Z
= f (0%+1, (Z,,
(4)
where Tz--1 and Tzz represents the invertible linear transformation defined by the natural isomor- phism and 7'(0,, x) E R.d is the feature at the 1-th layer.
According to Definition 2. natural isomorphisms between networks indicate the features modeled by two networks are functionally equivalent i.e. we could replace any feature maps of one model with the feature map from the other model through invertible linear transformation. Since it is a strong condition, simple transformations between different networks are unlikely to exist.
Specifically, we notice linear mode connectivity (LMC) (Frankle al., 2020), an empirical phe- nomenon that have drawn extensive attention. It says that there may exist a linear path between two different neural networks such that along the path, the loss is nearly constant. Various methods (En- tezari et al., 2022; Ainsworth et al., 2023; Liu et al., 2022) have been proposed to find networks satisfying LMC while recently a stronger notion of linear connectivity called Layerwise Linear Fea- ture Connectivity (LLFC) was observed coexist with LMC (Zhou et al., 2023). We show that LMC actually indicates the functional equivalence between features.
Proposition 2.2. [LMC indicates functionally equivalent features] (Proof in Appendix C). For two different parameter On and Oz. if there is a permutation such that 0n, and T (0sa) satisfy LMC, then there exists a natural isomorphism T between functor To, and Te.-
Proposition 2.2 provides an interpretation of linear mode connectivity from category theory per- spective such that there always exists natural isomorphism between two networks satisfying LMC, which indicates the two networks have functionally equivalent features. Note that Entezari et al. (2022); Ainsworth et al. (2023); Liu et al. (2022) showed that difffrrntt networks can be linearly connected after permutation, therefore we assume each Tu to be permutation in the following, which means establishing one one correspondence between the features of the two networks.
4

---

Published as a conference paper at ICLR 2024
Accuracy between the two models
150
0.8
200
0.6
60
300
Tain, random permute. random
400
500
0.0
ModelA
Model A Parmused
100
200
300
403
500
Offorent features
(a) Weight matching distance between different (b) Testing accuracy CIFARIO of model interpolating between VGGI6 and same model after permutation.
features (normalized).
The empirical evidences that functionally equivalent features exist. (a): The heatmap of the distances between different features from the last convolution layer of VGG16 on CIFARIO. (b):The test accuracy on CIFARIO of model interpolating between VGG16 and the same model after permutation on each layer. The solid lines are the result of permutation on functionally equivalent features while the dashed lines are the result of random permutation the number of features.
Figure 2:
2.4 DEFINITION OF FEATURE COMPLEXITY FROM THE EQUIVALENT FEATURE
Based on the definition of functionally equivalent feature, we could further define the feature com- plexity. Consider natural transformations mapping the functor Ta to itself. The natural transforma- tion that maps each arrow to itself must exist. In the case of other natural transformations, they map 7 (0) to 7. (0) where P E Rdd x is non-identity permutation matrix, which means mapping feature (0) to Z3 (0) with # j. i E [1, dij. This approach allows to establish a partial order among features. For two features (0). (0). E [1, di] I-th layer of network f(0.-) exists a simple transformation between To to itself that maps ((( to (f). we say ( I (0). It indicates that feature (0) covers the feature represented in (0) and could be used to replace Z (0). It obvious that if ()) and ()) is comparable then ( = (0).
((0.-). ), if there
Theorem 2.3. [Feature duality] (Proof in Appendix C). For L-layer neural network f(0, .), there are multiple natural isomorphisms between To itself if and only if
31 E [1, Ji [1, s.t. Z = (0), i
(5)
Finally, we formally define feature complexity based on the partial ordering between features.
Definition 2.4. [Feature complexityl Given L-layer neural network (00,)) the feature at 1-th layer compose a poset (0) [1, di]}. The maximum number of features that are not equivalent i.e. the width of the corresponding poset is defined as the feature complexity at l-th layer.
MEASURING NEURAL NETWORK FEATURE COMPLEXITY
In this section, we propose an algorithm to empirically measure the complexity of the features by finding equivalent features with weight matching and merging the equivalent features. Firstly, we devise and introduce two components to find and merge equivalent features.
Feature weight matching: Drawing inspiration from the weight matching method used to identify permutations in linear mode connectivity literature, we propose matching the weights corresponding to each feature. Denote W [6.]] as the i-th row of Wi and W/ as the j-th column of W For two features 2 2% m. n E [1, di), the weight distance between the two features is defined
D mn = || W /r...l 112
|| - W
(6)
When matching the weights, we consider the layers such as linear fully connected layers or convo- lution layers the 1-th layer, while ignoring the activation and normalization layers in the network.
5

---

Published as a conference paper at ICLR 2024
10-
test accuracy (%5
test accuracy
60-
22-
VGG13 trature
91-
VOC15
VOC15 tratur
Reshetl2
W5011
Reshella afor festar
WEG13
Recker3:
VCG13 crp no accuraty VCG16
19-
Recket3: aroer frature merging
Rechet50
VCGL6 ori accuracy
Reshet50 ater teature merging
0.75 1,00
1.25
1.50
1.7
2:50
2.25
remaining params (%)
remaining params
107
(a) Testing accuracy to the percentage of remaining pa- (b) Testing accuracy to the number of remaining pa- rameters VGG networks
rameters VGG and ResNet
Figure 3: Results for iterative feature merging CIFARIO. (a): Testing accuracy to the percentage of remaining parameters. Each dot corresponds to different hyper-parameter 33 in iterative feature merging. (b): Testing accuracy the number of parameters. We conduct grid search on 3 and choose the largest / with testing accuracy larger than 95% of the testing accuracy before merging.
Feature merging: Given two functionally equivalent features Z Z", at the 1-th layer, where m, n [1.d]] To obtain the merged weight at the /-th layer W E (2( the row of weight matrix 4 corresponding to the merged feature is calculated by:
W Wergged = W [m..]] + W [n,]]
The other di 2 rows of W correspond the ds 2 rows in W such as W (... i m,
Therefore the merged feature at 1-th layer becomes Z" E Rd-- where the merged feature Z
z + z and the other di 2 features correspond to Z i * m,
To process the Z", we adjust the weight at / + 1-th layer. For W((+1) E R411xd-1, the column of (r(l-1) corresponding to the merged feature is:
W1(+) mean(WL+1 (WL+1 merged mean
[im]] WH+1
(8)
The other d, columns of the weight Wr(l+1) corresponds to the di - columns W [6]] * m, n.
Using this method, we could merge features to obtain an approximately functionally equivalent neural network. Note that when merging several features the process is similar to Eq. and Eq, 8.
Iterative Feature Merging with Weight Matching: Composing the two components proposed above, get the algorithm to measure the feature complexity. We iteratively merge the features in each layer / with the smallest weight distance defined in Eq. until
min (L.i)
Dr
max
D D
m.nE mid
With hyper-parameter B, D! D'.,. corresponds the weight distance as detined in Eq.
To get the mean value in Eq. 8, we also keep track of the number of features merged into one feature. Consider merging two feature Z and Z where Zi is the merged feature of N,, features and Z is the merged feature of N₂, features, then we have:
Whhh ((-1) (Nmm.
Wl+1 [Emmmen)) Nn [(rtmann))
Wi+1
(N
(10)
merged
The detailed algorithm for Iterative Feature Merging (IFM) represented in Algorithm I.
EXPERIMENTS
In Sec. 4.1, we empirically verify the existence of functionally equivalent features. Sec. 4.2, show that the iterative feature merging could greatly reduce the number of parameters while

---

Published as a conference paper at ICLR 2024
70-
test accuracy (%6
60%
30-
test accuracy
SenNee50
TeaNet50 originl RECUINCY
40
ResNee50
VOG12 original
VGG13
10-
1ssNee50 original RECUINCS
VOG13
V0013 original ascutacy
10
VGC1 original accurweyy
19
VOC15
VDC15 original accurwcy
50
40
remaining params (%)
remaining params (%6
(a) Testing accuracy to the percentage of remaining pa- (b) Testing accuracy to the percentage of remaining rameters VGG networks
parameters two different ResNet50
Figure 4: Results of iterative feature merging on ImageNet. (a): Testing accuracy (top-1) in relation to the percentage of remaining parameters of VGGs after feature merging with different 3. (b): Test- ing accuracy (top-1) 1) to the percentage of remaining parameters of ResNet50 after feature merging. "ResNet50 v1" and "ResNet50 v2" corresponds to the two different checkpoints torchvision.
maintaining the performance without fine-tuning. In Sec. 4.3, provide more empirical results regarding the iterative feature merging. We conducted experiments using VGG and ResNet models on CIFARIO and ImageNet, respectively. For more details, refer Appendix D.
4.1 EXISTENCE OF EQUIVALENT FEATURES
In this section, we empirically show that the functionally equivalent feature as defined in Sec. 2.3 actually exist in vanilla networks. We conduct experiments on VGG16 trained on CIFARIO with SGD optimizer for 150 epochs. For details, refer to Appendix D.
As shown Fig. 2(a), we visualize the normalized distances between 512 different features from the last convolution layer of the VGG16. The minimum distance between two features is 2e 15. For most features, the distance between them is relatively low. Similar results were observed for other layers. For more results please refer to Appendix E.
For model f(,,) :) we further consider the permutation x that swaps the functionally equivalent features found by the feature weight matching introduced in Sec. 3. Similar the metric used to identify LMC, we further interpolate between the parameter 0 and the parameter after permutation (0) and test the accuracy of model f ((1 X)0 + X= (z)( The results shown in Fig. 2(b) where solid lines correspond to the permutation swapping equivalent features and dashed lines cor- respond random permutation swapping the same number of features for comparison. The results shows that interpolating between functionally equivalent features does not affect the performance.
4.2 ITERATIVE FEATURE MERGING
In this section, we present the results of iterative feature merging with VGG (Simonyan & Zisser- man, 2015) and ResNet (He et al., 2016) on CIFAR10 (Krizhevsky et al., 2009) and ImageNet (Deng al., 2009). We trained models on CIFARIO and use pretrained checkpoints on ImageNet provided by torchvision (maintainers & contributors, 2016). For details, refer to Appendix D.
Iterative Feature Merging on CIFARIO
By varying 3 in Algorithm 1, we could observe the change of testing accuracy different number of features are merged. The results for VGG networks on CIFARIO are shown in Fig. 3(a), with the x-axis representing the percentage of remaining parameters and the y-axis representing testing accuracy. It clear that the larger the network is, the more equivalent features could be merged before significantly affecting the performance. Specifically, for VGG16, could reduce the number of parameters to 23.03% while keeping the testing accuracy at 91. which is relatively close to the original testing accuracy 93.51%.
We further demonstrate the relationship between testing accuracy and the number of network pa- rameters in Fig. 3(b). Here we apply grid search B and choose the largest / with the testing
7

---

Published as a conference paper at ICLR 2024
320
A
VCG23 V5G16
Feature Complexity plexity
420
200
200
13
Layers
Truck
Cabin
Wheel
Container
(a) Feature complexity at each layer of VGGs
(b) Guided backpropagation results
Figure 5: Empirical results regarding feature complexity. (a): The plot feature complexity at each
layer of VGG networks that is trained on CIFARIO. (b): The results of guided backpropagation of three different sets equivalent features in the VGG16 on three different images from CIFAR10.
accuracy larger than 95% of the testing accuracy before feature merging. With iterative feature merging, larger models (e.g. VGG16 and ResNet50) could reduce more parameter with smaller decrease on testing accuracy. Note that we also find ResNet18 outperforming ResNet34 and ResNet50 on CIFARIO. For the reason of this phenomenon, we believe that the capacity of ResNet18 8 is already enough to deal with the simple classification task on CIFAR10, e.g. we could reduce up to 60% the parameters of ResNet- without significantly affecting the testing accuracy.
Iterative Feature Merging ImageNet
We find that neural networks learn more complex features solve the more difficult task, i.e. image classification on ImageNet. As shown in Fig. 4, iterative feature merging only reduce approximately 5% parameters for VGG and approximately 10% parameters for ResNet50.
We observe the same trend as CIFARIO that the larger the network is, the more equivalent features could be merged, as shown Fig. 4(a). For different parameters with the same network structure, we conduct experiments achieves higher top-1 accuracy than "ResNet50_v1 (80.86% to 76. 13%) due to different training ppper-parameters.. As shown in Fig. 4(b), the testing accuracy of ResNet50 v2 decreases faster than the testing accuracy of ResNet50 v1 as more features are merged. It indicates that ResNet50 v2 learns more diverse features, which may explain the superior performance of ResNet50_v2.
two different parameters of ResNet50 in torchvision, where "ResNet50_ve v22"
The potential of IFM for pruning
IFM merges functionally equivalent features, which is similar to the channel pruning (Wen et al., 2016; Ye al., 2018) that remove channels reduce computa- tional cost. Note that conventional pruning methods re- quire fine-tuning the pruned model while our proposed method does not. In Fig. 6, we compare our IFM with recently SOTA pruning method "INN" (Solodskikh et al., 2023) which also does not require the fine-tuning. Specifically, INN requires changing the training proce- dure while our IFM can be directly applied to vanilla pre-trained models with high efficiency. See Appendix E for the time complexity analysis. However, it is worth noting that the pruning effect IFM may be limited on complex tasks such image classification ImageNet.
test eccuracy (%)
remaining params (9)
Figure 6: Pruning results of ResNet18 CIFARIO. The INN experiments conducted with the released official code.
4.3 EMPIRICAL RESULTS ON FEATURE COMPLEXITY
Fig. 5(a) shows the feature complexity at each layer of the VGG networks trained on CIFARIO. We find that the feature complexity is increasing at the first several layers and decreasing at the last several layers while the maximum feature complexity is reached in the middle. Several possible reasons could be used explain this phenomenon, e.g. the neural network starts forget
8

---

Published as a conference paper at ICLR 2024
features or many simple features compose complex features at the last several layers. We leave the explanation of this phenomenon for future work.
In Fig. 5(b), we visualize the guided backpropagation of various features in VGGI6. Guided back- propagation (Springenberg et al., 2015) is a method to visualize the part of the image that activate neurons in the neural network by applying ReLU on the gradient through backpropagation. Here we visualize the guided backpropagation of group of functionally equivalent features by setting the gradient of other features to be zero. As shown in Fig. 5(b), a group of functionally equivalent features capture the similar features on different images. The three different groups of functionally equivalent features we demonstrate capture the cabin, wheel and container of trucks respectively. It shows the possibility to align semantics to group of equivalent features learned by the neural network for classification. Please refer to Appendix for more results.
5 RELATED WORKS
Model Similarity. To harness the behaviour of neural networks in train/test. efforts have been made especially for features (representations) learned by networks: the transferability of features (Yosin- ski et al., 2014), the training dynamics (Morcos et al., 20, the effect of network width and depth on the learned feature (Nguyen al., 2021). Among them, measuring the feature similarity (rep- resentational similarity) has become important issue, with various metrics to measure the fea- ture similarity including those based on canonical correlation analysis (Raghu et al., 2017; Morcos et al., 20;8); the measures based on alignment (Li et al., 2016; Williams et al., 2021); the mea- sures based on representational similarity matrix (Shahbazi et al., 2021; Tang et al., 2020) and the measures based topology (Barannikov et al., 2022). Based on various feature similarity mea- sures, previous works have shown that different networks with similar performance learn similar features (Li al., 2016; Wang et al., 2018) and there are similar features within network with in- creased width/depth (Nguyen et al., 2021). For measuring similarity between models, another focus functional similarity (Madani et al., 2004; Bansal al., 2021; Bhojanapalli et al., 2021; Klabunde et al., 2023) i.e. measuring the similarity of outputs. However, Ding et al. (2021); Hayne et al. (2022) show that functional similarity and feature similarity are not necessarily correlated such that similar features can yield different outputs while similar outputs can be obtained from different fea- tures. In this paper, we propose functionally equivalent feature, extending the concept of equivalent feature in feature similarity literature and functional measures in functional similarity literature.
Linear Mode Connectivity, Freeman & Bruna (2017); Draxler et al. (2018); Garipov et al. (2018) observed Mode Connectivity, i.e., different well-trained models can be connected through non- linear path of nearly constant loss. Frankle et al. (2020) first proposed the notion of Linear Mode Connectivity (LMC), where models are connected through linear path of constant loss. Frankle et al. (2020) observed LMC for networks that are jointly trained for a short amount of time before going through independent training (referred as the spawning method). Later, Entezari et al. (2022); Ainsworth et al. (2023); Liu et al. (2022) showed that even independently trained networks can be linearly connected when permutation invariance is taken into account. In particular, Ainsworth al. (2023) utilized the permutation invariance to align the neurons of two neural networks and formu- late the neuron alignment problem as a bipartite graph matching problem. More recently, Zhou al. (2023) discovered a stronger notion of linear connectivity, called Layerwise Linear Feature Connec- tivity (LLFC), which indicates that the feature maps of every layer in different trained networks are also linearly connected, and demonstrated the co-occurrence of LLFC and LMC.
Category Theory in Machine Learning. Category theory have been used in machine learn- ing (Shiebler et al., 2021) for various topics including backpropagation (Fong et al., 2019), cate- gorical probabilities (Fritz, 2020), conditional independence (Mahadevan, 2022a), supervised learn- ing (Harris, 2019), reinforcement learning (Mahadevan, 2022b) and so More recently, Yuan (2023) provides an analysis upon the power of perfect foundation models using category theory.
CONCLUSION
We have defined functionally equivalent features and introduce the concept of feature complexity, as learned by the network. To measure the feature complexity, we propose an efficient algorithm called Iterative Feature Merging (IFM). Experiments CIFAR10 and ImageNet show its efficiency and inspire insights regarding the feature complexity. Our IFM also shows potential in pruning pre-trained models without fine-tuning. See more discussion in Appendix A.

---

Published as a conference paper at ICLR 2024
REFERENCES
Jifí Adámek, Horst Herrlich, and George Strecker. Abstract and concrete categories. Wiley- Interscience, 1990.
Samuel Ainsworth, Jonathan Hayase, and Siddhartha Srinivasa. Git re-basin: Merging models mod- ulo permutation symmetries. In The Eleventh International Conference on Learning Representa- tions, 2023.
Yamini Bansal, Prectum Nakkiran, and Boaz Barak. Revisiting model stitching to compare neural representations. Advances neural information processing systems, 34:225-236, 2021.
Serguei Barannikov, Ilya Trofimov, Nikita Balabin, and Evgeny Burnaev. Representation topology divergence: A method for comparing neural network representations. ICML, volume 162 of Proceedings of Machine Learning Research, pp. 1607-1626. PMLR, 2022.
Srinadh Bhojanapalli, Kimberly Wilber, Andreas Veit, Ankit Singh Rawat, Seungyeon Kim, Aditya Menon, and Sanjiv Kumar. On the reproducibility of neural network predictions. arXiv preprint arXiv:2 033349 2021.
Yann N. Dauphin, Razvan Pascanu, Çaglar Gülgehre, KyungHyun Cho, Surya Ganguli, and Yoshua Bengio. Identifying and attacking the saddle point problem in digh-dimensional non-convex op- timization. In NIP. pp. 2933-292 2941, 2014.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Icee, 2009.
Frances Ding, -tan-Stanislas Denain, and Jacob Steinhardt. Grounding representation similarity through statistical testing. Advances in Neural Information Processing Systems, 34:1556-1563 2021.
Felix Draxler, Kambis Veschgini, Manfred Salmhofer, and Fred Hamprecht. Essentially no barriers in neural network energy landscape. International conference on machine learning, pp. 1309- 1318. PMLR, 2018.
Rahim Entezari, Hanie Sedghi, Olga Saukh, and Behnam Neyshabur. The role of permutation in- variance in linear mode connectivity of neural networks. In International Conference on Learning Representations, 2022.
Brendan Fong, David Spivak, and Rémy Tuyéras. Backprop as functor: A compositional perspec- tive on supervised learning. In 2019 34th Annual ACM/IEEE Symposium on Logic Computer Science (LICS), pp. 1-13. IEEE, 2019.
Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. In ICLR. OpenReview.net, 2019.
Jonathan Frankle, Gintare Karolina Dziugaite, Daniel M Roy, and Michael Carbin. Stabilizing the lottery ticket hypothesis. arXiv preprint arXiv: 1903.0090 2019.
Jonathan Frankle, Gintare Karolina Dziugaite, Daniel Roy, and Michael Carbin. Linear mode con- nectivity and the lottery ticket hypothesis. In International Conference on Machine Learning, pp. PMLR, 2020.
C. Daniel Freeman and Joan Bruna. Topology and geometry half-rectified network optimization. In International Conference on Learning Representations, 2017.
Tobias Fritz. A synthetic approach markov kernels, conditional independence and theorems sufficient statistics. Advances in Mathematics. 370:107239, 2020.
Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry P Vetrov, and Andrew G Wilson. Loss surfaces, mode connectivity, and fast ensembling of dnns. Advances in neural information pro- cessing systems, 31, 2018
10

---

Published as a conference paper at ICLR 2024
Yunchao Gong, Liu Liu, Ming Yang, and Lubomir Bourdev. Compressing deep convolutional net- works using vector quantization. arXiv preprint arXiv:1412.6115, 2014.
Stephen Hanson and Lorien Pratt. Comparing biases for minimal network construction with back- propagation. volume 1, 1988.
Kenneth D Harris. Characterizing the invariances of learning algorithms using category theory. arXiv preprint arXiv:1905.02072 2019.
Babak Hassibi, David G Stork, and Gregory J Wolff. Optimal brain surgeon and general network pruning. In IEEE international conference on neural networks, pp. 293-299. IEEF, 1993.
Lucas Hayne, Heejung Jung, Abhijit Suresh, and R McKell Carter. Grounding high dimensional representation similarity by comparing decodability and network performance. 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR. pp. 770-778. IEEE Computer Society, 2016.
Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in neural network. CoRR, Ahs/1550.00531, 2015.
Max Klabunde. Tobias Schumacher, Markus Strohmaier, and Florian Lemmerich. Similarity of neural network models: survey of functional and representational measures. arXiv preprint (rXXi:2335.063222 2023.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Yann LeCun, John Denker, and Sara Solla. Optimal brain damage. volume 2, 1989.
Namhoon Lee, Thalaiyasingam Ajanthan, and Philip H. S. Torr. Snip: single-shot network pruning based on connection sensitivity. In ICLR (Poster). OpenReview.net, 2019.
Yixuan Li, Jason Yosinski, Jeff Clune, Hod Lipson, and John E. Hoperoft. Convergent learning: Do different neural networks learn the same representations? In ICLR, 2016.
Chang Liu, Chenfei Lou, Runzhong Wang, Alan Yuhan Xi, Li Shen, and Junchi Yan. Deep neural network fusion via graph matching with applications to model ensemble and federated learning. In ICML, pp. 13857-13869. 2022.
Saunders Mac Lane. Categories for the working mathematician, volume 5. Springer Science & Business Media, 2013.
Omid Madani, David Pennock, and Gary Flake. Co-validation: Using model disagreement un- labeled data to validate classification algorithms. Advances in neural information processing systems, 17, 2004.
Sridhar Mahadevan. Categoroids: Universal conditional independence. arXiv:2208. 1077, 2022a.
arXiv preprint
Sridhar Mahadevan. Unifying causal inference and reinforcement learning using higher-order cate- gory theory. arXiv preprint arXiv:2009.06262, 2022b.
Viscon maintainers and contributors.
Vision: PyTorch's Computer Vision library,
November 2016.
Ari S. Morcos, Maithra Raghu, and Samy Bengio. Insights representational similarity in neural networks with canonical correlation. In NeurlPS, pp. 5732-5741, 2018.
Thao Nguyen, Maithra Raghu, and Simon Kornblith. Do wide and deep networks learn the same things? uncovering how neural network representations vary with width and depth. In ICLR. OpenReview.net, 2021.
11

---

Published as a conference paper at ICLR 2024
Hanyu Peng, Jiaxiang Wu, Shifeng Chen, and Junzhou Huang. Collaborative channel pruning for deep networks. In ICML, volume 97 of Proceedings of Machine Learning Research, pp. 5113- 5122. PMLR. 2019.
Maithra Raghu, Justin Gilmer, Jason Yosinski, and Jascha Sohl-Dickstein. SVCCA: singular vector canonical correlation analysis for deep learning dynamics and interpretabillty. In NIPS, pp. 6076- 6085, 2017.
Mahdiyar Shahbazi, Ali Shirali, Hamid Aghajan, and Hamed Nili. Using distance on the riemannian manifold to compare representations in brain and in models. Neurolmage, 239: 18271, 2021.
Dan Shicbler, Bruno Gavranović, and Paul Wilson. Category theory in machine learning. arXiv preprint arXiv:2 (0t.:22006.00..2.. 2021.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.
Kirill Solodskikh, Azim Kurbanov, Ruslan Aydarkhanov, Irina Zhelavskaya, Yury Parfenov, Dehua Song, and Stamatios Lefkimmiatis. Integral neural networks. In CVPR, pp. 16113-16122. IEEE, 2023.
Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin A. Riedmiller. Striving for simplicity: The all convolutional net. In ICLR ( (Workshop), 2015.
Suraj Srinivas and R Venkatesh Babu. Data-free parameter pruning for deep neural networks. arXiv preprint arXiv: 1507 0496666199999 2015.
Cheng Tai, Tong Xiao, Yi Zhang, Xiaogang Wang, al. Convolutional neural networks with low- rank regularization. arXiv preprint 1rXiv:1511.0600 2015.
Hidenori Tanaka, Daniel Kunin, Daniel Yamins, and Surya Ganguli. Pruning neural networks without any data by iteratively conserving synaptic flow. volume 33, pp. 6377-6389, 2020.
Shuai Tang, Wesley J. Maddox, Charlie Dickens, Tom Diethe, and Andreas C. Damianou. Similarity of neural networks with gradients. CoRR, abs/2003. 1449, 2020.
Huan Wang. Can Qin, Yue Bai, Yulun Zhang, and Yun Fu. Recent advances on neural network pruning at initialization. arXiv preprini arXiv:2103.cmm 064660 2021.
Liwei Wang, Lunjia Hu, Jiayuan Gu, Zhiqiang Hu, Yue Wu, Kun He, and John E. Hoperoft. Towards understanding learning representations: To what extent do different neural networks learn the same representation. In NeurIPS, pp. 9607-9616, 2018.
Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li. Learning structured sparsity in deep neural networks. In NIPS, pp. 2082,, 2016.
Alex H. Williams, Erin Kunz, Simon Komblith, and Scott W. Linderman. Generalized shape metrics on neural representations. In NeurlPS, pp. 4738-4750, 2021.
Jiaxiang Wu, Cong Leng, Yuhang Wang, Qinghao Hu, and Jian Cheng. Quantized convolutional neural networks for mobile devices. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4820 4828, 2016.
Jianbo Ye, Xin Lu, Zhe Lin, and James Z. Wang. Rethinking the wwwssr-oooo-------neeeeee assumption in channel pruning convolution layers. ICLR (Poster). OpenReview.net, 2018.
Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? In NIPS, pp. 3320-3328, 2014.
Yang Yuan. On the power of foundation models. In International Conference Machine Learning, pp. 40519-40530. PMLR, 2023.
Hao Zhou, Jose M Alvarez, and Fatih Porikli. Less is more: Towards compact cnns. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part /V 14, pp. 662-677. Springer, 2016.
Zhanpeng Zhou, Yongyi Yang. Xiaojiang Yang. Junchi Yan, and Wei Hu, Going beyond linear mode connectivity: The layerwise linear feature connectivity. arXiv preprint 2000:22300002886 2023.
12

---

Published as a conference paper at ICLR 2024
A DISCUSSION ON LIMITATION AND FUTURE WORK
Since we do not proactively modify the structure and training procedure of the neural network, when facing difficult tasks (ImageNet classification), the number of parameters that could be reduced by IFM is limited. For better pruning performance, designing network structure and training algorithms to learn a more compact representation could be promising direction, which we leave for future researches. Though our proposed definition and method are structure and data agnostic, another major limitation is that we only conduct experiments for image classification task. More experiments could be done regarding different network structures and different tasks. We hope that our proposed feature complexity will inspire future research for better understanding and enhancement of neural network behavior.
B ALGORITHM FOR ITERATIVE FEATURE MERGING
In this section, we present the detailed algorithm for iterative feature learning (IFM) in Algorithm
Algorithm Iterative Feature Merging with Weight Matching
Input: Parameter for a L-layer neural network f(;s)) :), where the weight at the I-th layer W E hyperprrmetrt 3 for / L]. do
while True do for m E [1, di] do
Nm +-1
for n E [1, dz] do
D mn < I|
W [n..] 112 |W [mm] - W
Im.:
end for end for
if min Dr mn 8 max D then
break else
Mmin; Benin f- arg minlm,n,m74 Dr
mn
Wil
marged (N Wh+ [Aminss]
merged + W!
[mminne] + W!
(((+))
+ Nnmon W++1
Fomtmnnn// )/ + N Numon
N Nmerged f- N Nmmsn N Bomen
end if
end while
end for
C PROOFS
C. PROOF FOR PROPOSITION 2.2
Let us first formally define linear mode connectivity.
Definition C.I. [Linear Mode Connectivityl Given dataset D and neural networks of the same structure and different parameters 02 and 0g with similar loss (D()a) ss (D()),, the two networks are linearly connected if
Co(Oa) xe CD(0s) LD (ada + (1 a))))) Va E [0.
(11)
Beyond LMC, the linear layer-wise feature connectivity (LLFC) was observed coexist with LMC.
Definition C.2. [Layerwise Linear Feature Connectivity] Given a dataset D and two L-layer neural networks of the same structure and different parameters 0, and 0g. They satisfy LLFC if
Vx D, vl (1, T.J Va (0, 1), Z (aoa + (1 -a)00, oZ (0., x) + Z) x). (12)
13

---

Published as a conference paper at ICLR 2024
Proof. [Proof for Proposition 2.2] Consider two parameter 8, and 0s satisfying LMC, according
to Eq. 12 we have
I' (ala + (1 a)0b, az az'' x) + (1 az) (0.), x)) = of (0a, 2' (0a,2--1
x)) 20.2"'1000 (0a, +(1 (0,, 2' (0s, x))
(13)
When fl is activation layer such as ReLu, we have aα (0a: 22--z-1100 (00: x)) + (1 a)o (0o, 2' (0,,
(az" + (1 a))z" (0.), x))
Here (.) represent the non-linear function, therefore we have
z'(0,, x) = f'(Oa, Z' (0b,
For layers that perform linear transformation such as linear layer, write Eq. 13 in matrix form:
(aW), + -11-a)1 (1-)) ( (az) (0a, (1- x)) aW, 1 (0.. (1 2'-1,, x0,,x)
Then we have
Wo. 2' 20.''(0.x0 + WW. zt (0,, x) Wo. 2/--z-1-00 (06, + W/. zz (0,,
Derive it, we get
(Wo. - -We, 10,, x) - (0,, =
Therefore, we have
Z' x) = (0a: Zt (0b,
Similarly, when Oa and π (06) satisfy LMC, we have
(','(0,, = z' (TT)),, x) (0000, (T (0o), x)) = f' .1p (0-1 Here pl and pt-1 is the permutation matrix defined by π. corresponding natural isomorphism such that each Tgt is pl
(14)
Therefore, we could simply define
C.2 PROOF FOR THEOREM 2.3
Proof. According to the partial order between features defined in Sec. 2.4, when there is more than one natural isomorphisms between To itself then we have
31 E [1, Ji,j E [1,d]], s.t. z < (0), +
(15)
Next, we need to prove that
( - 2)
(16)
Consider the poset { (0) n [1, dil), since the natural isomorphism defines one-to-one respondence between the features, the objects in the same chain are equal. Otherwise, the maximal
element Z Zmaz.s chain S must have
Zmar.s S Z
Z $ S
(17)
which contradict to the statement that Z maz.s in chain S. Therefore we have
31 E [1, MJ, Ji E [1, d.). 8.1. 2 = 7) (0),
(18)
C.3 DETAILED INTRODUCTION OF THE FORMULATION WITH CATEGORY THEORY
The Category Corresponding to Model Structure. As we abstractly present the model structure with a category, it could be depicted as follows (we omit identity circle and composites of arrows for simplicity).
14 2 -..., f(,
The objects (x, Zt, etc.) in the category are the shape of the feature maps (including inputs and output). For example, for neural network structures designed for ImageNet, the x here is tensor of shape x 224 x 224. Arrows between the objects indicate the type of transformations applied on the feature maps. For example, the first layer f' for VGG is convolution with a 3 x 3 convolution kernel. For it to be a category, according to the definition of category, two additional conditions should be satisfied:
14

---

Published as a conference paper at ICLR 2024
Identity: For each object there exists anrow idg
Composition: For each pair of arrows f.g with domg codf, there an arrow g o domf codg called their composite.
Identity condition could be easily satisfied where we add identity arrow to each object which means we conduct no transformation to the feature maps. For composition, we add composition of arrows means we firstly apply transformations
corresponds to composition of transformations (e.g. f² the first layer on x then apply transformations in the second layer and produce z²).
The Functor Corresponding to Parameterization of Neural Network. Functors, by definition, map each object and arrow from on category to another category, while preserving the structure. For functors in our definition, it map the category of network structure to the category of tensors where objects are tensors while arrows are transformations between them. For a functor T, it maps each arrow (the type of transformation) specific transformation with specifie parameters.
D EXPERIMENT DETAILS
Training details for models CIFARIO We train models with the same hlyee--paamnee each model, we train using SGD with momentum at 0.9 and weight decay at le - for 150 epochs. The initial learning rate is at 0.1 and reduce learning rate at 80 and 120 epoch by multiply it with 0.1. For data augmentation, we only use random horizontal flip with probability set at 50%.
For
Note that the model structures number of layers for the classifier is reduced instead of 3. For ResNet, we apply the conventional change such that the convolution kernel size is set be at the first layer.
little bit different to the structures ImageNet. For VGGs, the
Checkpoint details for models on ImageNet For each model, we use pretrained check- points ImageNet provided by pytorch (maintainers & contributors, 2016). Note that there are two different checkpont for ResNet50 named "ResNet50. Wew.::...Wihhh.MMMAEEEEEEEE "ResNet50. Weights. Triighs.MAAEEETTISS IMAGENETIK.V2"
and
Details for Iterative Feature Merging For IFM, the only hyper-parameter is the 3 in Algorithm 1. We grid search in the list [0.01, 0.03, 0.05, 0.07, 0.1, 0.12, 0.14, 0.15, 0.18, 0.2].
Note that for ResNet, we only merge the features of the last two block for better merging effect.
ADDITIONAL. RESULTS
E
TIME COMPLEXITY OF IFM
The comutational cost of IFM is relative low since we only need compute the weight matching distance between features, which is faster than conducting model inference. Table E. shows the time consumption one iteration for different models on CIFARIO 10 and ImageNet where each result if average over 100 iteration. The result is conducted with one Geforce RTX 2080Ti.
Table 2: Time consumption for one iteration with different models on different dataset. The result is averaged over 100 iterations. dataset
model
time for one iteration (s)
VGGT3
0.018 + 0.003
VGG16
0.027 + 0.005
CIFARIO
ResNet18
0.015 + 0.003
ResNet50
0.127 + 0.012
VGG13
0.802 0.053
VGG16
0.814# :.054
ImageNet
VGG19
0.822 0.045
ResNet18
0.015 + 0.002
ResNet50
0.127 4 0.010
15

---

Published as a conference paper at ICLR 2024
Broed boluns
Layer1
Layer2
Layer3
Layer4
J0her Prikare
Layer5
Layer6
Layer7
Layer8
Layer9
Layer10
Layer11
Layer12
Figure Distance weight between features the first 12 layers of VGG16 trained CIFAR10.
E.2 WEIGHT DISTANCE BETWEEN FEATURES
In Fig. 2.4, we present weight distance between features at the last convolution layer of VGGI6 trained on CIFAR10. In this section, we present the result of the first 12 layers in Fig. E.2.
E.3 GUIDED BACK- PROPAAGTAAAA RESULTS
In this section, show more empirical results of the guided backpropagation of different groups of functionally equivalent features. Fig. 5(b), we show that group of features may corresponds to semantic in class. In Fig. 8, further show that group of functionally equivalent features may correspond to a semantic across different classes, where the specific group of features are activated by legs which is presented both horses, cats and dogs. We also provide results in Fig. 9 that similar to Fig. 5(b) where different semantics regarding boats are learned by groups of functionally equivalent features. Note that all these results are from features at the middle of VGG16, since the result of most features at the last layer cover the whole image. Future works may explore more on the different groups of functionally equivalent features.
F FURTHER DISCUSSION: RELATIONSHIP BETWEEN IFM AND MODEL COMPRESSION METHODS
Brief Introduction of Model Compression and Channel Pruning. The increasing size of neural networks have motivated the standing research on network compression over the decade. To reduce the computation cost, various methods have been proposed: knowledge distillation (Hinton et al., 2015), quantization (Gong et al., 2014; Wu et al., 2016), low-rank factorization (Tai et al., 2015), etc. Specifically, network pruning (Hanson Pratt, 1988) tend to reduce redundant parameters that are not sensitive to the performance. The pruning methods include unstructured pruning (LeCun et al., 1989; Hassibi et al., 1993; Srinivas & Babu, 2015) and structured pruning (Zhou et al., 2016; Wen al., 2016; Ye al., 2018). The unstructured pruning remove connections between neurons that
16

---

Published as a conference paper at ICLR 2024
Horse
Cat
Dog
Legs
Figure 8: Guided backpropagation result of the VGG16 on CIFARIO. Here we show that a group of functionally equivalent features may corresponds to semantic across several classes.
Boat
Bridge
Side
Fore
Figure 9: Guided backpropagation result of the VGG16 on CIFARIO. Here we show that a group of functionally equivalent features may corresponds to semantic regarding boats.
Table 3:
Comparison between popular pruning methods about whether they require access to data before and after pruning. Note that our method requires no data both before and after pruning.
Method
access to data before pruning
training (fine-tuning) after pruning
Structured Sparsity Learning (SSL)
(Wen et al., 2016)
Iterative Magnitude Pruning(IMPP)
(Frankle Carbin, 2019; Frankle et al., 2019)
SNIP
(Lee et al., 2019)
Iterative Synaptic Flow Pruming (SynlFlow)
(Tanaka el al., 2020)
x
Integral Neural Network (INN)
(Solodskikh et al., 2023)
Iterative Feature Merging (IFM)
(Ours)
the dimensionality of the feature map does not change. The structured pruning, on the other way, removes groups of weights (filters, channels, etc.) and reduces the dimension of the feature map.
17

---

Published as a conference paper at ICLR 2024
Difference between IFM and Channel Pruning Methods. In general, the effect of IFM and chan- nel pruning is similar: reducing the dimension the feature map. The key difference is that IFM has different purpose compared with channel pruning methods. The proposed IFM aims at finding the most compact representation without significantly affect the performance while the purpose of channel pruning is better preserving the performance under certain computation budget. The em- pirical difference is that the proposed IFM does not require access to any data. Efforts have also been made develop data agnostic prune methods. For pruning initialization (Pal) (Wang al., 2021), Tanaka et al. (2020) propose data agnostic method to prune the network before training. For pruning after training (PaT), Solodskikh et al. (2023) propose new training algorithm that does not require fine-tuning to prune the model after training. However, they either require the access to training (fine-tuning) procedure after pruning or before pruning. When considered as a prune method, to the best of our knowledge, the IFM is the first data-agnostic prune method not requiring any training or fine-tuning (as shown in Table 3). On the other hand, the compression ratio could not be precisely controlled in IFM. Therefore, fine-tuning might also be required after IFM for the proposed method to be an actual prune method.
18


Reviews:
[{"review_id": "jLqvFsh5EO", "reviewer": "Reviewer_oxNv", "summary": "This paper proposes the concept of neural network feature complexity which is (claimed by the authors) a more formal and comprehensive description of the behavior of the trained neural networks. This idea, along with the technical part including theory, approach and empirical results, to my best knowledge, are basically new in literature. Specifically, the authors develop a quantitative metric to enable the reduction of neural network parameters based on the proposed concept of feature equivalence. In fact, the layer-wise feature complexity is also well supported by a number of empirical studies which also well align with the intuition: e.g. higher feature complexity with the same size of networks can achieve better performance; a larger network tends to learn redundant feaures; equivalents features may correspond to certain low-level semantics. It also well establishes the connection to linear mode connectivity (LMC).\n\nOne other interesting aspect of the paper is introducing the powerful tool of category theory to establish the (clear) theoretical foundation of the work: represent the network structure as a category and a certain neural network as a functor that maps this structure to specific parameters. The authors also give a clear discussion to differentiate their work for using the category theory. This perspective along with its technical derivation and results are new to my best knowledge and well fit with the proposed paradigm for understanding the feteature complexity.", "questions": "How the proposed pruning comapre with other (SOTA) pruning method? Additional experiments will further enhance the paper.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The authors have well stated the contributions in their paper and my main concerns are mostly from writing and presentation. I think the authors are very familiar to the field of the paper and their presented theoretical results and methods are new, well-motivated and well verified by the experiments (espeically from the pruning perspective).", "weaknesses": "Yet the paper need to be more self-contained when introducing the ideas and background There are a few specific suggestions:\n1) in the abstract, the authors are suggested to emphasize the feature complexity is layer-wise to make it more clear as this is a new concept to my knowledge; they may use the term layer-wise ASAP in the abstract.\n2) it is vague to use the saying: merge of features in the first paragraph. Its exact and clear meaning need be better explained for self-containess. I think it refers to merge the weight to merge the feature; \n3) I am a bit confused with Eq. 4, how \\tau_z^{l+1} is imposed on Z^{l+1}? Is it a dot operation or we shall write it as a function of \\tau_z^{l+1}?\n4) it would also be good to clarify what are the Functionally Equivalent Features in Definitions 3.1, I think they are layer-wise f^{l}(theta_a^{l}) vs. f^{l}(theta_b^{l})?\n5) it a bit abuses the notations? does Z() also equals to f()? Please clarify it.\n6) in the introduction part, the authors may mention the partial order idea to make the complexity concept more tangible to readers, rather than until give the specific definition in Section 3.\n\nsome minor typos:\nIn another words->In another word\nit simply apply no transformation->it simply applies no transformation\nin definition 3.1 Functionally Equivalent Feature->Functionally Equivalent Features\nat l-th layer of network->at the l-th layer of network\nin Theorem 3.3 to a itself->to itself\n\nFinally I suggest the authors consider to put an overview of their work in the introduction part to improve the readability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4vOouY4BKE", "reviewer": "Reviewer_QeEM", "summary": "The paper attempts to tackle a pertinent issue in neural networks, focusing on understanding and measuring the similarity between features learned by different neural network initializations. The concepts presented seem promising, especially in defining what are termed as functionally equivalent features and the subsequent proposal of an algorithm, Iterative Feature Merging (IFM). However, there are concerns and areas of improvement that need addressing.", "questions": "* The conclusion acknowledges a limitation in testing only for image classification tasks. Why was the scope of experiments limited to image classification tasks, and how would the approach perform on other tasks? It would strengthen the paper to include experiments from a wider range of tasks or at least provide a rationale for why image classification was chosen as the primary focus. \n\n* Can the authors provide a more intuitive or illustrative example of \"functionally equivalent features\" to aid understanding?\n\n* The Iterative Feature Merging (IFM) algorithm is a central piece of this work. How does the Iterative Feature Merging (IFM) algorithm work in detail, and what makes it efficient? A clear and detailed algorithmic procedure, possibly with pseudocode, should be provided for a comprehensive understanding.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "* The paper addresses a significant issue in neural networks—understanding the behavior and feature representation across different initializations.\n\n* The introduction of the term \"functionally equivalent features\" offers a fresh perspective on understanding neural networks.\n\n* The Iterative Feature Merging (IFM) seems promising as an approach to quantify feature complexity.", "weaknesses": "* The term \"functionally equivalent features\" is central to the paper. It would be beneficial to the reader if a simple illustrative example or intuitive explanation accompanied its introduction.\n\n* While the abstract and conclusion provide a concise overview, it remains unclear how rigorous the definitions and proofs, especially related to category theory, are in the main content of the paper. The paper would benefit from a detailed walkthrough of the mathematical formulations and proofs to ensure the robustness of the claims made.\n\n* Given the recent interest in understanding neural network behavior and their feature representations, it is vital to contextualize this work concerning existing literature. How does this work differ or extend previous work on the topic? A comprehensive comparison is essential.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qQP8kgcL3u", "reviewer": "Reviewer_R8Xc", "summary": "The paper proposes an algorithm for merging of functionally equivalent neurons in a neuron network - neurons that do the same job due to having same input weights and same output weights on the same respective input and output connections.  Empirical evaluation is provided showing that sometimes a substantial pruning of the neurons (or merging of features, as the authors call it) can be achieved for a relatively minor drop in accuracy performance.  Authors propose to use the size of the network after the pruning/merge as the means of quantifying network complexity.", "questions": "Aside from my objection to the group theory aspect of the paper, I don't understand what the concept of shape of the features relates to.  Is it the shape of the matrix/tensor representing the features?   \n\nRepeating my question from the previous section - is feature complexity related to generalisation?  And what is actually feature complexity?  Is it the total count of neurons in a network after pruning?  How do we know it means something?    \n\nThe statement \"feature complexity corresponds to the most compact representation of a neural network\" sound like something related to the minimum description length (MDL) or minimum message length (MML) principles.  Any comment how your work relates to those?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "The method is extremely straight forward - compare the distance between concatenated vector of in-going and out-going weights of two neurons in a layer, and if they are below threshold, declare them the same.\n\nEmpirical evaluation seems to show this to be an effective method of pruning neurons in larger networks.", "weaknesses": "I don't understand what the point of dragging of the reader through the group theory part of the paper is accomplishing.  It doesn't seem to me like group theory is used to arrive at the proposed feature merging/pruning algorithm.  Permutation of pairs of neurons in a network by swapping their input and output weights (provided they are connected to the same inputs and outputs) is sort of obvious, and in that light, the merging of the features/neurons is quite simple and straight forward.  Defining categories, functors and objects in this setting doesn't seem to give us any extra insight, or produce different tools for how to do the merging.  I find that the theoretical part of the paper has very little to do with the proposed practical aspect, other than perhaps we can name things using group theory terminology.  And if anything, it seems to obfuscate a very straight forward pruning technique that is proposed.  \n\nIn this race to the state of the art accuracy (which I still think we are all in) it seems that any pruning technique that sacrifices even a fraction of accuracy for some gains in computational/training time costs, is not going to be of practical use.\n\nI can't tell if the proposed feature complexity is meaningful in any way.  Does it correlate with generalisation?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RRXiEgjXmh", "reviewer": "Reviewer_Po5f", "summary": "The paper delves into the feature similarity and further proposes the feature equivalence, and finally feature complexity of a trained neural network. The authors also introduce the well-established math tool of category theory to elegantly estaiblish the theory of the introduced concepts as well as methods. Beyond new theoretical understanding, the authors further devise an iterative algorithm to achieve the computing of feature complexity based on which the functionally equivalent features can be found in a neural network such that pruning of neural networks can be fulfilled.", "questions": "How sensitive of the proposed IFM method to the hyperparameter \\beta? The authors shall discuss it and give some ablation studies if possible.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1) an important understanding to the trained neural networks by introducing the new concept of feature equivalence and feature complexity which go beyond the feature similarity.\n2) an effective algorithm for computing the feature complexity and as a side-product can be used to prune the trained networks on the fly without access to the training dataset, neither with any iterations for finetuning which are needed in existing pruning methods.\n3) the authors further draw a few interesting observations which are well interpretable that further consolidates the impact and soundness of this work.", "weaknesses": "The paper could be improved in some aspects: 1) due to the density of the new information in this paper, the authors may move some empirical results in plot to Section 1, to make the readers more easily to jump into the main idea and discovery of the work; 2) the related work part can be extendeded. Specifically, please discuss the difference and relation to the recent work in ICML 2023: On the power of foundation models. As far as I know, the paper also intensively uses the category theory tools for interpreting neural networks especially foundations models. Also, as the paper proposes a new pruning mehtod, the related work part need also discuess peer methods; 3) for experiment part, the authors are required to compare peer pruning methods as the main technical approach presented in this paper is a pruning technique. Also the authors shall more comprehensively discuss the pros and cons of their pruning method in the context of network pruning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)