Function Feature Learning of Neural Networks
Authors: Guangcong Wang
Jianhuang Lai
Guangrun Wang
Wenqi Liang
Venue:  (2020)

Abstract: We present a Function Feature Learning (FFL) method that can measure the similarity of non-convex neural networks. The function feature representation provides crucial insights into the understanding of the relations between different local solutions of identical neural networks. Unlike existing methods that use neuron activation vectors over a given dataset as neural network representation, FFL aligns weights of neural networks and projects them into a common function feature space by introducing a chain alignment rule. We investigate the function feature representation on Multi-Layer Perceptron (MLP), Convolutional Neural Network (CNN), and Recurrent Neural Network (RNN), finding that identical neural networks trained with different random initializations on different learning tasks by the Stochastic Gradient Descent (SGD) algorithm can be projected into different fixed points. This finding demonstrates the strong connection between different local solutions of identical neural networks and the equivalence of projected local solutions. With FFL, we also find that the semantics are often presented in a bottom-up way. Besides, FFL provides more insights into the structure of local solutions. Experiments on CIFAR-100, NameData, and tiny ImageNet datasets validate the effectiveness of the proposed method.

Under review as a conference paper at ICLR 2020
FUNCTION FEATURE LEARNING OF NEURAL NET- WORKS
Anonymous authors Paper under double-blind review
ABSTRACT
We present a Function Feature Learning (FFL) method that can measure the sim- ilarity of non-convex neural networks. The function feature representation pro- vides crucial insights into the understanding of the relations between different local solutions of identical neural networks. Unlike existing methods that use neuron activation vectors over given dataset neural network representation, FFL aligns weights of neural networks and projects them into a common function feature space by introducing chain alignment rule. We investigate the function feature representation on Multi-Layer Perceptron (MLP), Convolutional Neural Network (CNN), and Recurrent Neural Network (RNN), finding that identical neural networks trained with different random initializations on different learn- ing tasks by the Stochastic Gradient Descent (SGD) algorithm into different fixed points. This finding demonstrates the strong connection be- tween different local solutions of identical neural networks and the equivalence of projected local solutions. With FFL, we also find that the semantics are often pre- sented in a bottom-up way. Besides, FFL provides more insights into the structure of local solutions. Experiments on CIFAR-100 NameData, and tiny ImageNet datasets validate the effectiveness of the proposed method.
be projected
INTRODUCTION
Neural networks have achieved remarkable empirical success in wide range of machine learning tasks (LeCun et al. 1989 Krizhevsky et al. 2012 He et al. 2016) by finding good local solution. How to better understand the characteristies of local solutions of neural networks remains an open problem. Recent evidence shows that identical neural networks trained with different initializations achieve nearly the same classification accuracy. Are these trained models (local solutions) equiva- lent? et al. 2016) claimed that neural networks converge to apparently distinct solutions in which is difficult to find one-to-one mappings of neuron units. (Raghu et al,, 2017 Morcos et al. 2018 Kornblith et al., 2019) concentrated comparing representations of neural networks using the in- termediate output of neural networks over a given dataset. These studies provide important insights into the understanding of similarity of neurons by probing and aligning the intermediate output (or neuron activation) representation of data points, but they do not focus on how to directly measure the similarity of function feature representations of neural networks using weights of networks.
In this paper, we propose a Function Feature Learning (FFL) method to measure the similarity be- tween different trained neural networks. Instead of using intermediate activation/respoonee values of neural networks over bunch of data points, FFL directly learns an effective weight feature represen- tation from trained neural networks. To address the problem of random permutated weights (Figure [ chain alignment rule is introduced to eliminate permutation variables. The aligned weights are then learned to project into function feature representation space by classifying different classes of local solutions. The learned function features can be used to describe the characteristics of local solutions. With FFL, one can validate some assumptions about the similarity of local solutions.
Function feature learning is built upon data feature learning. Given set of data points, data feature learning is to learn a function fa that can describe the underlying representations to measure data similarity. Similarly, given set of data representation functions (fr). function feature learning is to leam a function F that can measure the similarity of [fa]. Specifically, an identical neural network with different weights forms family of functions {f:} that could cover different function types (an

---

Under review as a conference paper at ICLR 2020
identical neural network with different weights be used different function types for different learning tasks in practice). Function feature learning attempts to discover characteristics of functions and thus provides an effective metric for function similarity measure. In this paper, we propose to describe the function feature representation by using weights of neural networks instead of network structures because neural networks often share common set of functional building blocks, e.g., global/local linear units, activation units, and normalization units.
Overall, we make four main contributions follows.
We propose a Function Feature Learning (FFL) method to measure the similarity of iden- tical neural networks trained from different initializations. FFL first addresses the random permutation of weights of neural networks by using chain alignment rule and then projects the aligned weights into a common space. We find that there exist strong relations between different local solutions optimized by the Stochastic Gradient Descent (SGD) algorithm.
We investigate function feature representations of Multi-Layer Perceptron (MLP), Convo- lutional Neural Network (CNN), and Recurrent Neural Network (RNN) on the CIFAR-100 100, NameData, and tiny ImageNet datasets. With the chain alignment rule, the proposed FFL approach achieves consistent high accuracy for three types of neural networks, which shows the effectiveness of FFL and the soundness of the aforementioned finding.
We investigate the chain based semantics and the results suggest that the semantics are hierarchical. The projection directions of all layers are arranged in order along with the depth of neural networks. In short, the semantics are presented in bottom-up way.
We analyze several factors of neural networks and find that 1) adding more layers or chang- ing the ReLU activation function into leaky ReLU has little impact on the structure of local solutions; 2) changing plain networks into residual networks has some impact on local so- lutions; 3) SGD often converges to stable structure of local solutions while the Adam optimizer does not.
Related Work. Neural networks are often regarded black-boxes due the non-convexity. To bet- ter understand these black-boxes, various approaches provide effective tools for visual interpretabil- ity of neural networks (Simonyan al., 2013] Dosovitskiy & Brox 2016 Zeiler & Fergus| 2014] Zhou et al. 2015) Selvaraju et al., 2016). These approaches utilized gradient of the class scores with respect to input or de-convolution operations to visualize the attention activations high-level semantics.
Instead of building visual interpretability foundations between input and output, recent research (Raghu et al. 2017 Morcos et al. 20188] Kornblith et al. 2019) focused representations of neural networks by exploiting intermediate activations/eeatures to describe the similarity neural net- works. For example, SVCCA (Raghu al. 2017) used singular value decomposition and canonical correlation analysis tools for network representations and similarity comparison of neural networks. After that, projection weighted CCA approach developed for better understanding similarity of neural networks. In (Kornblith et al. (2019), centered kernel alignment method was proposed to measure the relation between data representational similarity matrices. Our approach concentrates on the function/weigee feature representation but not intermediate representations of data points, which is greatly different from these works.
PRELIMINARIES
In this section, we first introduce related notations and then describe the permutation problem of neural networks.
2.1 NOTATION
Let L-layer neural network contain series of stacked units {gr(2; Wi)lf with global/local lin- ear operations, where æ E R is input and W; E R x denotes the weights of the I-th unit. ny denotes the number of neurons in the I-th layer. We formulate the stacked units as F, = (o(g)-11..)) (z; W1)) Wi,1))); Wt)). F,, is a family of functions that share an identical network structure and activation function. (W1, W2,... WI-1, Wt. determines the
2

---

Under review as a conference paper at ICLR 2020
permutate
0.8
0.2
0:5
h2
0.1
0.7
11
0.3
X
0.9
Y/1
X
J
D.7
h
0.2
h
Q
0.9
0.3
x
8,8
0.5
y22
x
0%8
0.5
J:
0.2
0.5
0.2
0.4
h
0.2
0.5
0.2
0.4
h
W
W
h2
W
R
h
W
hg
W
(a)
(b)
Figure 1: Permutation of neural networks.
function representation the neural network. We denote h; = - (x; W1)) Wi)) as the 1-th hidden vector, hy R When 9i is local linear/convolutional operation, Fz represents CN- N. When 9i is a global linear operation, FL represents an MLP. When {gi Wi) share weights, Fr. represents RNN. Because CNN can regarded a patch-based MLP and an RNN can be regarded as variant version of MLP, we consider MLP for formulation. Besides, the term function, local solution, and trained model will be used synonymously for better understanding.
2.2 RANDOM PERMUTATION OF NEURAL NETWORKS
Neurons in the intermediate layers of neural networks (termed intermediate neurons) are often sym- metric without any constraint. Neurons in the first and last layers (side neurons) manually strained by data and label structures, respectively. Under such constraints, permutating intermediate neurons and their corresponding weights could produce the same output, mentioned (Li al. 2016 We refer to them neuron permutation and weight permutation.
Permuting neurons and weights according certain rule could produce the same output. Because activation functions are often element based operations and do not have any impact on the permuta- tion of neural networks, we consider 4-layer linear MI.P for simplification, (a). The neural network takes x (2), as input and outputs tains two intermediate hidden layers, denoted as f1 (0.2, 0.7, 0.5)" and h2 = (h2, h23 hg)T The weights W1 and W2 are [0.8, 0.9; 0.1, 0.7; 0.8, 0.2]" and [0.5, 0.9, 0.5; 0.3, 0.1, 0.4; 0.4, 0.3, 0.2] (MATLAB-like notation), respectively. Now we permutate the first and second neurons of h2 and permutate the corresponding columns of W1 and rows of W2. We obtain hi = (0.7, 0.2, 0.5)T and W 10 0.7; 0.8, 0.9; 0.8, 0.2)" and W² [0.9, 0.5, 0.5; 0.3, 0.4; 0.3, 0.4, 0.2)"", as shown Figure T (b). Given any input x, the output of these two functions (a) and (b) are equivalent be- (W,)² (WW))TT Neurons and weights are just like flexible nodes and wires that can be easily exchanged. Weight permutation (blue and green wires are exchanged) is exchanged in accord with neuron permutation. Due to neuron permutation and weight permutation, repeating the optimization procedure of identical neural networks would generate different permutations of equivalent local solutions even if we assume neural networks are convex. Therefore, it is important to align the weight permutation of neural networks before analyzing the similarity of local solutions of neural networks.
shown in Figure] (V1, J2) The network con-
cause W:W''' WT
3 METHOD
In this section, first provide principle for validation foundation. We then introduce rule for the alignment neural networks. Finally, we propose to learn a function feature representation based on the aligned weights.
3

---

Under review as a conference paper at ICLR 2020
3.1 LEARNING TELLS THE TRUTH PRINCIPLE
In machine learning, an effective way to learn underlying patterns or rules is to exploit labeled data points perform supervised learning. However, in some cases, it is difficult know if rule is true or an annotation approach is correct. To this, we introduce a Learning tells the truth principle. Suppose there exists a learning algorithm such that an assumptive rule learned from a training set can be also well-validated on a test set, then the rule holds. Learning tells the truth provides an assumption-leaming www.r/oon--aaaiiinaaaaiiiiiiiioi validation paradigm to validate some assumptions.
The key point of this principle is to make reasonable assumptions, assign labels to set of objects and find an effective learning algorithm. In this paper, the assumptions are that local solutions across different runs of an identical neural network converge to similar local minimum under some condi- tions (e.g., different optimizers and different neural network structures). Based on the assumptions, an intuitive annotation way is to assign the same label to local solutions of a learning task in different runs. Different learning tasks produce different local solutions and their corresponding labels. We define them as local solution labels classes. Under such assumption, we can create solution set that contains different solution classes by training different task data sets.
We next try to find an effective learning algorithm such that the assumptive rule learned from the training set can be well-validated on the test set. We discuss this step in the next section.
3.2 CHAIN ALIGNMENT RULE OF NEURAL NETWORKS
The common approach of aligning weights of neural networks is to transform (W, WW,WW,.... W,.... D2....,W-1.. WL-1. WL) into a standard form (Wr, W.. Wi W2) that is invariant to weight permutation. However,
difficult to define such an ideal standard form or directly match two solutions because the structure of local solutions is not only affected by the symmetry of neurons but also determined by the non-convex optimization algorithm. To achieve this, we attempt to eliminate the permutation factors by considering the relations between variables of different layers.
We first consider the weight W1. W1 is Ro X Y4 matrix. If we want to permutate neurons of h1 and keep the output unchangeable, we have to permute the columns of W1 and the rows of W correspondingly, illustrated in Figure Given any non-standard W1, suppose there is a column permutation matrix Q1 E Ru11x1 such that W, can be transformed into WT. We have
W1Q1 WT.
In Eq. [ we cannot directly solve W because both Wi and Q1 are unknown. Instead, we eliminate the permutation factor Q1 by
(Wi (WIQ)) = -W
(2)
Because Q1 is the permutation of the identity matrix I and thus a normalized orthogonal matrix.
Hence, Q = [d1 Xm1, We obtain
W W.WT = WTWT".
which is invariant to random permutation Q1-
We then consider W2. W₂ could be affected by the column permutation of WT and the permuta- tion of W3. Given any non-standard W2, suppose there are a row permutation matrix P2 E Rmuxms Xmi and column permutation matrix Q2 R=axx=2 such that W2 can be transformed into W We have
PaW,Q2 -W2,
(4)
where Q1 = PT because the standardization of W2 is jointly affected by Wi, as illustrated in Figure Q2 and P2 are orthogonal matrixes. Hence, QiP3 P Pa mixm Q2Q2" m2222 In Eq. it is difficult to eliminate both P2 and Q2. Combining Eqs. 1 and we obtain
WiQ1 Pa W. = W, W2Q2 WiW,
(5)
Similar to Eq. 2 eliminate the permutation factor Q2 by
(Wi (WW::o))(WWWW33))
(6)
4

---

Under review as a conference paper at ICLR 2020
Hence, we obtain
WWW = WWWgw?
In this way, we can easily generalize Eq. to the case of the 1-th layer
W.. W W = wWwW," ..W,.""
(8)
It is observed that the left of Eq. 8 is independent of permutation factors. We term Eq. g as the chain alignment rule of neural networks. Here, a chain is defined as sequence of layers of neural network that begins with the first layer. The I-th chain is from the 1-st layer to the I-th layer.
3.3 FUNCTION FEATURE LEARNING OF NEURAL NETWORKS
Data feature learning is often achieved by minimizing the distance between intra-class data points and maximizing the distance between inter-class data points. Similar to data feature learning, func- tion feature learning can be also achieved by minimizing the distance between intra-class local solutions and maximizing the distance between inter-class local solutions. Here, intra-class local solutions are a family of local solutions trained by similar procedures on the same learning task dataset. Inter-class local solutions are those who are trained on different learning task datasets. For each function (local solution) class, we repeat the training procedure m, times and thus obtain M = =E m, trained models, where N is the number of solution classes. We then use these local solutions metadata points to perform function feature learning to measure the function similarity of neural networks.
We investigate the function feature representation based on chains. For the 1-th chain, the aligned weight Wi WWWT W WT with size of (M1, n1) is reshaped into a (n1 x n1, 1) vector and then projected into a common function feature space by learning projection matrix O₁. We use the cross-entropy loss for function classification
N
L: ==
Σ V log(d)
where }) is the i-dimensional value of the one-hot label y. of represents the probability of the i-th function class of the 1-th chain. We train function elassifiers for types of chains. Note that we normalize the weights of each layer in MLP during the function feature learning. When measuring the local solution similarity of two neural networks, we extract function feature representation by using projected vectors. We normalize projected vectors and use the cosine similarity to compute the function similarity. We evaluate the chains from / = to L and find that the isometric chains of local solutions are strongly related by the SGD optimization algorithm. We empirically evaluate that local solution classification can achieve about 99% top-1 accuracy.
4 EXPERIMENT
In this section, we study the effectiveness of the proposed function feature representation of MLP, CNN, and RNN, and validate several assumptions on three datasets, i.c., CIFAR-100 (Krizhevsky & Hinton 2009), tiny ImageNet (Russakovsky et al. 2015). and NameDDta (Paszke et al.) 2017).
The assumptions in the experiments are that under some conditions the training of the same learning tasks converges to highly similar local solution structures (even though neural networks are non- convex functions) and that of different tasks does not. We use this assumption to label local solutions (weights), which are used as training data points to train classifier. The learning algorithm the chain alignment rule plus the linear projection. High test accuracy means that assumptions reasonable.
Local solution classification is to validate that under the label assumption the rule/knowledge learn- ing from a training set also holds on the test set. Local solution retrieval aims to validate if function feature representation can be used for unseen solution classes, because solution classes between training set and a test set are non overlaallgg in the retrieval setting. These protocols follow image classification and retrieval and thus have the same motivation.
5

---

Under review as a conference paper at ICLR 2020
top-1 accuracy (%
100
99.7
100
99.5
rank-J accuracy (%)
100
98.8
98.0
100
98.5
96.3
97.0
85.3
top-1 accuracy
rank-1 accuracy
una unaligned
62.9
allgned
2.4
2.8
9.7
4.4
4.0
9.5
5.3
15.3
8.8
11.0
2
3
1
2
2
3
layer/chain
layer/chain
layer/chain (c) MLP retrieval
layerichain (d) CNN retrieval
(a) MLP classification
(b) CNN classification
Figure 2: Evaluations on the tiny ImageNet Dataset. "layer" is the x-axis of the unaligned method and "chain" is the x-axis of the aligned method. The following figures the same.
Specially, Section 4.1 4.2 and 4.3 assume that under the SGD optimizer condition, local solutions of each task (different runs) are highly similar even though neural networks are non-convex functiions. In Section 4.4. the assumptions are slightly different. We change one factor of the baseline to form one new setting each time. Under new condition, we investigate if the assumption in Section and 4.3 still holds. We also study if the proposed function feature representation that is trained under the baseline condition is available under another condition. If under both conditions the classification accuracy is high, then the structure of local solutions could nearly not affected by this condition impact factor.
4.1 EVALUATIONS ON THE TINY IMAGENET DATASET
The tiny ImageNet dataset, which is drawn from the ImageNet (Russakovsky et al. 2015), has 200 classes, 64 x 64 in size. Each class has 500 training images, 50 validation images, and 50 test images. We evaluate the function features of MLP and CNN on the tiny ImageNet dataset. The goal of this experiment is validate that SGD based local solutions of learning task are highly similar even though neural networks are non-convex functions. It also aims to demonstrate that the proposed function feature learning is effective.
Local solution sets. We assume that SGD based local solutions of learning task share the same property and thus have the same label. Based on the assumption, we generate local solution sets. We train 5-layer convolutional network (PlainNet-5) and a 4-layer MLP (MLP-4) create two local solution sets for evaluation. The PlainNer-5 network consists convolutional units and one fully-comnected layer. Each convolutional unit contains one convolutional layer with kernel size of 3 x 3, one ReLU function, one BatchNorm layer, and one pooling layer. The MLP-4 consists of 4 fully connected layers (followed by one ReLU functions), among which three layers have 500 hidden neurons and one has neurons. We split 200 classes into 50 groups as 50 data subsets with solution labels 0-~49, Each data subset contains classes. For both MLP and CNN, we repeat the training procedure 100 times to obtain 100 local solutions for each data subset. We generate 5,000 local solutions (weights) for MLP-4 and PlainNet-5,, respectively.
Implementation. When generating local solution sets, we use SGD with batch size of 128. For PlainNet-5,, the learning rate starts from 0.1 and divided by 10 after 30 epochs and train for 50 epochs. For MLP-4, the learning rate starts from 0.1 and is divided by 10 after 70 epochs and we train for 100 epochs. For saving memory, we resize images into 32 32 as input and only analyze the first three chains in function feature learning (the fourth chain of CNN takes 72G GPU memory even through the batch size 1).
When training the function feature representation, we use SGD with a batch size 1. The learning rate 0.001 and is divided by 10 after epochs. We also set baseline without weight alignment. For each layer, we directly leam to classify its weights.
Local solution classification. We use local solution classification to validate the similarity of local solutions. We evaluate the performance of local solution classification on MLP and CNN. Similar to image classification, we predict the solutions labels of trained models by using the chain alignment rule and the vector projection. For each solution class, sample 60 local solutions for training while the other 40 for test. The training set contains 3,000 trained local solutions while the test set contains 2,000 local solutions. We train the function features by classifying 50 solution classes.

---

Under review as a conference paper at ICLR 2020
100
85.
98.0
96.8
top-1 accuracy (%)
100
99.3
99.6
99.2
rank accuracy (%)
100
26.c99.5
97.0
98.8
rank-1 accuracy (%6
100
98.8
99.8
95.8
top-1 accuracy
unaligned
aligned
50
50
50
2,5
2.9
12.0
6.7
7.0
18.5
9.3
15.
3
2
2
3
3
layerichain
layer/chain
layer/chain
layer/chain
(a) MLP classification
CNN classification
MLP retrieval
(d) CNN retrieval
Figure 3: Evaluations on the CIFAR-100 Dataset.
The experimental results are shown in Figure 2 (a) and (b). local solution classification, our proposed method achieves 99,4%, 99.7% and 99.1% % top-1 accuracy using MLP and 99.6%, 99.3% and 99.5% using CNN. The high performance validates the function feature representation of SGD based local solutions with different solutions labels can be closely projected into different fixed points. This suggests SGD based local solutions share similar function features and thus have strong connections. Without using the chain alignment rule, the performance drops significantly, e.g., 62.9%, 2.4% and 2.8% using MLP, 9.7%, 4.4% and 4.0% using CNN. Besides, for the baseline without alignment, the first layer also obtains good performance, because the first layer only has one column permutation variable and thus the row of the matrix contains discriminative information. We also observed that all types of chains can achieve high performance. This suggests that semantics of neural networks are hierarchical in bottom-up way. In other words, the projection directions of neural networks are arranged in order. Otherwise, the out-of-order projection directions confuse the system and thus lead to poor performance. We finally conclude that local solutions of each learning task or optimization process are highly similar and the chain alignment rule plus linear projection is an effective learning algorithm for validation.
Local solution retrieval. local solution retrieval, we use the image retrieval metric for local solu- tion retrieval evaluation, i.e., cumulative matching characteristic Gray et al. (2007). Local solution retrieval aims to validate that function feature representation can be generalized to measure the sim- ilarity of unseen local solutions. As shown in Figure 2 (c) and (d), method achieves 99.3%, 98.8% and 98.0% rank-1 accuracy using MLP. For the function feature representation of CNN, the proposed model achieves 98.5%, 96.3%, and 97. 97.0%. Without using the chain alignment rule, the performance drops to 85.3%, 9.5%, and 5.3% on MLP, 15.3%, 8,8%, and I 11.0% on CNN. The local solution retrieval results show the robustness of the function representation learning for unseen so- lution classes. And the high accuracy demonstrates the soundness of the assumption under the SGD optimizer.
4.2 EVALUATIONS THE CIFAR-100 DATASET
The CIFAR- 100 dataset (Krizhevsky & Hinton 2009), 32 x 32 in size. has 100 classes containing 600 images each. There are 500 traiiing images and 100 testing images per class. The assumptions in this section is similar to Section 4.1
Local solution sets. We train PlainNet-5 and MLP-4 to form two local solution sets on CIFAR- 100. The 100 classes of CIFAR-100 is split into 50 groups 50 data subsets with solution labels 0~49. Each data subset has 2 image classes. For each data subsets, we repeat the training procedure 100 times to obtain 100 local solutions. Finally, we obtain 5,000 local solutions for MLP and CNN, respectively.
Implementation. The implementation of CIFAR-100 is similar to that of tiny ImageNet. The structure of PlainNet-5 and MLP-4 slightly differs from previous ones because each data subset of CIFAR-100 contains 2 classes and the dimension of the last fully connected layer is 2.
Local solution classification and retrieval. In both local solution classification and retrieval, our method obtains about 98.0% top-1 accuracy and 98.0% rank- accuracy, shown in Figure These results demonstrate the soundness of the assumption and the effectiveness of our proposed function feature learning again.

---

Under review as a conference paper at ICLR 2020
Table Evaluations on the NameData Dataset using RNN.
unaligned
aligned
local solution classification
17.2% top-
100.0% top-
local solution retrieval
21.3% rank-1 1
95.7% rank-
100
100
top-1 accuracy
Baseline
rank-1 accuracy
Baseline
Res5
Res5
50
Leaky
50
Leaky
Depth
Depth
Adam
Adam
1
layer/chain
layerfchain (b) Performance different settings retrieval
(a) P Performance different settings classification
Figure 4: Performance of different settings.
4.3 EVALUATIONS ON THE NAMEDATA DATASET
The NamelData dataset (Paszke et al. 2017 contains a few thousand surnames from 18 languages of origin. is used to train a character-level RNN that can predict which language a name is from based on the spelling. The assumption similar Section 4.1 but is about Recurrent Neural Network (RNN).
Local solution sets. We train GRU (Chung et al. 2014) based RNN with GRU cells (GRU-2) on Nata A fully connected layer is added after the GRU module for classification. We do use LSTM (Sundermeyer et al., 2012) because we find that GRU is better than LSTM in our setting. We split 18 classes into 9 groups as 9 data subsets with solution labels 0~8.. Each data subset contains 2 classes. We repeat the training procedure 100 times to obtain 100 local solutions for each class. Finally, we generate 900 local solutions.
Implementation. When generating the local solution set, we use SGD with a mini-batch size of 1. The learning rate starts from 0.1 and is divided by 10 after 7,000 batches and we train for 10,000 batches. When training the function representation, we use SGD with a mini-batch size 1. The learning rate is 0.001 and is divided by 10 after 6 epochs.
Local solution classification and retrieval. We evaluate the function feature representation of RNN with solution classification and retrieval metrics as discussed in Section 4.1 With the chain rule alignment, the proposed method obtains 100.0% top-1 accuracy in the classification setting and 95.7% rank-1 in retrieval setting. Without the alignment approach, the accuracy is 17.2% in classi- fication and 21.3% in retrieval. These results demonstrate the soundness of the assumption and the effectiveness of proposed function feature learning RNN.
4.4 EFFECT OF DIFFERENT FACTORS
We then study four potential factors that could affect the assumptions made in Section
and 4.3 They are listed as follows. 1) Baseline. The baseline is implemented by PlainNet-5 with the ReLU activation function and is optimized by SGD. 2) Network depth ("Depth"). Depth is implemented by adding a convolutional unit to PlainNet-5,, referred to as PlainNet-6. 3) Network structure ("Res5"). Res5 designed as a 5-layer residual network (ResNet-5). Note that weight size is kept the same. 4) Optimizer ("Adam""). Adam is implemented by replacing the SGD opti- mizer with an Adam optimizer. The initial learning rate is 0.001. 5) Activation function ("Leaky"). Leaky is implemented by replacing all of the ReLU activation functions with the leaky ReLU func- tions. Experiments are conducted on the CIFAR- 100 dataset.
We implement four settings by changing one of four factors while keeping the other factors un- changed. For each setting, we train 5,000 local solutions (models) and the solution set is split into the training and test sets mentioned in 4.2
8

---

Under review as a conference paper at ICLR 2020
top-1 accuracy
100
Baseline
rank-1 accuracy
100
Baseline
Res5
Res5
Leaky
Leaky
50
50
Depth
Depth
Adam
Adam
layer/chain
layer/chain
(a) E Effect 4 factors on local solution in classification
(b) Effect of factors on local solution in retrieval
Figure 5: Effect of four factors on the structure of local solution.
Assumptions and conclusions under four new conditions. We investigate if the assumption under the new condition still holds four new solution sets. In Figure 4 (a) and (b), we have two obser- vations: (1) In both local solution classification and retrieval, using the residual network structure, leaky ReLU activation function adding one more layer also obtain high performance of so- lution classification and retrieval; (2) The Adam optimizer cannot achieve good performance, the reason could be that the Adam optimizer converges to unstable structures of local solutions and thus leads to dissimilar local solutions. Therefore, the assumption in this setting does not hold.
Effect of four factors the structure of local solutions. Previous discussions focus the eval- uation in a certain setting. In this experiment, we want to find out the effect of these factors on the structure of local solutions. This can be achieved by validating the effectiveness of the function feature representation across different conditions. We train models by using the baseline setting and test by another setting. In Figure 5 (a), we use local solution classification, which aims to validate if the structure of local solutions affected by these factors. The training and test sets share the same learning tasks. We find several key points. First, when applying the function feature representa- tion of PlainNet-5 to that of ResNet-5, the performance drops in the solution classification setting. That means the structure of local solutions of ResNet-5 is changed to some extent compared with PlainNet-5.. Second, replacing ReLU by Leaky ReLU adding one layer to PlainNet-5 nearly does not change the structure of local solutions because the accuracy is still high in solution classifica- tion. Third, it is observed that Adam obtains low accuracy. That is, the structure of local solutions of Adam is quite different from that of SGD, the reason could be the unstable adaptive convergence against the convexity of neural networks. In (b) we local solution retrieval, which aims to validate the effectiveness of function feature learning across conditions. It is observed that the first three factors do not affect the high accuracy of the assumptions, which shows the function fea- tures are robust across different conditions to validate these assumptions. Local solutions of Adam optimizer could converge to an unstable structure, which has been discussed. Therefore, function feature learning cannot extract shared features from them.
5 CONCLUSION
In this paper, we present Function Feature Learning (FFL) method that can measure the similarity of non-linear neural networks and thus provides crucial insights into the understanding of the relation between different local solutions of identical neural networks. FFL introduces novel chain align- ment rule for parameter alignment. FFL is used for Multi-Layer Perceptron (MLP), Convolutional Neural Network (CNN), and Recurrent Neural Network (RNN) and evaluated on three datasets. The promising results demonstrate the strong connection between different SGD based local solutions of identical neural networks and the equivalence of projected local solutions by SGD. Besides, the semantics are often presented in a bottom-up way. Finally, FFL provides more insights into the structure of local solutions.
We intend to extend FFL in several directions. First, FFL could be used measure the transferability between different learning tasks based on the similarity of local solutions. Second, FFL could be used to select diverse learners for ensemble learning based on the dissimilarity of local solutions. Third, FFL could be helpful to find better local solutions in non-convex optimization according to the strong relation between different local solutions.

---

Under review as a conference paper at ICLR 2020
REFERENCES
Junyoung Chung, Çaglar Gülgehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR, abs/1412.3555, 2014.
Alexey Dosovitskiy and Thomas Brox. Inverting visual representations with convolutional networks. In CVPR, pp. 4829-4823 2016.
Doug Gray, Shane Brennan, and Tao Hai. Evaluating appearance models for recognition, reacquisi- tion, and tracking. 2007.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. arXiv preprint arXiv:1905.00414 2019.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Tech- nical report, Citeseer, 2009.
Alex Krizhevsky, Iya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks. In Advances in neural information processing systems, pp. 1097-1105, 2012.
Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard Howard, Wayne Hub- bard, and Lawrence D Jackel. Backpropagation applied to handwritten zip code recognition. Neural computation, 1(4):541-551, 1989.
Yixuan Li, Jason Yosinski, Jeff Clune, Hod Lipson, and John E Hoperoft. Convergent learning: Do different neural networks learn the same representations? In Iclr, 2016.
Ari Morcos, Maithra Raghu, and Samy Bengio. Insights on representational similarity in neural networks with canonical correlation. In Advances in Neural Information Processing Systems, pp. 5727-5736, 2018
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
Maithra Raghu, Justin Gilmer, Jason Yosinski, and Jascha Sohl-Dickstein. Sveca: Singular vector canonical correlation analysis for deep learning dynamics and interpretability. In Advances in Neural Information Processing Systems, pp. 6076-6085, 2017.
Olga Russakovsky, Jia Deng. Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visu- al recognition challenge. International journal of computer vision, 1 115(3):211-25) 2015.
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruy Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. 2016.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional network- s: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034, 2013.
Martin Sundermeyer, Ralf Schlter, and Hermann Ney. Lstm neural networks for language modeling. In Interspeech, 2012.
Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In ECCV, pp. 818-888888 833. Springer, 2014.
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. features for discriminative localization. 2015.
Learning deep
10


Reviews:
[{"review_id": "SJgHjqzIqr", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "I first wanted to thank the authors for their proposed approach in this paper. The paper discusses an interesting idea for quantizing the similarity of neural networks, based on weight similarity. However, overall this assumption is based on the fact that similar layers within a particular architecture will learn similar semantics across different runs (and surprisingly authors add tasks to this as well, which I don't understand why this is the case). \n\nUnfortunately, the paper is hard to read. It is not easy to understand the research questions, and experimental setup. Partly due to overloaded terms, such as “solution” being used for describing multiple different concepts across the paper (solution class, local solution classification, local solution retrieval, none of them particularly well defined in the paper). I highly recommend the authors to describe their findings in a more concrete manner. I did not see any attempts at giving *insights*, mostly numerical comparison. \n\nMy concerns with the methodology of the paper are as follows:\n\n1. What are the findings of the paper? Permutation of the neural networks is certainly an area worth studying. However, in this paper, authors make the assumption that weights across the same layer (say layer #2) are somehow always going to learn similar values (except in permutation) across different runs of the model? Major clarification in this area is required. \n\n2. I am not quite sure what the term solution class means, or what the authors want the reviewer to believe it means. Please elaborate. Also solution label? This terminology seems a bit obsolete and cumbersome, unless properly defined at the beginning of the paper. \n\n3. Can you elaborate more on the goals of the experiments? Right now the outcome of the experiments are a bit vague based on lack of hypotheses. \n\n4. Can you elaborate what the goal of local solution classification is? It is not clear if this is simply the classification accuracy of a trained model, or what is vaguely described in section 3.3\n\nI will make a re-evaluation of the paper after the above questions are answered. Overall, I suggest a rewrite of the paper to make claims, experimental hypotheses and design more clear. ", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "HygT-omAtB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a method called ‘function feature learning’ which do not learn the data distribution but the parameters distribution of several neural networks types. The main idea is to generate many weights from different NNs trained with different random initializations for different subtasks and use them as training data for ‘function feature learning’. The experiments were done on three different datasets.\n\nOverall, the idea is quite interesting and new. However, I’m not 100% sure about the usefulness of the method. The authors claimed to provide more insights of neural networks with their method which I did not see when reading the paper. Furthermore, the authors used a neural networks - a black box model - to provide insights for other neural networks, also black box models. It sounds odd, doesn’t it? Moreover, one assumption from this paper that networks trained with different initializations for the same subtasks produce the same local solution is wrong. Therefore, I’m not 100% sure whether the results produced from all the experiments are trustable. \n\nIn sum, I rate this paper as a borderline paper and lean towards rejection due to several aforementioned uncertain points. ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper tackles an important problem: understanding if different NN solutions are similar or different. In the current form, however, the main motivation for the approach, and what the empirical results tell us, remains unclear. I read the paper after the updates and after reading reviews and author responses, and still had difficulty understanding the goals and outcomes of the experiments (such as what exactly is being reported as test accuracy and what is meant by: "High test accuracy means that assumptions are reasonable."). We highly recommend that the authors revisit the description of the motivation and approach based on comments from reviewers; further explain what is reported as test accuracy in the experiments; and more clearly highlight the insights obtain from the experiments. 
Decision: Reject