Information distance for neural network functions
Authors: Xiao Zhang
Dejing Dou
Ji Wu
Venue:  (2021)

Abstract: We provide a practical distance measure in the space of functions parameterized by neural networks. It is based on the classical information distance, and we propose to replace the uncomputable Kolmogorov complexity with information measured by codelength of prequential coding. We also provide a method for directly estimating the expectation of such codelength with limited examples. Empirically, we show that information distance is invariant with respect to different parameterization of the neural networks. We also verify that information distance can faithfully reflect similarities of neural network functions. Finally, we applied information distance to investigate the relationship between neural network models, and demonstrate the connection between information distance and multiple characteristics and behaviors of neural networks.

Under review as a conference paper at ICLR 2021
INFORMATION DISTANCE FOR NEURAL NETWORK FUNCTIONS
Anonymous authors Paper under double-blind review
ABSTRACT
We provide practical distance measure in the space of functions parameterized by neural networks. It is based on the classical information distance, and we propose replace the uncomputable Kolmogorov complexity with information measured by codelength of prequential coding. We also provide method for di- rectly estimating the expectation such codelength with limited examples. Em- pirically, we show that information distance is invariant with respect to different parameterization of the neural networks. We also verify that information distance can faithfully reflect similarities of neural network functions. Finally, we applied information distance to investigate the relationship between neural network mod- els, and demonstrate the connection between information distance and multiple characteristics and behaviors of neural networks.
INTRODUCTION
Deep neural networks can be trained to represent complex functions that describe sophisticated input-output relationships, such image classification and machine translation. Because the func- tions are highly non-linear and are parameterized in high-dimensionall spaces, there is relatively little understanding of the functions represented by deep neural networks. One could interpret deep models by linear approximations (Ribeiro et al. 2016), or from the perspective of piece-wise linear functions, such as (Arora et 2018).
If the space of functions representable by neural networks admits a distance measure, then it would be useful tool to help analyze and gain insight about neural networks. major difficulty the vast number of possibilities of parameterizing function, which makes it difficult to characterize the similarity given two networks. Measuring similarity the parameter space is straightforward but is restricted networks with the same structure. Measuring similarity at the output is also restricted to networks trained on the same task. Similarity representations produced by intermediate layers of networks is proved to be more reliable and consistent (Kornblith et al. 2019), but is not invariant to linear transformations and can fail in some situations, as shown in our experiments.
In this paper, provide distance measure of functions based information distance (Bennett jet al. 1998), which is independent of the parameterization of the neural network. This also removes the arbitrariness of choosing "where" to measure the similarity in a neural network. Information distance has mostly been used in data mining (Cilibrasi & Vitányi 2007: Zhang et al.) (007)). Intu- itively, information distance measures how much information is needed transform one function to the other. We rely on prequential coding to estimate this quantity. Prequential coding can efficiently encode neural networks and datasets (Blier & Ollivier 2018). we regard prequential coding as compression algorithm for neural networks, then the codelength can give an upper bound of the information quantity in a model.
We propose a method for calculating approximated version of information distance with prequen- tial coding for arbitrary networks. In this method, we use KL-divergence in prequential training and coding, which allow us directly estimate the expected codelength without any sampling process. Then we perform experiments to demonstrate that this information distance invariant to the pa- rameterization of the network while also being faithful to the intrinsic similarity of models. Using information distance, we are able to sketch a rough view into the space of deep neural networks and uncover the relationship between datasets and models. We also found that information distance can

---

Under review as a conference paper at ICLR 2021
help understand regularization techniques, measure the diversity of models, and predict model's ability to generalize.
METHODOLOGY
Information distance measures the difference between objects by information quantity. The information distance between two functions FA and fB can be defined as (Bennett et al. 1998):
df) = (fa
(1)
This definition makes of Kolmogorov complexity: ((BB(A) is the length of the shortest pro- gram that transforms fA into fB. and information distance d is the larger length of either direction. (Note that this is not the only way define information distance with Kolmogorov complexity, however we settle with this definition for its simplicity.) Intuitively, this is the minimum number of bits we need to encode fB with the help of fA, or how much information is needed know fB LA is already known.
Given two functions fa parameterized by neural network with weights A and lB. we want to estimate the information distance between fA and fB. The estimation of Kolmogorov complexity term done by calculating the codelength of prequential coding, what we get is an upper bound of d, which we denote by dp (p for prequential coding).
and JB + Y defined on the same input space X, each
2.1 ESTIMATING K (JB(JA) WITH PREQUENTIAL CODING
To send fB to somcone who already knows fA. we generate predictions ys from fø using input æy sampled from X. Assume that { known, can use prequential coding send labels {y:}. If we send enough labels, the receiver can use {2;, Vi} to train a model to recover fB-
If fA and fB have something in common, i.e. K(falfA) < (fB)), then with the help of fA we can reduce the codelength used to transmit fg. A convenient way of doing so is to use A as the initial model in prequential coding. The codelength of k samples is:
1 (
(yLL ( = Σ logPa, (111-11:)1)1))
where 0, is the parameter of the model trained on {-14-11 1, and 0, = With sufficient large k; the function parameterized by 0, would converge to f'-
If both fA and fa are classification models, we can sample from the output distribution of fa- In this case, the codelength (2) not only transmits fB, but also specific samples we draw from fB- The information contained in these specific samples Σx , (vs about estimating K ([B|JA), using the "bits-back protocol" (Hinton & van Camp. 1993) the infor- mation of samples can be subtracted from the codelength, resulting in an estimation of K K(BBfA) as Lx(fllfa)
log (vs (zz). Because we only care
[x(BafA) Σ
log Pe (3r1 V1--1) + Σ log Pa (y/ (x;))
In practice, we want use sufficiently large such that fo, can converge to fB, for example by the criterion
Ex, DD (xI (x)) Se
However, empirically found that this often means a large is is needed, which make the estimation using (3) unfeasible when the number available x small. Also the exact value (3) depends the specific samples used, introducing variance into the estimation.
2

---

Under review as a conference paper at ICLR 2021
2.2 THE PRACTICAL INFORMATION DISTANCE dp
We propose to directly estimate the expectation of L& (x(nnff)), which turns out to be much more ef- ficient the number of examples 2, by leveraging infinite y samples. The expectation of codelength B [L] over all possible samples Vick from fB is:
E Maw-f (4
Σ ynn log Po Fiss Via-i)
Σ
By log pan (3))ar))
Σ By log E Po (u Miri + 1=1
Bgg log Pas (1i ax;)
i=1
(5)
Dru(fa(c)llo,((t))
(6)
In (). Ey fa. (2:) represents an infinite ensemble of models 02 estimated from all possible sam- ples V1-i-1- We replace this ensemble with a single model 0; that is directly trained on all the samples. 0; is trained using KL-divergence as objective, which is equivalent to training with infinite samples (see Appendix A for details from (5) to (6).
The expected codelength [[[xa] related, via (6), the KI.-divergence between the outpur distri- butions of fg and fa.- Another interpretation of the above estimation is, we finetune model 04 with an increasing number of outputs generated by 0p. and aggregate the KL-divergence between the two models along the way. The more information fA shares with fB, the faster the KL-divergence decreases, resulting lower estimation of K (fs fa)-
Now dp is the approximation of ① we propose in this paper:
fB). L'xAll((
2.3 PROPERTIES OF dp
The information distance d in ① applied to functions defines metric on the space of functions. Now we check if dp satisfy the axioms metric:
1. fB) 0 + fa fs: fA fa if and only if they always produce the same predictions, which is equivalent to (n)
2. dpl(A, = dp, by definition.
3. d, fR) < (p fc) + (fc, whether dp keeps this property of d depends on the efficiency of prequential coding, which in turn depends on model optimization.
Another important property of the information distance d is the invariancy with respect to the pa- rameterization function f. We found that d, is also largely invariant to parameterization of the functions. dp can be used to compare models trained differently, having different structures, trained on different tasks. The only condition that both models should have sufficient expressibil- ity to allow approximation of each other.
There is also a connection between La (falfA) and the information transfer measure L (Zhang al. 2020):
(Or prey ( (218) (yL.M
as n + 00, On + 0.. and when y/s ~ (L((;)), we have
ELLLT (0..) =E (v1 (v120(11))
[F
3

---

Under review as a conference paper at ICLR 2021
2.4 DATA-DEPENDENCY AND EQUIVALENT INTERPRETATIONS OF DATA
In machine learning, we often only care about the output of model on the data distribution of the task. Neural network models are trained input data from specific domain, for example, image classification models take natural images in RGB format as valid input. It would be meaningless to discuss the behavior of such a model on non- non-RGB images. This is an important distinction between a neural network function and function in the mathematical sense.
This motivates take a data-dependent formulation of distance measure. In this paper, we limit our discussion to distribution distribution-depeedeent dependent information distance:
(B) max (KKA() K(BB(A)}
(9)
where
f' arg min (flffn)
f mig min (flfa)
(10)
JEJ
SETB
equivalencies of fA and fB in the below function family (0 can be A or B):
Fu D/lL {
(11)
Fo is a set containing all the functions producing outputs almost indistinguishable from Ju, in the expected sense over x drawn from data distribution D. Because they produce almost identical outputs for æ ~ D, we call them equivalent interpretations data D. Intuitively, this means that instead of transmitting fB. we can transmit f'. which is equivalent fB on D. if f'z can be transmitted in fewer bits.
A quick note why data-dependency here in the context of neural network models does not break the definition of information distance: if is a neural network trained on dataset D, then f is fully determined by the information in D plus random seed (which is of negligible information).
By introducing data- data-depen,,ncy. enables us to approximate Kolmogorov complexity by coding samples drawn from data distribution D, in other words we use the training set for coding.
EMPIRICAL STUDY
The proposed information distance dp relies on an estimation of Kolmogorov complexity of neural networks with prequential codelength, which unfortunately does not have known theoretical guaran- tees. Therefore validate the performance of dp mainly with empirical results. We use experiments to show the advantages of dp and in what situations are dp useful.
3.1 EXPERIMENT SETUP
All experiments in this section performed using ResNet-56 models (He al. 2016) Tiny- ImageNe(] a 200-class image classification dataset. To make the codelength estimation more reli- able, we try to achieve lower codelength in prequential coding by optimizing the model estimation process in sequential coding. We performed hyper-parameter search to select optimal hyper- parameters that result in the lowest codelength. Unless otherwise stated, we use h 10000 in experiments throughout this paper, which we found in most cases allow the difference between the coding model and the reference model E, (f (z))] converge.
3.2 INVARIANCES OF INFORMATION DISTANCE
prominent advantage of information distance is independent of parameterization. Neural networks like multi-layer perceptrons can have a very large number of different configurations (units, weights) that correspond the same function. Because there is no "canonical" way of parameterizing neural networks, comparing the input-output functions represented by different neural networks can be difficult by merely looking at the network parameters.
There exist many metrics for measuring the similarity distance between neural networks. But because neural networks are so versatile, similar neural networks can look similar under some metric
4

---

Under review as a conference paper at ICLR 2021
Scaling
Neuron swapping
Perturbation
2000
Param confee
Param 12
Rep CKA
Rep EMD
2000
Param cosine
Param 12
Rep OKA
Rep TMD
2000
Parsm
Param coutre
Ree EAI
Rep CLD
Param END
Out G.
Param EMD
Out RL
Param END
Cuelg
1500
L2
1500
L2
1500-
L2
oasine
auiife
1000
1000
1000
500
0.5
1.0
1.5
2.0
2.5
3.0
0.0
0.2
0.4
0.6
0.5
1.0
0.0
0.5
1.0
1.5 Scale
2.0
2.5
3.0
Coofficiant
Coofficient
Inila licationn
Architecture
Adversarial perturbation
2000
Pa L2
Rep OKA
2000
--- Rop CKA
Rep 2010
Out KL
2000
Param cosine
Param L2
Rap DGA
END
Palam EHD
KL
1500
L2
1500-
Rep L2
comtre
Rep couns.
1000
1000
500
500
500
0
from srratc
traneder learnin
Res 48
Resiee--d
Reclet. 56
Reshss-45
Restee 201
0.4
0.8
Scal
Figure 1: Distance dp changes with respect to changes in the parameterization of the networks. We measure the distance between pair of ResNet-56 models 0 and 0g. trained on two subsets of Tiny-ImageNet, respectively. We modify the configuration of A while keeping OB the same. Mod- ifications include: scaling, where we multiply the weights of the network by a coefficient c, neuron swapping. where randomly permute fraction e of the units within a layer, perturbation, where Gaussian random noise of zero mean and standard deviation e is added to the weights the network, and adversarial perturbation, where vector of standard deviation is added to the weights of the network to maximize the change of second-to to-ttst layer representations. We also include training A with different initializations and using different network architectures. Some commonly used dis- tance measures are used as baselines. These include measures in three different spaces: parameter space distances, which measure distance using metrics on the parameter matrices, including plain L2 distance, cosine similarity, and Earth Mover Distance (EMD) (Monge 1781] Rubner et al. 1998) that computes the cost to align neurons. Representation space distances measure distance on the second-to-last layer representation vectors, including plain L.2, cosine, the EMD cost of aligning feature dimensions, and Linear Centered Kernel Alignment (CKA) (Kornblith et al. 2019) which based on pairwise sample similarity and outperforms previously proposed similarity measures. For output space distance, we use the KL-divergence between the probability distribution generated by the final softmax layer. The baseline measures include both common and straightforward measures and more sophisticated measures like EMD and CKA that possess invariance properties. Underlined labels on the æ-axis denote the distances measured between unmodified 0 and 0g. Curves for each measure are individually scaled (only scaled, not shifted) to ease viewing on the same graph. Cosine similarity and CKA value within [0,1] and is inversely correlated with distance.
while very dissimilar by other metrics. There lacks a universal definition of similarity, which is precisely the problem we try solve with dy-
To empirically examine the invariancy of dp, we evaluated d, under different re-parametrrzzations of a neural network. We also include a number of distance metrics as baselines for comparison. Descriptions of test scenarios, baselines, and the results are shown in Figure I Table summarizes the observed invariancy of distance measures with quantitative measure.
Results indicates that dp is relatively stable under different kinds of re- paramtterizationn of the net- work and is the most invariant overall. Other distance measures all exhibit strong dependency on cer- tain kind of parameterization or is inapplicable for some parameterizations. For re- parameterizations that does not change or only minimally change the function f (scaling, neuron swapping, initializa- tion, architecture), dp also exhibit minimal change. For adding perturbations to the network, infor- mation distance only starts to increases when the perturbation is large enough. This is because only large noise will start "wipe out" information in the network. d, is also robust small adversarial perturbations, while also showing that adversarial perturbations destroy information in the network faster than random noise.
5

---

Under review as a conference paper at ICLR 2021
Table 1: Measuring invariancy of distance measures, Invariancy is measured by the relative change of distance in each test scenario (averaged over all data points). Lower value means more invariant. Oracle refers to an ideal distance measure. N/A means the method cannot be used in that test.
Neuron swapping
Adversarial perturbation
Space
Method
Scaling
Perturbation
Initialization
Architecture Mean
Parameter
0.51
0.16
0.55
0.14
0.62
n/a 0.39
Cosine
0.04
0.19
0.16
0.02
0.88
n/a 0.26
FMD
2.39
0,03
0.76
0.01
0.04
n/a 0.65
Representaiton
L2
0.02
0.55
0.69
0.15
0.43
n/a 0.37
Cosine
0.00
0.30
0.08
0.04
0.18
n/a 0.12
EMD
0.02
0.00
0.60
0.15
0,36
0.08
0.20
Linear CKA
0.00
0.00
0.12
0.33
0.10
0.01
0.10
Output
DKL
0.01
1.38
3.59
0.08
0.91
Other
dp
0.00
0.03
0.06
Oraele
>0,41
>0,
~0
In Initialization experiments, initialize O with network pre-trained on CIFAR- 10 (Krizhevsky & Hinton 2009). d, increase 7% compared to random initialization, because 0^ carries over some information from CiFAR 10, making A and 8g slightly less similar. In Architecture experiments, if A uses a different architecture than 8 (which more different the models are (in terms of number of layers) form ResNet-56, the distance d, also slightly higher. This indicates that while d, is largely invariant to model parameterizations, it is also consistent with intuitive similarities models. This is not observed with EMD and CKA distances.
ResNet-56), we also observe increase in dp- The
3.3
INFORMATION DISTANCE FAITHFUL?
Having seen the invariance properties of dp, we next check if dp is faithful to reflect true model distance. Empirically, we verify dp is aligned with intuitive indicators of model distance. We experiment with two settings: examining model interpolation and the model training progress. These can serve as sanity-check that dp does reflect differences the information of models.
In model interpolation use two ResNet-56 models 0 and 0g trained on Tiny-lmagenet, and perform linear interpolation in parameter space get model smoothly from A to 8g. We apply dp to measure the distance from the interpolated model to 0g for different c. The results are shown in Figure
straightforward way to manipulate the distance between models:
c) A 0g. As change from 0 1, the model 0 goes
Different task, same initialization
Same task, different initialization
2000
Param EMD --- Out KL
2000
Param EMD
Out KL
Rep CKA
Rep CKA
Method
Identifi cation
Rep
Rep
1500
1500
Param EMD
x
1000
1000
Rep FMD
/
Rep CKA
/
Out DEL
x
0.0
0.2
0.4
0.6
0.8
3.0
0.0
0.2
0.4
0.6
0.8
1.0
Coefficient
Coefficient
Figure 2: Distance from the interpolation model 9 to AB. Left: 0 and 0B are trained on different but related tasks (first 100-class and last 100-class of ImageNe) from the same initialization. Middle: 0a and 0B are trained on the same task but from different initializations. Right: whether the measure's trend is correct, and whether it can be used to identify different interpolation coefficients.
As we interpolate two functions fA and fB the parameter space, if A and 0 are parameterized similarly, observe df( fa) monotonically decrease as f gets closer I' (Figure 2 At the beginning when is small, increase in introduces more "fresh information" about fB, thus dp
To avoid clutter in graphs, did not include 1.2 and cosine measures in Figure and they fail basic invariancy in Section 3.2 Full results are given in Appendix B.2

---

Under review as a conference paper at ICLR 2021
decreases faster than later in interpolation. On the other hand, if we interpolate two functions that parameterized differently, because linear mixing of A and g in parameter space leads to a degraded network, the distance would first increase then decrease, indicating a loss of information middle in the interpolation. Overall, the general trend of dp agrees with advanced similarity measures like representation EMD and CKA in this scenario.
In the experiment training progress, we use d, to examine the training progress of a model. Starting from Og. we train the model n epochs to converge and measure the distance from the i-th epoch model 0g to the initial model 0g as well as to the final model On. Results are shown in Figure
Training process: distance start
Training process: distance end
1750
1750
Param EMD
Out
Param EMD
NL
1500
Rep CKA
d
1500
Rep CKA
Method
Trend
Identili-
1250
Rep EMD
1250
Rep EMD
cation
1000
1000
Param EMD
5
Rep EMD
750
750
Rep CKA
X
Out Die
x
250
250
dp
/
epach
epach
Figure 3: Distance from the i-th epoch model 0 to the initial model du (left) and the final model 014 (middle). Right: whether the measure has monotonic trend with respect to the training progress, and whether it can be used to identify models from different epochs.
With the training progress after each epoch, (ft: fo) steadily decreases and df fnx increases. Change in distance is faster in earlier epochs because most of the learning happens in the first few epochs. In this scenario, dp is monotonic with training and correctly depicts the training progress. Both representation EMD and CKA fail to show the dynamics of training beyond the first epoch.
To summarize, parameter space distances fail when function similarity does not correspond to pa- rameter value similarities, and representation space distances can be too noisy to be reliable when similarity is high. Only information distance d, remains faithful in both scenarios.
4 APPLICATION
To illustrate the utility of a universal function distance, provide few scenarios where dp for understanding and making predictions.
4.1 SKETCHING THE GEOMETRY OF DATA AND MODEL SPACE
distance measure can help us understand the relationship between datasets and between models. Datasets and models usually live in very high-dimensional spaces, which makes it hard directly perform a comparison. Instead, we can use dp to get the information distance between datasets and models. In computer vision there is a myriad of datasets and model structures, and use the Visual Task Adaptation Benchmark (VTAB) (Zhai et al.) 2019) as a collection of vision datasets. On each dataset, model is trained to represent the input-output function of the task. Then we use dp to measure pairwise distances between these functions. To help to visualize the relationship, we use Isometric Mapping (Tenenbaum al. 2000). manifold learning algorithm to generate three- dimensional embeddings for each function. The distance of points in three-dimensional space is optimized to keep the original structure.
Distances can tell a lot about the relationship between models. In the nine large datasets of VTAB, datasets cluster largely according to the three categories proposed VTAB (natural, specialized, and structured). CIFAR- 100 is very different from any other datasets, but is relatively closer to satellite image datasets than to artificial shape datasets. SVHN (Netzer et al., 2011) is close to 2D shape datasets. The four small datasets are evenly distributed space: no pair of them is very similar. In terms of model architecture, ResNet variants are relatively similar, while AlexNet (Krizhevsky) 2014) and VGG (Simonyan & Zisserman 2014) farther away. VGG with batch normalization
7

---

Under review as a conference paper at ICLR 2021
caltech101
dmimb
dsprites, position Asprites, orientation reelsc4S
2334
2013
2200
550
exfend
exford, litt
mmllnorb alevation
dfar100
2000
2277
1100
1000
4000
2000
2000
4000
resnet18
500
1000
reanet34
4000
reshoxt50
wide reanet5
Squeszenet
vgg11
1000
4000
4000
4000
-1002
-8000
Figure 4: Visualizing distances between datasets and models three-dimensional space. Top-left: large datasets in VTAB. Top-right: small datasets in VTAB. Bottom: various model architectures trained on ImageNet. The numbers on colored lines are pairwise distances.
closer to ResNet than without. Res.Net-50, ResNeXt-50 (Xie et al. 2017) and WedeResNet-50 Zagoruyko & Komodakis 2016) closest they indeed very similar.
4.2 UNDERSTANDING REGULARIZATIONS
Regularization techniques like L2 regularization can bias the learned neural networks toward less complex functions, while for techniques like dropout (Srivastava al. (2044) and self-distillation (Furlanello et al., 2018), the regularization effect may be less straightforward to explain. We can use dp examine the (information) complexity of network f. by measuring its distance (f 0) to empty function.
Weight decay
Label smoothing
Dropout
12000
12000
10000
10000
5000
8000
9000
6000
6000
8000
4000
10
10-2
0.0
0.2
0.4
0.0
0.1
0.2
0.3
Iterations
Coefficient
Coefficient
Figure 5: Distance to an empty function df, 0) for models with different kinds of regularization and varying strength. From left to right: Weight decay (L2 regularization), self-distillation, label smoothing, dropout. For dropout, a different base model without batch normalization is used.
we observe that all the listed techniques result a reduction of do.ff. 0), which means that the information complexity of the model function f is reduced. For weight decay, infor- mation complexity only starts to decrease after the regularization coefficient larger than a thresh- old. Self-distillation has similar effect regularization, with the number of distillation iterations controlling regularization strength. This agrees with the theoretical analysis in (Mobahi et al. 2020). Label smoothing and dropout also result in simpler models, highlighting their regularization effect.
From Figure 5
8

---

Under review as a conference paper at ICLR 2021
Loss
Accuracy
2.20
1.-
Generalization gap
2.26
0.9
archmiafle::
Inw0.0022 urchercon:220
Accuracy
2.24
Ire
0.7
Spt-
0.6
bs-120
dclout
0.5
bx-a
2.20
0012 augmon
0.3
1500
2000
2500
3000
3000
3500
4000
4500
Distance
Distance
Figure 6: Relation between ensemble perfor- mance and model diversity given by dp(fi.fa)
Figure 7: Relation between generalization gap and model complexity df.).
4.3 ENSEMBLES AND MODEL DIVERSITY
Distance can be used indicator of model diversity: the larger dp between models, the more diverse the models are. Ensembling is a common technique to use the consensus multiple models to deliver superior performance than single model. We speculate that larger model diversity will result in more performance gain from ensembles.
To verify this connection, we train number of models Tiny- Tiny-Imagenet, all to the same perfor- mance on the validation set, but with different initializations and different subsets of the training Then choose models in pairs measure their ensemble performance tance dp (f between them. The result is given in Figure 6 and we found a clear correlation between dp and ensemble performance, and the relationship is about linear. This also indicates that de captures model diversity.
well as the dis-
4.4 PREDICTING GENERALIZATION
Finally, dp is also linked with model generalization. Generalization of neural networks is heavily affected by eyper-parameters and optimization. There have been several works aiming to find the relationship between generalization performance and properties of the network, but it turns out that predicting generalization gap can challenging task (Jiang et al. 2019 2020).
We perform small-scale experiment to illustrate the connection between information distance and generalization gap. We train number of models with different hyper- syper-pprameters (batch size, learn- ing rate, optimizer, etc.) all to the same loss on the training set, and then measure the distance to random model by df, 0). In Figure 7 we observe that the information complexity the model also linked with generalization gap, which also turns out to be roughly linear relationship. Models that generalize better are farther away from a random model than less performing models.
5 DISCUSSION AND CONCLUSION
The proposed distance dp is based on information distance defined with Komolgorov complexity K. We do not attempt give good estimation of K, but instead relying on the efficiency of prequential coding, we empirically illustrate that dp share the invariance properties of information distance, and reflects the similarity relationships of functions parameterized by neural networks. We also found that dp is linked with behaviors of models, making it potential tool for analyzing neural networks.
The most notable difference between dp and other similarity metries is universality. Theoretically rooted in information distance, dp is independent of parameterization and widely applicable in sit- uations involving different tasks and models. However, d,'s utilization of prequential coding also introduces limitations that it might not work in situations where prequential coding fails, for exam- ple, when f cannot be efficiently approximated by neural networks.
parameterization-fe free geometry of space spanned by neural models. Optimization with manifold descent by d,, could also remove the dependency on parameterization, thus avoiding ill- ill-posed conditions in some parameterizations (Dinh et al.) 2017).
de could introduce potential scale-free,

---

Under review as a conference paper at ICLR 2021
REFERENCES
Raman Arora, Amitabh Basu, Poorya Mianjy, and Anirbit Mukherjee. Understanding deep neural networks with rectified linear units. In 6th International Conference on Learning Representations, ICLR 2018, 2018.
Charles Beattie, Joel Z. Leibo, Denis Teplyashin, Tom Ward, Mareus Wainwright, Heinrich Küttler, Andrew Lefraneq. Simon Green, Victor Valdés, Amir Sadik, Julian Schrittwieser, Keith Ander- son, Sarah York, Max Cant, Adam Cain, Adrian Bolton, Stephen Gaffney, Helen King, Demis Hassabis, Shane Legg, and Stig Petersen. Deepmind lab. CoRR, abs/1612.03801, 2016.
C. H. Bennett, Gacs, Ming Li, M. B. Vitanyi, and W. H. Zurek. Information distance. JEEE Transactions Information Theory, 44(4):1407-1423, 1998.
Léonard Blier and Yann Ollivier. The description length of deep learning models. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa- Bianchi and Roman Gamett (eds.), Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, pp. 2220-2230, 2018.
Gong Cheng, Junwei Han, and Xiaoqiang Lu. Remote sensing image scene classification: Bench- mark and state of the art. Proceedings of the IEEE, 105(10):1865-188) 2017.
Rudi Cilibrasi and Paul M. B. Vitányi. The google similarity distance. IEEE Trans. Knowl. Data Eng., 1703)377-338, 2007.
Mircea Cimpoi, Subhransu Maji, lasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. De- scribing textures in the wild. In 2014 IEEE Conference on Computer Vision and Pattern Recog- nilion, CVPR 2014, pp. 3606-3613, 2014.
Laurent Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio. Sharp minima can generalize for deep nets. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, ICML 2017, volume 70 of Proceedings of Machine Learning Research, pp. 1019-1028, 2017.
Tommaso Furlanello. Zachary Chase Lipton, Michael Tschannen, Laurent Itti, and Anima Anand- kumar. Born-again neural networks. In Jennifer G. Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning. ICML 2018, volume of Proceedings of Machine Learning Research, pp. 1602-1611, 2018
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Sel. Top. Appl. Earth Obs Remote. Sens., 12(7):2217-2226, 2019.
Geoffrey E. Hinton and Drew van Camp. Keeping the neural networks simple by minimizing the description length of the weights. In Lenny Pitt (ed.), Proceedings of the Sixth Annual ACM Conference on Computational Learning Theory, COLT 1993, pp. 5-13, 1993.
Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In 2017 IEEE Conference on Computer Vision and Puttern Recognition, 2017.
CVPR 2017. pp.
Forrest N landola, Song Han, Matthew W Moskewicz, Khalid Ashraf, William J Dally, and Kurt Keutzer. Squeezenet: Alexnet-level accuracy with 50x fewer parameters and < 0.5 mb model size. arXiv preprint arXiv: 2016.
Yiding Jiang, Dilip Krishnan, Hossein Mobahi, and Samy Bengio. Predicting the generalization gap in deep networks with margin distributions. 7th International Conference on Learning Representations, ICLR 2019, 2019.
10

---

Under review as a conference paper at ICLR 2021
Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. Fantastic generalization measures and where to find them. In 8th International Conference on Learning Representations, ICLR 2020, 2020.
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey E. Hinton. Similarity of neural network representations revisited. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Pro- ceedings of the 36th International Conference on Machine Learning, ICML. 2019, volume 97 of Proceedings of Machine Learning Research, pp. 3519-3529, 2019.
Alex Krizhevsky. One weird trick for parallelizing convolutional neural networks. arXiv preprint arXiv: 1rX4v:1404.599) 599479997 2014.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Tech- nical report, 2009.
Yann LeCun, Fu Jie Huang, and Léon Bottou. Learning methods for generic object recognition with invariance to pose and lighting. In 2004 IEEE Computer Society Conference Computer Vision and Pattern Recognition (CVPR 2004), pp. 97-104, 2004.
Fei-Fei Li, Robert Fergus, and Pietro Perona. One-shot learning of object categories. IEEE Trans. Pattern Anal. Mach. Intell., 18 2006.
Xuhong Li, Yves Grandvalet, Rémi Flamary, Nicolas Courty, and Dejing Dou. Representation transfer by optimal transport. arXiv preprint arXivv2007.06737, 2020.
Loic Matthey, Irina Higgins, Demis Hassabis, and Alexander Lerchner. dsprites: Disentanglement testing sprites dataset.
2017
Hossein Mobahi, Mehrdad Farajtabar, and Peter L Bartlett. Self-distillation amplifies regularization in hilbert space. arXiv preprint w0Xi:22002.05755 2020.
Gaspard Monge. Mémoire sur la théorie des déblais et des remblais. Histoire de l'Académie Royale des Sciences de Paris, 1781.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. NIPS Workshop on Deep Learning and Unsupervised Feature Learning. 2011.
Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over large num- ber of classes. In Sixth Indian Conference on Computer Vision, Graphics & Image Processing, ICVGIP 2008, pp. 722-729, 2008.
Omkar M. Parkhi, Andrea Vedaldi, Andrew Zisserman, and C. Jawahar. Cats and dogs. In 20/2 IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp.
2012.
Marco Túlio Ribeiro, Sameer Singh, and Carlos Guestrin. "why should I trust you?": Explaining the predictions of any classifier. In Balaji Krishnapuram, Mohak Shah, Alexander J. Smola, Charu C. Aggarwal, Dou Shen, and Rajeev Rastogi (eds.), Proceedings of the 22nd ACM SIGKDD International Conference Knowledge Discovery and Data Mining, 2016, pp. 1135-1144, 2016.
Yossi Rubner, Carlo Tomasi, and Leonidas J. Guibas. A metric for distributions with applications to image databases. In Proceedings of the Sixth International Conference Computer Vision (ICCV-98), pp. 59-66, 1998.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 1 15(3):211-252 2015.
Mark Sandler, Andrew G. Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chich Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, pp. 4520- 2018

---

Under review as a conference paper at ICLR 2021
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 2014.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way prevent neural networks from overfitting. Journal of Machine Learning Research, 15956):19,--195, 2014.
Joshua B Tenenbaum, Vin De Silva, and John C Langford. A global geometric framework for nonlinear dimensionality reduction, science, 290(5500):2319 2323, 2000.
Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual trans- formations for deep neural networks. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 1492-1500, 2017.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In Richard C. Wilson, Edwin R. Hancock, and William A. Smith (eds.), Proceedings of the British Machine Vision Conference 2016, BMVC 2016, 2016.
Xiaohua Zhai, Joan Puigcerver, Alexander Kolesnikov, Pierre Ruyssen, Carlos Riquelme, Mario Lucic, Josip Djolonga, Andre Susano Pinto, Maxim Neumann, Alexey Dosovitskiy, et al. A large-scale study of representation learning with the visual task adaptation benchmark. arXiv preprint
2019.
Xian Zhang, Yu Hao, Xiaoyan Zhu, Ming Li, and David R. Cheriton. Information distance from question answer. In Pavel Berkhin, Rich Caruana, and Xindong Wu (eds.), Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2007, pp. 874-883, 2007.
Xiao Zhang, Xingjian Li, Dejing Dou, and Ji Wu. Measuring information transfer in neural net- works. arXiv preprint 47X::22009.07624, 2020.
12

---

Under review as a conference paper at ICLR 2021
A
TECHNICAL DETAILS IN CALCULATING dp
In equations 9-16
we used fø. place of the ensemble model E E (3a)
&
B (4 2
(12)
Dkulfa(zi)llfa((t)) (x:)) + (ass)) log Byn- So (21))
f0, (2i)
(13)
0, is trained with objective function CKL:
(kL Σ Dkelfa(z;)llf((e))
(14)
Σ He((f(zz)) -((e()))))(((((( H(fa(a,)))
(15)
E fn
Σ
Σ H(fa(a,))
(16)
j=1
SCE [u(fo(z;))
(17)
where CC is the cross-entropy objective function used to train 0. H. stands for cross-entropy.
In other words, 0, trained with the average loss used in training 0, (the entropy term in (17) does not depend on 9 and has effect in training). Therefore fa, should mimic the behavior of the infinite ensemble fó, (x;) reasonably well and make the second term in small.
Using (6) instead of (3) to estimate codelength not only makes estimations independent of the sam- pling process, but also requires fewer input examples x. This is because we are making the most use of each by essentially drawing infinite y samples from each (xx) Generally speaking. to estimate dy. one first needs sample some input from D (based data-dependency introduced in Section 2.4). Usually is unknown, but we have a dataset S containing samples from D so that we can use examples from S instead. When the size of S is small, there may not be enough examples train fA to converge to fB by (3). We found that this often the case for small datasets, for example in computation time by using a smaller sample size k.
Section 4.1 Even when S is large,
In Section 2.4 we introduced data dependency, where we study function restricted to the data distri- bution. We can quickly see that if represents model trained data distribution D, then
K(fl', D) K K(')f. = 0
where f means f restricted to D. Because f is fully determined by f'. And
K(f'slf',, D) = K(flfA,
follows from above. We can study ((ff% f/A. D) by coding examples from D. This requires the functions to be compared (ʃn and fa) trained on the same kind of input. This is reasonable re- striction because is unlikely one would be concerned about the distance between functions defined on different input spaces.
13

---

Under review as a conference paper at ICLR 2021
B EXPERIMENT DETAILS
B.I INVARIANCE EXPERIMENTS
We provide details for re-parametreizzations used in invariance experiments:
Scaling: the weights in layer is multiplied by c, and the weights in layer multiplied by l/c. For relu networks, this keeps the output of network unchanged.
Neuron swapping: we randomly permute e-(total number of neurons) neurons in layer i. We also correspondingly permute the input of layer + that the network output is unchanged.
Perturbation: we add Gaussian noise zero mean and standard deviation e to each indi- vidual weight of the network.
we add vector e of standard deviation c to the weight vector of the network, and optimize the vector to maximize deviations of second-to-last layer representations. i.e. max [ - (x))²
Adversarial perturbation:
we experiment with random initialization and initialize with pre-trained network on another dataset (CIFAR-10).
Initialization:
Architecture: we use ResNet architecture with different width and depth. ResNet-56-s refers to ResNet-56 with half the width in each layer.
Next we list the baseline distance (or similarity) measures and describe how to calculate them for network FA and fB:
Parameter space distances: we denote the i-th layer weight matrix of network fA as w For L2 and cosine measures, we first flatten and concatenate all weight matrices w and biases of the network into a long vector wall (excluding parameters batch normalization layers, because some statistic variable in them can be large and dominate the norm of vector w).
L2: d12 1/wart wall 2
Cosine: deosine wall ahl / /[llw all
w l2).
EMD: we use "Optimal Transport of Neurons" in (Li et al., 2020). The distance matrix M is taken to the pairwise L2 distance between weights of each neuron
M Il wnn-lll
The EMD distance the optimal transport cost of matching neurons from one network neurons in the other network
demd min Pell(a))
(P,M),
where P is the optimal transport plan.
Representation space distances: we sample x; from data distribution D and denote the output repre- sentation vector of the recond-to-lase layer of model fa by fai-
L2: d12 = E
llfei - f2sll2-
Cosine: deasine
EMD: same as in "parameter space distances". except that the distance matrix M is taken to be the pairwise L2 distance between the activation vector of each neuron
Mmn (fAim - fBmn)
#=1
Linear CKA: use implementation provided by (Kornblith et 2019 II T fA.113.
deka IIfA ff
fAAlllllf II fB-
where Sa. is matrix whose vectors fAI,-., SAk-
14

---

Under review as a conference paper at ICLR 2021
Output space distances: the output distributions of fa and fg-
KL-divergence: Ey (f(Dxx((f(()lll((())))
B.2 MORE RESULTS OF SECTION 3.3
Figure and 9 shows the results in model interpolation experiments and training progress experi- ments, for all distance measures studied in this paper. We also show that if each method gives the correct trend, and whether it can be used to identify different models.
Different task, initialization
Same task, different initialization
Identifi.
Method
Trend
2000
Param EMD
Param cosine
2000
Param EMD
Param cosine
cation
Bep END
RAS Cosine'
Rep EMD
RRe "oSinE"
1500
Dut
1500
Oue kt
Param cosine
X
Param EMD
1000
1000
Rep
Rep cosine
500
500
Rep EMD
Rep CKA
Dxz
0.0
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
dp
Coefficient
Coefficient
Figure 8: Full results of Figure Distance from the interpolation model 0 to 0g.
Training process: distance to start
Training process: distance to end
Identif-
Param LMQ
Param cosine
Pacam BMD
PJPAM E85h
Method
Trend
cation
1500
Rep CKA
Rep L2
1500
Rep OKA
Rep L2
cosine
Param L2
Out e.c.rre
dg
1250
Out
Param cosine
1000
Param
1000
Param L2
Param EMD
750
750
Rep L2
X
X
Rep
Rep EMD
250
250
Rep
Out Dxz.
dp
epoch
epoch
Figure 9: Full results of Figure Distance from the i-th epoch model 0, to initial model 0g and final
model f11.
B.3 GEOMETRY EXPERIMENT'
From the 19 datasets included in VTAB (Zhai al. 2019), we were able download 13 datasets for using in this work. Because the dataset size vary greatly among the 13 datasets, we divide them into two groups: larger datasets (size > 10000), which include:
cifar100: CIFAR-100 Krizhevsky Hinton 2009
svhn: SVHN (Netzer et al.
2011
eurosar: EuroSAT (Helber et al.
2019
resisc45: Resisc45 (Cheng et al.
2017)
dsprites positiio: dSprites/location (Matthey et al. 2017)
dSprites/orientation
smallnorh azimuth: SmallNOORB/azimuth (eeCCn et al. 2004
smallnorb_elevation SmallINORB/cllvatiionn
dmlah: DML.ab (Beattie et 2016
and smaller datasets (size 10000), which include:
15

---

Under review as a conference paper at ICLR 2021
caltech101: Caltech101 (Li al. 2006)
dtd: DTD (Cimpoi et al. 2014)
oxford 102: Flowers 102 (Nilsback & Zisserman 2008
oxttrr Pets (Parkhi al. 2012)
For larger datasets, use k 10000 as with other experiments. For smaller datasets, we use 2000. Model is ResNet-56 trained from scratch.
In model geometry experiments, we use the following models trained on ImageNet, provided by torchvision
resnet18: ResNet-18 (He et al. 2016
resmet34: ResNet-34
resmet50: ResNet-50
vggl1: VGG-11 without batch normalization (Simonyan & Zisserman, 2014)
vggll VGG-11 with batch normalization
alexnet: AlexNet (Krizhevsky
2014
resnex150: ResNeXt-50-32x4d (Xe et 2017)
ride_resmet50: WideResNet
(Zagoruyko & Komodakis 2016
densener/21: Densenet-12)
(Huang al. 2017
xqueezenet: SqueezeNet 1.1 (landola al.
2016)
mobilenet: MobileNet V2 Sandler et al.
2018
Codelength is calculated on the training set of ILSVRC 2012
(Russakovsky et al. 2015).
B.4 ENSEMBLE EXPERIMENTS
We train multiple ResNet-56 models with 2 different random initializations and half of the examples sampled form the TinylmageNet training set. This means the training examples seen by each model can overlap from 0% to 100%. We then select two models. Ensemble performance and distance between the two models measured by df f8) given Table We list pairs with same or different initialization, and with different overlap in training examples. Generally speaking, the less the training examples overlaps the larger the distance between the models. Different initialization can also makes the models more dissimilar. Note that from Figure 6, we that distance (f fB) is correlated with ensemble performance regardless of whether diversity comes from difference in training examples or difference initializations.
models out of this collection and ensemble the
Table 2: Details of configurations in ensemble experiments
Initialization seed
Training overlap
Ensemble loss
Ensemble accuracy
dn(fA, fD)
0%
2.197
47.22
2502
same
25%
2.211
46.59
2234
same
50%
2.229
46.40
1946
same
75%
2.229
45.93
1569
different
2.192
47.09
2957
different
25%
2,201
47.09
2481
different
50%
2.214
47.06
2195
different
75%
2.235
45.99
1772
different
100%
2.257
45.75
1436
16

---

Under review as a conference paper at ICLR 2021
B.5 GENERALIZATION EXPERIMENTS
We run experiments on CIFAR-I0 with different hyperparameters and model configurations, and in all configurations we train the model to cross-entropy loss of 0.1 training set. Then we measure the generalization gap as the loss on testing set minus the loss on training set. Starting from a default configuration (which the same hyperparameters we use in other experiments in this paper). each time we modify one of the hyperparameters. Results are listed in Table [3] In terms of studying generalization gap, our experiments is far less thorough than in (Jiang et al., (020), but here we would like to illustrate the connection between dp with generalization gap under different experiment settings, without spending too many machine hours.
Table 3: Details of configurations in generalization experiments.
Learning rate
Data augmentation
Generalization
Configuration
Architecture
Optimizer
Batch size
dp(S.0)
gap
default
ResNet-56
Adam
32
0.002
off
0.32
4605
1==0.0002
ResNet-56
Adam
32
0,0002
off
0.80
3045
In=0.02
ResNet-56
Adam
32
0.02
off 0.77
3322
bs=8
ResNet-56
Adam
0.002
off
0.50
3908
bym128
ResNet-56
Adam
off
opt=SGD
ResNet-56
SGD
32
0.2
off
0.68
3344
arch=resner20
ResNet-20
Adam
32
0.002
off
0.79
3348
arch=L.NNtt
LeNet-5
Adam
32
0.002
off 0.84
2873
data-aug=on
ResNet-56
Adam
0,002
0.62
3615
17


Reviews:
[{"review_id": "tKm1GWSXzc7", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nThe authors proposed a distance measure to characterize the information space among neural networks. They derive this measure by empirically compute the expected code length. They applied the distance measure to neural network models in didfferent settings and compared with some baselines.\n\n\nStrength:\n\n+ the information theoretic perspective of the deep learning is a useful approach to compare neural networks.\n+ the spectrum of the empirical study is pretty wide, covering some interesting aspects of neural network behaviors.\n+ the connection with generalization of neural networks is an interesting observation.\n\nWeakness:\n\n- lack of important details on the experiments and related work. Despite briefly introduced the baselines in the Appendix, the main text should be self-contained. Why are these baselines chosen is also not clear. Without a clear description of the baselines, experiments, or codes greatly hinders the reading and evaluation of the results.\n- limiting novelty: the concept of computing code length is not news in deep learning (Blier et al., 2018; Kim et al., 2018), and the authors are only incrementally extending the complete formulation of information transfer in Zhang et al., 2020 into the expected codelength. It is not clear the contribution, both theoretically and empirically.\n- why is eq 10 a valid assumption? if not, the entire basis for this measure is not solid.\n- missing definition of math notations: e.g. what is f_square in eq 14? the technical sections are below standard to top-tier conferences.\n- didn't show a clear benefit over existing methods: section 3.2 explored the invariances of the measure in different parameterization settings of neural networks. however, we see that many baselines are equally invariant if not better than the proposed measure.  even in section 3.3, we see EMD performing just as well in capturing the change in training process.\n- lack of comparison with other methods in Fig 4. It is likely that other similarlity measures offers interesting visualizations like this in three-dimensional space as well. Why is this method better in any way?\n- several acronyms used without introducing their definition first.\n\n\n\nSuggestion:\n\nThis submission appears to only preliminary work to a potentially interesting approach to understand the behaviors of deep learning. \n\nThe writing is a little bit hard to follow, mostly due to a lack of details in the empirical evaluations from section 3 onward. I think it would require some significant rewriting before ready for publication.\n\nThe contribution of the work is also a bit confusing. It feels like the heavy lifting of the idea actually comes from the formulation of information transfer in Zhang et al., 2020. The empirical study also didn't show any clear benefit of the proposed approach over existing ones. \n\n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "eSqDDe41bJx", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: \n\nThe authors provide a practical distance measure among different neural networks. They extend the classical information distance by replacing the uncomputable Kolmogorov complexity in terms of code length of prequential coding. Empirically, they show several practical advantages of the proposed distances.  \n\nPros: The proposed distance may be easy to estimate. \n\nCons:\n\n1. For the proposed information distance definition and invariance property, are there any concrete examples or analytical formulas to demonstrate its effectiveness? E.g., if a simple linear function gives the neural network, what is the concrete value of this distance. \n\n2. The distance is data-dependent. Suppose the data is given by a particularly known distribution, even Gaussian, \nis there a mathematical way to demonstrate the defined quantities? \n\nSome sentences need to be revised:\n\n`\"` Information distance dp is based on information distance defined with Komolgorov complexity K.\"\n\nSome questions:\n\n1. In literature, there are already many works in studying the distance and geometry associated with the neural networks, such as Fisher information geometry (S. Amari) and Wasserstein information geometry (W. Li). What is the relation between this new distance based on prequential coding's codelength and two distances mentioned above? Especially the role that KL-divergence plays in this framework?\n\nLi, Zhao, Wasserstein information matrix. \n\nAmari, Matsuda, Wasserstein statistics in one-dimensional location-scale model. \n\n\n2. Fisher information metric is known to be invariant under parameterization and is even characterized as the unique metric on probability space to have this property. In your paper, you mention that this new practical distance is also invariant under parameterization. Does this have something to do with the classical Fisher metric?\n\n3. To obtain the data-dependency definition of information distance, do you have to calculate an empirical version of it? If this is the case, what is the convergence properties, such as the convergence rate of this empirical information distance?", "extra_scores": {}, "extra_text": {}}, {"review_id": "YABwDP7OLQD", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper explores the problem of designing a distance measure in the space of the functions parameterized by neural networks. Ideally, such a measure should be independent of the parameterization of the networks. Also, the measure should support quantifying the distance between the networks with different structures and/or different underlying training tasks. The information distance meets this natural requirement. However, information distance is computational infeasible as it is defined by Kolmogorov complexity.\n\nThis paper utilizes the length of prequential coding to approximate the information distance which leads to a practical distance measure for neural networks. The paper empirically verifies that this distance measure is indeed invariant to the parameterization of the network. Furthermore, the paper shows that the distance measure explains/captures various behaviors of neural networks, e.g., the effect of regularization, connection between model diversity and performance for network ensembles, and generalization.\n\nThe paper studies an interesting and timely problem. The empirical results demonstrate the value of prequential coding based distance measure for understanding the behavior of neural networks. That said, there are some concerns about the novelty and presentation of the paper:\n1. Prequential coding has been previously used in the context of neural networks, e.g., [1], which limits the novelty of this paper.\n2. There is significant room for improvement in term of the presentation of the paper:\n   i) Consider organizing the text in Section 2 (before Section 2.1) in various subsections. This would help streamline the presentation. E.g., prequential coding can be introduced in a subsection. Similarly, it is not clear why the discussion on information transfer measures is necessary after (6). Perhaps, the authors can introduce (11) after (6) and then talk about information transfer measures in a subsection.\n    ii) The last two paragraphs of Section 2.2 are not very clear. Please consider expanding those paragraphs to make the message more clear.\n    iii) Please use the expansion of EMD and CKA the first time these terms are used.\n\n######## Post rebuttal ###########\n\nThank you for your response and the efforts to improve the presentation of Section 2. After going through other reviewers' comments and the authors' responses to those, I am comfortable with my original score. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "Kf8I1McNAI", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "1, Summary of contribution:\nThe paper proposes a new distance measure between a pair of neural networks, which is invariant to the reparameterization. Specifically, it introduces prequential coding to approximate the Kolmogorov complexity between two neural networks, and \nThe paper also conducts empirical studies on the properties of the proposed distance. (especially the relationships to model diversity and generalization in the section 4.3) \n\n\n2, Strengths and Weaknesses:\nThe idea of introducing Kolmogorov complexity is interesting, and many interesting ablation studies are done.  At the same time,  there seems to be a mistake in the important step of their theoretical justification.  Also, although the paper presents many empirical studies,  it does not extensively compare the proposed distance to other distances on a theoretical basis, and this makes it difficult for the readers to understand the position of this study in the field. \n\n3, Recommendation:\nBecause the validity of  “distance” in an application is rather subjective and depends on the tasks,  the significance of a study in this field most likely hinges on theoretical soundness and the depth of theoretical analysis. Because there seems to be a flaw in the theoretical soundness, I suggest rejection. \n\n4, Reasons for Recommendation: \nThere appears to be a mistake in the equation (4)-(6), which is the core of the justification of the proposed distance.  \nIn equation (4)-(6),  the authors endeavor to take an expectation with respect to {(x_i,  y_i)}.  However, in the transition from LHS of (4) to  RHS of (4), the authors seem to be forgetting the fact that theta_i is also a variable of {(x_m, y_m) ; k =1,...i-1}.  \nIn other words,   E_{x_{0:i}, y_{0:i}} [ log p_{theta_i} (y_i | x_{1:i},  y_{1:i-1}) ]  is not   E_{x_{i}, y_{i} } [ log p_{theta_i} (y_i | x_{1:i},  y_{1:i-1}) ] \nOn this basis, I do not believe that (4) can be simplified into (6).  \n\nIf the problem raised in 4 can be resolved, I would very much like to change my rate. Please let me know if there is some misunderstanding on my part.\n \nAlso, because the theoretical analysis is an important part of this research, more theoretical studies on the properties of the proposed distance.\n\n\\\n---Post rebuttal---\n\nThank you very much for the response, and I understood that some of the concerns I raised can be resolved empirically under appropriate conditions.   After the response, however, I think I am still comfortable with the original score, because this is (as I understood it) a theoretical paper and I felt that it is necessary to make a judge based on theoretical solidness. \n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All the reviewers agree that the paper studies an important and interesting problem. However the reviewers felt the paper is still in preliminary stages, with incorrect derivations, missing comparisons/references,  and writing. While the authors updated the paper during the discussion stage addressing some of the concerns, the paper still needs more work in adding appropriate comparisons and in presenting the concepts more clearly. Hence I believe the paper is not yet ready for publication and encourage authors to continue their work.
Decision: Reject