Original: T OWARDS I NTERPRETING D EEP N EURAL N ETWORKS VIA U NDERSTANDING L AYER B EHAVIORS
Revised: T OWARDS I NTERPRETING D EEP N EURAL N ETWORKS VIA U NDERSTANDING L AYER B EHAVIORS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Deep neural networks (DNNs) have achieved unprecedented practical success in many applications.
Revised: Deep neural networks (DNNs) have achieved unprecedented practical success in many applications.

Original: However, how to interpret DNNs is still an open problem.
Revised: However, how to interpret DNNs is still an open problem.

Original: In particular, what do hidden layers behave is not clearly understood.
Revised: In particular, what do hidden layers behave is not clearly understood.

Original: In this paper, relying on a teacher-student paradigm, we seek to understand the layer behaviors of DNNs by “monitoring" both across-layer and single-layer distribution evolution to some target distribution in the training.
Revised: In this paper, relying on a teacher-student paradigm, we seek to understand the layer behaviors of DNNs by “monitoring” both across-layer and single-layer distribution evolution.

Original: Here, the “across-layer" and “singlelayer" considers the layer behavior along the depth and a speciﬁc layer along training epochs , respectively.
Revised: Here, the “across-layer” and “single-layer” considers the layer behavior along the depth and a speciﬁc layer along training epochs , respectively.

Original: Relying on optimal transport theory, we employ the Wasserstein distance ( W -distance) to measure the divergence between the layer distribution and the target distribution.
Revised: Relying on optimal transport theory, we employ the Wasserstein distance ( W -distance) to measure the divergence between the layer distribution and the target distribution.

Original: Theoretically, we prove that i) the W -distance  of across layers to the target distribution tends to decrease along the depth.
Revised: Theoretically, we prove that i) the W -distance between the distribution of any layer and the target distribution tends to decrease along the depth.

Original: ii)  the W -distance of a speciﬁc layer to the target distribution tends to decrease along training iterations.
Revised: ii) For a speciﬁc layer, the W -distance between the distribution in an iteration and the target distribution tends to decrease along training iterations.

Original: iii)
Revised: iii)

Original: However, a deep layer is not always better than a shallow layer for some samples.
Revised: However, a deep layer is not always better than a shallow layer for some samples.

Original: Moreover, our results helps to analyze the stability of layer distributions and explains why auxiliary losses helps the training of DNNs.
Revised: Moreover, our results helps to analyze the stability of layer distributions and explains why auxiliary losses help the training of DNNs.

Original: Extensive experimentson real-world datasets justify our theoretical ﬁndings.
Revised: Extensive experiments justify our theoretical ﬁndings.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Deep neural networks (DNNs) have been successfully applied invarious computer vision tasks, such as image classiﬁcation (Chen et al., 2019b; Hsu et al., 2019), image generation (Brock et al., 2019;
Revised: Deep neural networks (DNNs) have been successfully applied in computer vision, such as image classiﬁcation (Chen et al., 2019b; Hsu et al., 2019), image generation (Cao et al., 2019;

Original:  Chrysos et al., 2019) and speech recognition (Yeh et al., 2019; Chen et al., 2019a).
Revised: Brock et al., 2019; Chrysos et al., 2019) and speech recognition (Yeh et al., 2019; Chen et al., 2019a).

Original: Despite their success, the internal mechanism of DNNs is still a black box.
Revised: Despite their success, the internal mechanism of DNNs is still a black box.

Original: In particular, understanding what do hidden layers do and how they achieve remarkable performance remain persistently elusive.
Revised: In particular, understanding what do hidden layers do and how to achieve remarkable performance remain persistently elusive.

Original: To answer these questions, we seek to understand across-layer and single-layer behavior.
Revised: To answer these questions, we seek to understand across-layer and single-layer behavior.

Original: For the across-layer behavior, wefocus on study the learning process by measuring the W -distance between the distribution in the layer and the target distribution.
Revised: For the across-layer behavior, we study the learning process by measuring the W -distance between the distribution of any layer and the target distribution.

Original: Recently, most methods only focus on ﬁnal predictions in different tasks (He et al., 2016).
Revised: Recently, most methods only focus on ﬁnal predictions in different tasks (He et al., 2016).

Original: Due to the end-to-end training, interpreting each intermediate layer behaviors of a DNN, which, however, is still not clear.
Revised: Due to the end-to-end training, interpreting each intermediate layer behaviors of a DNN, which, however, is still not clear.

Original: To provide interpretability, existing works try to produce a single prediction and observe the classiﬁcation performance of each layer (Papernot & McDaniel, 2018; Szegedy et al., 2013; Kaya et al., 2019).
Revised: To provide interpretability, existing works try to produce a single prediction and observe the classiﬁcation performance of each layer (Papernot & McDaniel, 2018; Szegedy et al., 2013; Kaya et al., 2019).

Original: For example, (Alain &
Revised: For example, (Alain &

Original: Bengio, 2016) experimentally observe that the linear separability of features increases monotonically along the depth of a DNN.
Revised: Bengio, 2016) experimentally observe that the linear separability of features increases monotonically along the depth of a DNN.

Original: Unfortunately, there is no theoretical analysis to support the experimental ﬁnding.
Revised: Unfortunately, there is no theoretical analysis to support the experimental ﬁnding.

Original: To understand the learning process of DNNs, one can explore the across-layer behavior by monitoring how the distributions propagate across different layers.
Revised: To understand the learning process of DNNs, one can explore the across-layer behavior by monitoring how the distributions propagate across different layers.

Original: However, how to open the internal mechanism of DNNs and investigate the across-layer behavior of a DNN remains an open question.
Revised: However, how to open the internal mechanism of DNNs and investigate the across-layer behavior of a DNN remains an open question.

Original: For the single-layer behavior, we seek to measure the W -distance between the distribution in some iteration and the target distribution.
Revised: For the single-layer behavior, we seek to measure the W -distance between the distribution in an iteration and the target distribution.

Original: it is important to analyze the distributional stability of one layer by measuring the change of the distributions.
Revised: It is important to analyze the distributional stability of one layer by measuring the change of the distributions.

Original: Recently, the distributional stability of a deep neural network attracts extensive attention (Santurkar et al., 2018).
Revised: Recently, the distributional stability of a deep neural network attracts extensive attention (Santurkar et al., 2018).

Original: Some studies try to visualize the training behavior of one layer by plotting the mean and variance of features (Santurkar et al., 2018).
Revised: Some studies try to visualize the training behavior of one layer by plotting the mean and variance of features (Santurkar et al., 2018).

Original: Unfortunately, the visualizations are subjective and lack of necessary theoretical justiﬁcations.
Revised: Unfortunately, the visualizations are subjective and lack of necessary theoretical justiﬁcations.

Original: To this end, (Sonoda & Murata, 2019) propose a transport analysis method and state that a denoising
Revised: To this end, (Sonoda & Murata, 2019) propose a transport analysis method and state that a denoising

Original: Autoencoder transports mass to decrease the Shannon entropy of the data distribution.
Revised: Autoencoder transports mass to decrease the Shannon entropy of the data distribution.

Original: However, the
Revised: However, the

Original: [Figure]
Revised: [Figure]

Original: method is limited and inﬂexible for analyzing a general case of deep neural networks.
Revised: method is limited and inﬂexible for analyzing a general case of deep neural networks.

Original: Therefore, it is very necessary and important to develop a new analytical method to interpret the stability of layers.
Revised: Therefore, it is very necessary and important to develop a new analytical method to interpret the stability of layers.

Original: In this paper, we apply optimal transport theory to analyze the behavior of distributions.
Revised: In this paper, we apply optimal transport theory to analyze the behavior of distributions.

Original: Speciﬁcally, we exploit Wasserstein divergence ( W -distance) to measure the difference between a distribution of any layer and the target distribution.
Revised: Speciﬁcally, we exploit Wasserstein divergence ( W -distance) to measure the difference between the distribution of any layer and the target distribution.

Original: By monitoring the change of  W -distance, we are able to study both across-layer and single-layer behaviors.
Revised: By monitoring the change of the W -distance, we are able to study both across-layer and single-layer behaviors.

Original: Our contributions are summarized as follows.
Revised: Our contributions are summarized as follows.

Original: • We analyze the across-layer behavior and prove that  W -distance between every layer distribution  and the target distribution decreases along the depth of a DNN.
Revised: • We analyze the across-layer behavior and prove that the W -distance between the distribution of any layer and the target distribution decreases along the depth of a DNN.

Original: It means that every layer of the network can express the target distribution progressively.
Revised: It means that every layer of the network can express the target distribution progressively.

Original: Upon this property, deep layers behave better when shallow layers behave well. • We analyze the single-layer behavior and prove that  W -distance between distributions of any layer decreases across the training iterations when introducing  loss in the layer.
Revised: • We analyze the single-layer behavior and prove that for a speciﬁc layer, the W -distance between the distribution in an iteration and the target distribution decreases across the training iterations when introducing a loss in the layer.

Original:  Moreover, we provide experiments and theoretical justiﬁcations on these ﬁndings.
Revised: • Moreover, we provide experiments and theoretical justiﬁcations on these ﬁndings.

Original: • The proposed analytical framework can facilitate future researches to interpret a deep neural network.
Revised: The proposed analytical framework provides a different view in understanding and interpreting neural networks.

Original: Experiments on real-world datasets justify our theoretical ﬁndings.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: At the feature level, many studies analyze the features of intermediate layers.
Revised: Many studies analyze the features of intermediate layers.

Original: Network Dissection (Bau et al., 2017) evaluates how hidden units align a set of semantic concepts to quantiﬁes the interpretability of latent representations of a CNN.
Revised: (Bau et al., 2017) evaluates how hidden units align a set of semantic concepts to quantiﬁes the interpretability of latent representations of a CNN.

Original: (Dosovitskiy & Brox, 2016) proposes a method to invert image representations with up-convolutional networks for studying image representations.
Revised: (Dosovitskiy & Brox, 2016) inverts image representations with up-convolutional networks for studying image representations.

Original: Based on optimizing an objective function with gradient descent, (Mahendran & Vedaldi, 2015) proposes a framework to invert shallow and deep representations.

Original: (Zeiler & Fergus, 2014) presents a visualization technique to investigate the function of intermediate feature layers and the classiﬁer.
Revised: (Zeiler & Fergus, 2014) presents a visualization technique to investigate the function of intermediate feature layers and the classiﬁer.

Original: (Zhang et al., 2018) proposes to learn an explanatory graph to reveal the knowledge hierarchy hidden inside a pre-trained CNN.
Revised: (Zhang et al., 2018) learns an explanatory graph to reveal the knowledge hierarchy hidden inside a pre-trained CNN.

Original: (Zhang et al., 2019) learns a decision tree to clarify the speciﬁc reason for each semantic prediction of a CNN.
Revised: (Zhang et al., 2019) learns a decision tree to clarify the speciﬁc reason for each semantic prediction of a CNN.

Original: At the performance level, some studies analyze the classiﬁcation accuracy of a deep neural network.
Revised: Some studies analyze the classiﬁcation accuracy of a DNN.

Original: For example, deeply supervised networks (Lee et al., 2015) introduces a classiﬁcation objective to the individual hidden layers, in addition to the overall objective at the output layer.
Revised: (Lee et al., 2015) introduces a classiﬁcation objective to the individual hidden layers, in addition to the overall objective at the output layer.

Original: Inception architecture (Szegedy et al., 2015) uses auxiliary classiﬁers to improve the classiﬁcation performance.
Revised: (Szegedy et al., 2015) uses auxiliary classiﬁers to improve the classiﬁcation performance.

Original: Shallow-deep network (Kaya et al., 2019) introduces internal classiﬁers into off-the-shelf DNNs to understand overthinking of networks by studying how the prediction changes during a DNN’s forward pass.
Revised: (Kaya et al., 2019) introduces internal classiﬁers into off-the-shelf DNNs to understand overthinking of networks by studying how the prediction changes during a DNN’s forward pass.

Revised: (Gupta & Schütze, 2018) explains recurrent neural networks by understanding the layer-wise semantic accumulation behavior.

Original: (Sonoda & Murata, 2019) investigates the feature map inside a DNN by tracking the transport map, and prove that a deep Gaussian DAE transports mass to decrease Shannon entropy of the data distribution.
Revised: (Sonoda & Murata, 2019) investigates the feature map inside a DNN by tracking the transport map, and prove that a deep Gaussian DAE transports mass to decrease Shannon entropy of the data distribution.

Original: (Alain & Bengio, 2016) experimentally observes that the linear separability of features increases monotonically along the depth of a DNN.
Revised: (Alain & Bengio, 2016) experimentally observes that the linear separability of features increases monotonically along the depth of a DNN.

Original: However, there is no theoretical result to analyze this phenomenon.
Revised: However, there is no theoretical result to analyze this phenomenon.

Revised: Existing studies using information bottleneck methods (Tishby &

Revised: Zaslavsky, 2015; Bang et al., 2019) mainly analyze the dynamics of across different layers.

Revised: However, it is hard for these methods to analyze the dynamics of a speciﬁc layer through different iterations.

Revised: In contrast, our proposed method is able to analyze both single-layer and across-layer behaviors.

Original:  P ROBLEM S ETTING
Revised:  P ROBLEM S ETTING

Original: Notation.
Revised: Notation.

Original: Speciﬁcally, we use bold lower-case letters ( e.g. , x ) to denote vectors, and bold uppercase letters ( e.g. , X ) to denote matrices.
Revised: We use bold lower-case letters ( e.g. , x ) to denote vectors, and bold upper-case letters ( e.g. , X ) to denote matrices.

Original: We denote the transpose of a vector ( e.g. , x T ) or matrix ( e.g. , X T ) by the superscript T .
Revised: We denote the transpose of a vector ( e.g. , x T ) or matrix ( e.g. , X T ) by the superscript T .

Original: Let 1 :=[1 , . .
Revised: Let 1 :=[1 , . .

Original: , 1] T and 1 S ( · ) be an indicator function of a set S . For two matrices A and B , the single-product is (cid:104) A , B (cid:105) = tr( A T B ) .
Revised: , 1] T and 1 S ( · ) be an indicator function of a set S , and let [ n ]= { 0 , 1 , . . . , n } .

Original: Let ∗ be the convolution operator, grad bethe gradient operator.
Revised: Let ∗ be the convolution operator, grad be gradient operator.

Original: Label distribution learning.
Revised: Label distribution learning.

Original: In this paper, we consider a  multi-label classiﬁcation problem.
Revised: In this paper, we consider a label distribution learning problem (Geng, 2016), especially multi-label classiﬁcation.

Original: Without loss of generality, multi-label can be treated as a probability distribution.
Revised: In this setting, the label distribution is deﬁned as a probability distribution to cover a certain number of labels, representing the degree to which each label describes the instance (see Figure 1 (c)). Note that the sum of the label distribution is equal to 1.

Original: Speciﬁcally, given training samples S , we seek to learn a model f from data space X into the label space Y .
Revised: Speciﬁcally, given training samples S , we seek to learn a model f from data space X into the label space Y .

Original: In Figure 1(a), for L -layered neural networks, we denote ˜ f 0: l = ˜
Revised: In Figure 1(a), for L -layered neural networks, we denote ˜ f 0: l = ˜

Original: f l ◦ · · · ◦ ˜ f 0 , l ≤ L as the output of the l -th layers, then the label distribution mapping can be deﬁned as f l := h  ◦ ˜ f 0: l , where h  is a probability function ( e.g. , FC+softmax (Frogner et al., 2015)).
Revised: f l ◦ · · · ◦ ˜ f 0 , l ≤ L as the output of the l -th layers, then the label distribution mapping can be deﬁned as f l := h l ◦ ˜ f 0: l , where h l is a probability function ( e.g. , FC+softmax (Frogner et al., 2015)).

Revised: Note that f l , l ∈ [ L ] have the same input domains and the same output domains.

Original: Our goal is to learn a model F l to let the predicted distribution  µ l close to the target distribution µ .
Revised: Our goal is to learn a model f l to let the predicted distribution (cid:98) µ l close to the target distribution µ .

Original: Then, we optimize the empirical risk:
Revised: Then, we optimize the empirical risk:

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: where E S [ · ] is the expectation w.r.t.
Revised: where E S [ · ] is the expectation w.r.t.

Original: the training set S , f l # denotes the pushforward operator of µ 0 , and d ( · , · ) is some distribution divergence, such as Cross-entropy (CE) loss and Wasserstein loss.
Revised: the training set S , f l # denotes the pushforward operator of µ 0 , and d ( · , · ) is some distribution divergence, such as Cross-entropy (CE) loss and Wasserstein loss.

Revised: In practice, the label distribution is obtained by training with CE loss.

Revised: Because of the convexity of CE loss, we derive the optimal label distribution for given features of every layer (Alain & Bengio, 2016).

Revised: In this sense, the label distribution reﬂects the actual distribution of feature maps in a speciﬁc layer.

Original: Label distribution propagation.
Revised: Label distribution propagation.

Original: In this paper, our goal is to explore how the layer distributions propagate in the across-layer and single-layer of a DNN.
Revised: In this paper, our goal is to explore how the layer distributions propagate in the across-layer and single-layer of a DNN.

Original: Speciﬁcally, we measure the divergence ( e.g. , Wasserstein distance (Villani, 2008)) to observe how the layer distributions change across different layers or iterations.
Revised: Speciﬁcally, we measure the divergence ( e.g. , Wasserstein distance (Villani, 2008)) to observe how the layer distributions change across different layers or iterations.

Original: In Figure 1(b), the distribution can be propagated in the across and single layer.

Original: Therefore, we explain the behaviors of layer from the following two perspectives.
Revised: In Figure 1(b), we explain the behaviors of layer from the following two perspectives.

Original: For the across layers, measuring the distance between each layer distribution  and the target distribution helps to understand the learning process and distribution expression ability.
Revised: For the across layers, measuring the distance between the distribution of any layer and the target distribution helps to understand the learning process and distribution expression ability.

Original: For the single layers, measuring the distance between the distribution of some training iterations and the target distribution helps to understand the dynamic behavior and training process of a DNN.
Revised: For a single layer, measuring the distance between the distribution in an training iteration and the target distribution helps to understand the dynamic behavior and training process of a DNN.

Original:  W ASSERSTEIN D ISTRIBUTION P ROPAGATION Q UANTIFICATION
Revised:  W ASSERSTEIN D ISTRIBUTION P ROPAGATION Q UANTIFICATION

Original: Optimal transport.
Revised: Optimal transport (OT).

Original: In this section, we introduce optimal transport (Villani, 2008) to quantify the discrepancy of label distributions.
Revised: Recently, the performance of multi-label classiﬁcation can be improved by using W -distance (Frogner et al., 2015; Zhao & Zhou, 2018). However, DNNs on the multi-label classiﬁcation task still lack of theoretical understanding. With the help of OT theory (Villani, 2008),

Original: Compared with other measures, such as Kullback–Leibler divergence and Hellinger distance, which are widely used to measure the distance between probabilityvalued outputs.
Revised: W -distance can be used to interpret DNNs.

Original: However, these distribution divergences often ignore any metric structure of distributionsince they treat the output dimensions independently (Frogner et al., 2015).
Revised: In contrast, some measures, such as Jensen–Shannon divergence, which, however, often ignore any metric structure of distribution (Frogner et al., 2015).

Original: Deﬁnition 1 ( W -distance (Villani, 2008))
Revised: Deﬁnition 1 ( W -distance (Villani, 2008))

Original: Given a prediction distribution (cid:98) µ and the target distribution µ , and the cost matrix C deﬁned as C κ,κ (cid:48)
Revised: Given a prediction distribution (cid:98) µ and the target distribution µ , and the cost matrix C deﬁned as C κ,κ (cid:48)

Original: =
Revised: =

Original: d p K ( κ, κ (cid:48) ) with the metric d K , where κ, κ (cid:48) are label tags, then  W -distance seeks to ﬁnd a transportation matrix T by transporting a mass (cid:98) µ to µ ,[Equation]
Revised: d p K ( κ, κ (cid:48) ) with the metric d K , where κ, κ (cid:48) are label tags, then the W -distance seeks to ﬁnd a transportation matrix T by transporting a mass (cid:98) µ to µ ,

Original: [Equation]
Revised: [Equation]

Original: where Π( (cid:98) µ, µ ) is the set of couplingscomposed of joint probability distributions with their marginals ˆ µ and µ , i.e., Π( (cid:98) 
Revised: where Π( (cid:98) µ, µ ) is the set of couplings and deﬁned as Π( (cid:98) µ, µ ) = { T [Equation] K × K +

Original: [Equation]
Revised: [Equation]

Original: In practice, the distance  in the cost matrixbetween label tags can be calculated by using word2vec (Mikolov et al., 2013) in the Euclidean space.
Revised: In practice, the distance of two label tags in the cost matrix can be calculated by using word2vec (Mikolov et al., 2013) in the Euclidean space.

Original: In practice, we calculate W -distance with entropic regularization by Sinkhorn algorithm (Knight, 2008).
Revised: We calculate W -distance with entropic regularization by Sinkhorn algorithm (Knight, 2008) with p =2 .

Original: See more details in Supplementary materials.
Revised: See more details in Supplementary materials.

Original: Distribution propagation measure.
Revised: Distribution propagation measure.

Original: In this paper, we use W -distance to measure the acrosslayer and single-layer label distribution propagation, as shown in Figure 1(b).
Revised: In this paper, we use W -distance to measure the acrosslayer and single-layer label distribution propagation, as shown in Figure 1(b).

Original: For an acrosslayer, we measure W -distance between the distribution in every layer and the target distribution, i.e. , W  ( (cid:98) µ 1 , µ ) , . . .
Revised: For an acrosslayer, we measure W -distance between the distribution of any layer and the target distribution, i.e. , W 2 ( (cid:98) µ 1 , µ ) , . . .

Original: , W  ( (cid:98) µ L , µ ) .
Revised: , W 2 ( (cid:98) µ L , µ ) .

Original: In this way, we can measure how the distributions propagate across different layers.
Revised: In this way, we can measure how the distributions propagate across different layers.

Original: For the single-layer, we measure W -distance between the distribution in every training iteration and the target distribution, i.e. , W  ( (cid:98) µ (0) l , µ ) , . . .
Revised: For the single-layer, we measure W -distance between the distribution in an training iteration and the target distribution, i.e. , W 2 ( (cid:98) µ (0) l , µ ) , . . .

Original: , W  ( (cid:98) µ ( t ) l , µ ) .
Revised: , W 2 ( (cid:98) µ ( t ) l , µ ) .

Original: Therefore, we can study the behavior of distributions in the same layer.
Revised: Therefore, we can study the behavior of distributions in the same layer.

Original: [Figure]
Revised: [Figure]

Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: … … … … … … … … … … … 𝑓 1 ′ 𝑓 2 ′ 𝑓 ∞′ 𝑓 1 ′ 𝑓 2 ′ 𝑓 𝐿 ′ 𝑓𝑓𝑓𝑓𝑓 𝑙
Revised: … … … … … … … … … … … 𝑓 1 ′ 𝑓 2 ′ 𝑓 ∞′ 𝑓 1 ′ 𝑓 2 ′ 𝑓 𝐿 ′ 𝑓𝑓𝑓𝑓𝑓 𝑙

Original:  T EACHER - STUDENT FRAMEWORK FOR LAYER B EHAVIOR A NALYSIS
Revised:  T EACHER - STUDENT FRAMEWORK FOR LAYER B EHAVIOR A NALYSIS

Original: We ﬁrst analyzeone-layer behaviors for different iterations, then we analyze distribution behaviors across different layers for inﬁnitely and ﬁnitely deep neural networks.
Revised: Teacher-student analysis framework is widely used to analyze and understand DNNs (Tian, 2017; Du et al., 2018; Cuturi, 2013). For the one-layer behavior, this framework helps to understand the dynamic of student network from the teacher network. For the multi-layer behavior, this framework helps to study the ability of the student network to express distributions of the teacher network.

Revised: Therefore, teacher-student analysis framework is important and necessary to understand DNNs.

Original: 5. O NE - LAYER B EHAVIOR A NALYSIS
Revised: 5. O NE - LAYER B EHAVIOR A NALYSIS

Original: Reformulation of the training loss.

Original: In our problem setting, the target distribution in the training loss (1) can be represented by an output of a teacher network (T-net).
Revised: In our problem setting, the target distribution in the training loss (1) can be represented by an output of a teacher network (T-net).

Original: Our goal is to make the output of the student network (S-net) to be close to the output of T-net, as shown in Figure 2 (a).
Revised: Our goal is to make the output of the student network (S-net) to be close to the output of T-net, as shown in Figure 2 (a).

Original: Therefore, we propose a teacher-student analytical framework to interpret a neural network.

Original: Speciﬁcally, given a S-net f and T-net f (cid:48) , our goal is to ﬁnd a function f that minimizes the following training loss, 
Revised: Speciﬁcally, given a S-net f and T-net f (cid:48) , our goal is to ﬁnd a function f that minimizes the following training loss, [Equation]

Original: [Equation]
Revised: [Equation]

Original: (3) where (cid:101) x
Revised: ) where (cid:101) x

Original: = x + ε , ε ∼ ν t is the corruption of data, and ν t is a noise distribution w.r.t.
Revised: = x + ε , ε ∼ ν t is the corruption of data, and ν t is a noise distribution w.r.t.

Original: the variance t I , i.e. , ν t = N (0 , t I ) .
Revised: the variance t I , i.e. , ν t = N (0 , t I ) .

Original: By optimizing the training loss, the distribution in one layer can be monitored by using a transport map.
Revised: By optimizing the training loss, the distribution in one layer can be monitored by using a transport map.

Original: Next, we provide the deﬁnition of transport map below.
Revised: Next, we provide the deﬁnition of transport map below.

Original: Deﬁnition 2 ( Transport map )
Revised: Deﬁnition 2 ( Transport map )

Original: A transport map f t : R m → R m can be deﬁned as f t ( x )=
Revised: A transport map f t : R m → R m can be deﬁned as f t ( x )=

Original: x + g t ( x ) with an update direction g t when t> 0 , and f 0 ( x )= x when t =0 .
Revised: x + g t ( x ) with an update direction g t when t> 0 , and f 0 ( x )= x when t =0 .

Original: Based on the deﬁnition of the transport map, we analyze the layer behavior of ResNet (He et al., 2016), i.e. , T-net is a residual block.
Revised: Based on the deﬁnition of the transport map, we analyze the layer behavior of ResNet (He et al., 2016), i.e. , T-net is a residual block.

Original: Note that our analysis method can be extended to a more general case.
Revised: Note that our analysis method can be extended to a more general case.

Original: Then, we solve the optimal transport map as follow.
Revised: Then, we solve the optimal transport map as follow.

Original: Theorem 1 (Optimal transport map)
Revised: Theorem 1 (Optimal transport map)

Original: If the teacher network is a residual unit f (cid:48) ( (cid:101) x )= x + σ ( (cid:101) x ) , where σ contains FC layer and an activation function.
Revised: If the teacher network is a residual unit f (cid:48) ( (cid:101) x )= x + σ ( (cid:101) x ) , where σ contains FC layer and an activation function.

Original: When ν t := N ( 0 , t I ) , the minimum f ∗ t is
Revised: When ν t := N ( 0 , t I ) , the minimum f ∗ t is

Original: [Equation]
Revised: [Equation]

Original: The minimizer f ∗ t updates x along the update direction g t ( x ) .
Revised: The minimizer f ∗ t updates x along the update direction g t ( x ) .

Original: Therefore, the optimal transport map f ∗ t with time t transports the mass at the activation x toward x + g t ( x ) .
Revised: Therefore, the optimal transport map f ∗ t with time t transports the mass at the activation x toward x + g t ( x ) .

Original: To associate with continuity equation (Sonoda & Murata, 2019), we introduce Wasserstein gradient ﬂow (WGF) as follows.
Revised: To associate with continuity equation (Sonoda & Murata, 2019), we introduce Wasserstein gradient ﬂow (WGF) as follows.

Original: Proposition 1 ( Wasserstein gradient ﬂow ) Assume µ t satisﬁes the continuity equation with the gradient vector ﬁeld ∂ t µ t = −∇· [ µ t ∇ V t ] of a potential function V t , and if the function F satisﬁes ∂ t F ( µ t )=
Revised: Proposition 1 ( Wasserstein gradient ﬂow ) Assume µ t satisﬁes the continuity equation with the gradient vector ﬁeld ∂ t µ t = −∇· [ µ t ∇ V t ] of a potential function V t , and if the function F satisﬁes ∂ t F ( µ t )=

Original: (cid:82) R m − V t ( x )[ ∂ t µ t ]( x ) d x , then WGF coincides with ∂ t µ t = − grad F ( µ t ) .
Revised: (cid:82) R m − V t ( x )[ ∂ t µ t ]( x ) d x , then WGF coincides with ∂ t µ t = − grad F ( µ t ) .

Original: Using the deﬁnition of WGF, we explore how the distribution propagate in one layer as follows.
Revised: Using the deﬁnition of WGF, we explore how the distribution propagate in one layer as follows.

Original: Theorem 2 Based on the equivalent condition in Theorem 2 (Belavkin, 2016), and let ∆ be the
Revised: Theorem 2 Based on the equivalent condition in Theorem 2 (Belavkin, 2016), and let ∆ be the

Original: Laplacian operator.
Revised: Laplacian operator.

Original: At the initial moment t → 0 , the pushforward f # µ t with Gaussian distribution satisﬁes the backward heat equation (BHE), and evolves according to Wasserstein gradient ﬂow:
Revised: At the initial moment t → 0 , the pushforward f # µ t with Gaussian distribution satisﬁes the backward heat equation (BHE), and evolves according to Wasserstein gradient ﬂow:

Original: [Equation]
Revised: [Equation]

Original: Note that W is the potential function.
Revised: Note that W is the potential function.

Original: Theorem 2 states that by introducing a loss in a layer, the W -distance can be decreased in the layer.
Revised: Theorem 2 states that by introducing an auxiliary loss in a layer, the W -distance can be decreased in the layer.

Original: It means that the label distribution can be close to the target distribution across training iterations.
Revised: It means that the label distribution can be close to the target distribution across training iterations.

Original: 5. M ULTI - LAYER B EHAVIOR A NALYSIS
Revised: 5. M ULTI - LAYER B EHAVIOR A NALYSIS

Original: (i) Inﬁnitely deep neural networks.
Revised: (i) Inﬁnitely deep neural networks.

Original: For an inﬁnitely DNN, we assume the number of layers and hidden units and the size of dataset is inﬁnite.
Revised: For an inﬁnitely DNN, we assume the number of layers and hidden units and the size of dataset is inﬁnite.

Original: Speciﬁcally, let 0= t 0 <t 1 < · · · <t L +1 = t , and let f 0 be trained on µ 0 with the variance t 1 − t 0 , then x 1 := f 0 ( x 0 ) and its distribution is µ 1 :
Revised: Speciﬁcally, let 0= t 0 <t 1 < · · · <t L +1 = t , and let f 0 be trained on µ 0 with the variance t 1 − t 0 , then x 1 := f 0 ( x 0 ) and its distribution is µ 1 :

Original: = f 0# µ 0 .
Revised: = f 0# µ 0 .

Original: Then, we train f 1 on µ 1 with the variance t 2 − t 1 .
Revised: Then, we train f 1 on µ 1 with the variance t 2 − t 1 .

Original: In the same way, we obtain f l ( x l ) and µ l := f l − 1# µ l − 1 .
Revised: In the same way, we obtain f l ( x l ) and µ l := f l − 1# µ l − 1 .

Original: The composition of residual blocks can be written as f t 0:
Revised: The composition of residual blocks can be written as f t 0:

Original: L ( x ):=
Revised: L ( x ):=

Original: f L ◦· · ·◦ f 0 ( x ) , where t is total time.
Revised: f L ◦· · ·◦ f 0 ( x ) , where t is total time.

Original: Then, the velocity of the composition coincides with the vector ﬁeld ∂ t f t = t l 0: l ( x )= ∇ V t l ( x ) .
Revised: Then, the velocity of the composition coincides with the vector ﬁeld ∂ t f t = t l 0: l ( x )= ∇ V t l ( x ) .

Original: Furthermore, we extend one-layer behavior analysis to the case of inﬁnite layers, as shown in Figure 2 (b).
Revised: Furthermore, we extend one-layer behavior analysis to the case of inﬁnite layers, as shown in Figure 2 (b).

Original: Based on Theorem 1, we have the dynamical systems as follows.
Revised: Based on Theorem 1, we have the dynamical systems as follows.

Original: Lemma 1 When the noise distribution is a Gaussian distribution, the continuous residual units is deﬁned as a ﬂow ϕ t : R m → R m of the following dynamical systems associated with the vector ﬁeld ∇ V t , i.e., ddt x ( t )=
Revised: Lemma 1 When the noise distribution is a Gaussian distribution, the continuous residual units is deﬁned as a ﬂow ϕ t : R m → R m of the following dynamical systems associated with the vector ﬁeld ∇ V t , i.e., ddt x ( t )=

Original: ∇ log( µ t ( x ( t )))
Revised: ∇ log( µ t ( x ( t )))

Original: + σ ( x ( t )) , where t ≥ 0 and µ t := ϕ t # µ 0 .
Revised: + σ ( x ( t )) , where t ≥ 0 and µ t := ϕ t # µ 0 .

Original: Based on the proofs of Theorem 2, we can achieve a similar conclusion as follows.
Revised: Based on the proofs of Theorem 2, we can achieve a similar conclusion as follows.

Original: Theorem 3 Based on the equivalent condition in Theorem 2 (Belavkin, 2016), and when the noise distribution is a Gaussian distribution, then the pushforward measure µ t :=
Revised: Theorem 3 Based on the equivalent condition in Theorem 2 (Belavkin, 2016), and when the noise distribution is a Gaussian distribution, then the pushforward measure µ t :=

Original: ϕ t # µ 0 evolves according to Wasserstein gradient ﬂow as follows: ddt µ t [Equation]
Revised: ϕ t # µ 0 evolves according to Wasserstein gradient ﬂow as follows: ddt µ t ( x )

Original:  − [Equation] [Equation] , µ 
Revised: = − grad W 2 [ µ t , µ ] ,

Original: [Equation]
Revised: [Equation]

Original: Theorem 3 states that  W -distance can be decreased across different layers.
Revised: Theorem 3 states that the W -distance can be decreased across different layers.

Original: It means that the label distribution can be close to the target distribution along the depth of the layer.
Revised: It means that the label distribution can be close to the target distribution along the depth of the layer.

Original: (ii) Finitely deep neural networks.
Revised: (ii) Finitely deep neural networks.

Original: When the number of layers and the hidden unit number are ﬁnite, the above analysis is not available.
Revised: When the number of layers and the hidden unit number are ﬁnite, the above analysis is not available.

Original: As shown in Figure 2 (c), given a composition of L T-nets, we try to analyze the ability of each layer of S-net to express the distribution of T-net.
Revised: As shown in Figure 2 (c), given a composition of L T-nets, we try to analyze the ability of each layer of S-net to express the distribution of T-net.

Original: Hence, we derive an approximation bound for ﬁnitely deep neural networks.
Revised: Hence, we derive an approximation bound for ﬁnitely deep neural networks.

Original: Here, Barron function is present a
Revised: Here, Barron function is present a

Original: T-net and its deﬁnition is provided as follows.
Revised: T-net and its deﬁnition is provided as follows.

Original: Deﬁnition 3 ( Barron function (Sonoda & Murata, 2019))
Revised: Deﬁnition 3 ( Barron function (Sonoda & Murata, 2019))

Original: The function ϕ is Barron function on
Revised: The function ϕ is Barron function on

Original: B if ϕ satisﬁes
Revised: B if ϕ satisﬁes

Original: [Equation]
Revised: [Equation]

Original: where (cid:107) w (cid:107) B
Revised: where (cid:107) w (cid:107) B

Original: = sup x ∈B |(cid:104) w , x (cid:105)| and (cid:98) φ is the Fourier transform of φ , and F B is the set of functions with the Fourier inversion formula holds on
Revised: = sup x ∈B |(cid:104) w , x (cid:105)| and (cid:98) φ is the Fourier transform of φ , and F B is the set of functions with the Fourier inversion formula holds on

Original: B .
Revised: B .

Original: Based on the deﬁnition of Barron function, (Barron, 1993) states that a Barron function can be approximated by a neural network with one hidden layer.
Revised: Based on the deﬁnition of Barron function, (Barron, 1993) states that a Barron function can be approximated by a neural network with one hidden layer.

Original: Given a ﬁxed composition of L Barron functions, we derive the error bound of the approximation between such composite function and a neural network with l hidden layers.
Revised: Given a ﬁxed composition of L Barron functions, we derive the error bound of the approximation between such composite function and a neural network with l hidden layers.

Original: Note that l is not necessarily equal to L .
Revised: Note that l is not necessarily equal to L .

Original: The approximation bound of w.r.t.
Revised: The approximation bound of w.r.t.

Original: Wasserstein distance can be provided as follows.
Revised: Wasserstein distance can be provided as follows.

Original: Theorem 4 ( Wasserstein distribution approximation ) Given a data distribution µ 0 and a function ϕ
Revised: Theorem 4 ( Wasserstein distribution approximation ) Given a data distribution µ 0 and a function ϕ

Original: i : R m i − 1 → R m i , and let L l = log( l )+1 ,
Revised: i : R m i − 1 → R m i , and let L l = log( l )+1 ,

Original: if support ( µ 0 ) ⊂K 0 and ϕ i ( K i − 1 ) ⊆K
Revised: if support ( µ 0 ) ⊂K 0 and ϕ i ( K i − 1 ) ⊆K

Original: i , 1 ≤ i ≤ l , the function ϕ
Revised: i , 1 ≤ i ≤ l , the function ϕ

Original: i is (cid:16) L l − 1 − 1 L l (cid:17) -Lipschitz and is a Barron function, i.e., ϕ 1 ∈ Ω K 0 ( C 0 ) and
Revised: i is (cid:16) L l − 1 − 1 L l (cid:17) -Lipschitz and is a Barron function, i.e., ϕ 1 ∈ Ω K 0 ( C 0 ) and

Original: ϕ i ∈
Revised: ϕ i ∈

Original: Ω K
Revised: Ω K

Original: [Equation]
Revised: [Equation]

Original: B
Revised: B

Original: [Equation]
Revised: [Equation]

Original: C
Revised: C

Original: [Equation]
Revised: [Equation]

Original: ) , then there exists a network f with l hidden layers with (cid:6) 4 C
Revised: ) , then there exists a network f with l hidden layers with (cid:6) 4 C

Original: [Equation]
Revised: [Equation]

Original:  m
Revised: l m

Original: [Equation]
Revised: [Equation]

Revised: L

Original: [Equation]
Revised: [Equation]

Original: neurons on the i -th layer,
Revised: neurons on the i -th layer,

Original: [Equation]
Revised: [Equation]

Original: where D l is the diameter of the set K l , and (cid:15), δ, s> 0 are parameters.
Revised: where D l is the diameter of the set K l , and (cid:15), δ, s> 0 are parameters.

Original: Theorem 4 provides an error bound when using a neural network with l hidden layers to approximate a ﬁxed composition of L Barron functions.
Revised: Theorem 4 provides an error bound when using a neural network with l hidden layers to approximate a ﬁxed composition of L Barron functions.

Original: Moreover, the upper bound decreases with the increasing of layers under certain conditions.
Revised: Moreover, the upper bound decreases with the increasing of layers under certain conditions.

Original: Here, we can conclude that deep layers are not always better than shallow layer for some speciﬁc samples.
Revised: Here, we can conclude that deep layers are not always better than shallow layer for some speciﬁc samples.

Original: Such samples are often classiﬁed correctly in the shallow layer rather than the deep layer, then the diameter D l can be large for this set of samples.
Revised: Such samples are often classiﬁed correctly in the shallow layer rather than the deep layer, then the diameter D l can be large for this set of samples.

Original: Hence, shallow layers can behave better than deep layers for these kinds of samples.
Revised: Hence, shallow layers can behave better than deep layers for these kinds of samples.

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: object labels per image.
Revised: object labels per image.

Original: For the evaluation metrics, we report the classiﬁcation accuracy, CF1, OF and mean average precision (mAP) in the experiments.
Revised: For the evaluation metrics, we report the classiﬁcation accuracy, CF1, OF and mean average precision (mAP) in the experiments.

Original: Implementation Details.
Revised: Implementation Details.

Original: All experiments are implemented by PyTorch on a single NVIDIA TITAN
Revised: All experiments are implemented by PyTorch on a single NVIDIA TITAN

Original: X GPU.
Revised: X GPU.

Original: Speciﬁcally, we use SGD optimizer with the learning rate as 0.001.
Revised: Speciﬁcally, we use SGD optimizer with the learning rate as 0.001.

Original: We experiment with
Revised: We experiment with

Original: ResNet ( ? ) and VGG ( ? ) with Binary Cross Entropy (BCE).
Revised: ResNet ( ? ) and VGG ( ? ) with Binary Cross Entropy (BCE).

Original: We train the model 100 epochs with the batch size of 128.
Revised: We train the model 100 epochs with the batch size of 128.

Original: Then, we train the internal classiﬁers ( i.e. , the network h l in Figure 1(a)) forepochs.
Revised: Then, we train the internal classiﬁers ( i.e. , the network h l in Figure 1(a)) forepochs.

Original: 6. I NTER - LAYER B EHAVIOR E XPLORATION
Revised: 6. I NTER - LAYER B EHAVIOR E XPLORATION

Original: In this section, we provide theoretical and experimental analysis to study the behaviors of distributions in different layers.
Revised: In this section, we provide theoretical and experimental analysis to study the behaviors of distributions in different layers.

Original: Next, we provide a new method to improve the performance of DNNs.
Revised: Next, we provide a new method to improve the performance of DNNs.

Original: How every layer express the distribution?
Revised: How every layer express the distribution?

Original: In this experiment, we study the ability of every layer to express the target distribution for ResNet and VGG.
Revised: In this experiment, we study the ability of every layer to express the target distribution for ResNet and VGG.

Original: In this paper, the expression ability can be measured by Wasserstein distance ( W -distance) between the label distribution of every layer and the target distribution.
Revised: In this paper, the expression ability can be measured by Wasserstein distance ( W -distance) between the label distribution of every layer and the target distribution.

Original: W -distance of different layers are shown in Figure 4.
Revised: W -distance of different layers are shown in Figure 4.

Original: As suggested in Theorem 4, deep layers have smaller error bound than shallow layers, meaning that they have sufﬁcient expression ability.
Revised: As suggested in Theorem 4, deep layers have smaller error bound than shallow layers, meaning that they have sufﬁcient expression ability.

Original: This experiment veriﬁes this conclusion by observing that the deeper layers have larger W -distance than shallow layers.
Revised: This experiment veriﬁes this conclusion by observing that the deeper layers have larger W -distance than shallow layers.

Original: Moreover, W -distance tends to decrease from layer to layer.
Revised: Moreover, W -distance tends to decrease from layer to layer.

Original: The front most layers have a little different ability to express distribution since they have similar value of W -distance.
Revised: The front most layers have a little different ability to express distribution since they have similar value of W -distance.

Original: When approaching to the last layer, W -distance drops to a very small value.
Revised: When approaching to the last layer, W -distance drops to a very small value.

Original: It implies that sufﬁcient layers is able to express the target distribution.
Revised: It implies that sufﬁcient layers is able to express the target distribution.

Original: How the distribution propagate across layers?
Revised: How the distribution propagate across layers?

Original: In this experiment, we investigate how distributions propagate across different layers, which help to understand the learning process from layer to layer in a deep neural network.
Revised: In this experiment, we investigate how distributions propagate across different layers, which help to understand the learning process from layer to layer in a deep neural network.

Original: Note that we normalize the prediction of layer as the probability distribution (that is the sum is 1).
Revised: Note that we normalize the prediction of layer as the probability distribution (that is the sum is 1).

Original: Figure 5 shows that the label distribution of one sample propagates from the ﬁrst layer to the ﬁnal layer.
Revised: Figure 5 shows that the label distribution of one sample propagates from the ﬁrst layer to the ﬁnal layer.

Original: At the early learning stage, the label distribution is far away from the target distribution, then it can be close to the target distribution at the latter learning stage.
Revised: At the early learning stage, the label distribution is far away from the target distribution, then it can be close to the target distribution at the latter learning stage.

Original: Different from the decreasing tendency in Figure 4, the label distribution of one sample may not close to the target distribution progressively.
Revised: Different from the decreasing tendency in Figure 4, the label distribution of one sample may not close to the target distribution progressively.

Original: For examples, on the ﬁrst line of Figure 5, the probability of the 15 -th label is large in the shallow layer, but goes down in the middle layer.
Revised: For examples, on the ﬁrst line of Figure 5, the probability of the 15 -th label is large in the shallow layer, but goes down in the middle layer.

Original: [Figure]
Revised: [Figure]

Original:  E XPERIMENT
Revised:  E XPERIMENT

Original: Implementation Details.
Revised: Implementation Details.

Original: All experiments are implemented on PyTorch.
Revised: All experiments are implemented on PyTorch.

Original: In the training, we use SGD optimizer with the initial learning rate as 0.01.
Revised: In the training, we use SGD optimizer with the initial learning rate as 0.01.

Original: The learning rate decays by a factor of 10 for everyepochs.
Revised: The learning rate decays by a factor of 10 for everyepochs.

Original: The momentum and weight decay is set to be 0.9 and 10 − 4 , respectively.
Revised: The momentum and weight decay is set to be 0.9 and 10 − 4 , respectively.

Original: We obtain models (including ResNet-18, ResNet-50 (He et al., 2016) and VGG-16 (Simonyan & Zisserman, 2014)) pre-trained on ImageNet(Deng et al., 2009) and train them for 100 epochs with the batch size of 128.
Revised: We obtain models (including ResNet-18, ResNet-50 (He et al., 2016) and VGG-16 (Simonyan & Zisserman, 2014)) pre-trained on ImageNet(Deng et al., 2009) and train them for 100 epochs with the batch size of 128.

Original: Then, we freeze the model weights and train the internal classiﬁers ( i.e. , the network h l in Figure 1(a)) for 100 epochs.
Revised: Then, we freeze the model weights and train the internal classiﬁers ( i.e. , the network h l in Figure 1(a)) for 100 epochs.

Original: Each internal classiﬁer contains fully connected layer and Sigmoid function.
Revised: Each internal classiﬁer contains fully connected layer and Sigmoid function.

Original: We train the network using Binary Cross Entropy.
Revised: We train the network using Binary Cross Entropy.

Original: We use training set to explore the distribution propagation of the across- and single-layer.
Revised: We use training set to explore the distribution propagation of the across- and single-layer.

Revised: In addition, we set α =0 .

Original:  For simplicity, we use ResNet to represent the ResNetmodel and VGG to represent the VGG-16 model in the main paper.
Revised: 01 in Eqn. 17 to achieve the balanced results. For simplicity, we use ResNet to represent the ResNet-18 model and VGG to represent the VGGmodel in the main paper.

Original: Data Sets and Evaluation Metrics.
Revised: Data Sets and Evaluation Metrics.

Original: We conduct experiments on two benchmark multi-label classiﬁcation datasets, i.e. , VOC2007 (Everingham et al., 2010) and MS-COCO (Maas et al., 2013).
Revised: We conduct experiments on two benchmark multi-label classiﬁcation datasets, i.e. , VOC2007 (Everingham et al., 2010) and MS-COCO (Maas et al., 2013).

Original: VOC2007 (Everingham et al., 2010) contains 9,963 images from 20 object categories, which is divided into train, val and test sets.
Revised: VOC2007 (Everingham et al., 2010) contains 9,963 images from 20 object categories, which is divided into train, val and test sets.

Original: MS-COCO (Maas et al., 2013) contains 82,783 images as the training set and 40,504 images as the validation set.
Revised: MS-COCO (Maas et al., 2013) contains 82,783 images as the training set and 40,504 images as the validation set.

Original: The objects are categorized into 80 classes with about 2.9 object labels per image.
Revised: The objects are categorized into 80 classes with about 2.9 object labels per image.

Original: We use the classiﬁcation accuracy, average per-class F1 (CF1), average overall F1 (OF1) and mean average precision (mAP) as the evaluation metrics.
Revised: We use the classiﬁcation accuracy, average per-class F1 (CF1), average overall F1 (OF1) and mean average precision (mAP) as the evaluation metrics.

Original: 6. A CROSS - LAYER B EHAVIOR E XPLORATION 6.1. T HE BEHAVIORS OF LABEL DISTRIBUTIONS IN DIFFERENT LAYERS
Revised: 6. A CROSS - LAYER B EHAVIOR E XPLORATION 6.1. T HE BEHAVIORS OF LABEL DISTRIBUTIONS IN DIFFERENT LAYERS

Original: How does every layer express the distribution?
Revised: How does every layer express the distribution?

Original: Here, we study the expression ability of every layer, which is measured by Wasserstein distance ( W -distance) between the label distribution of every layer and the target distribution.
Revised: Here, we study the expression ability of every layer, which is measured by Wasserstein distance ( W -distance) between the label distribution of any layer and the target distribution.

Original: As suggested in Theorem 4, deep layers have smaller error bound than shallow layers, meaning that they have better expression ability.
Revised: As suggested in Theorem 4, deep layers have smaller error bound than shallow layers, meaning that they have better expression ability.

Original: From Figure 3, deep layers have smaller  W -distance than shallow layers, which veriﬁes the conclusion in Theorem 4.
Revised: From Figure 3, deep layers have smaller the W -distance than shallow layers, which veriﬁes the conclusion in Theorem 4.

Original: Moreover,  W -distance decreases from shallow layers to deep layers.
Revised: Moreover, the W -distance decreases from shallow layers to deep layers.

Original: The shallow layers have similar ability to express the target distribution since they have similar values of  W -distance.
Revised: The shallow layers have similar ability to express the target distribution since they have similar values of the W -distance.

Original: When approaching to the last layer,  W -distance drops to a very small value.
Revised: When approaching to the last layer, the W -distance drops to a very small value.

Original: It implies that sufﬁcient layers are able to express the target distribution.
Revised: It implies that sufﬁcient layers are able to express the target distribution.

Original: More results are shown in Section F in Supplementary materials.
Revised: More results are shown in Section F in Supplementary materials.

Original: [Table]
Revised: [Table]

Original: How does the distribution propagate across layers?
Revised: How does the distribution propagate across layers?

Original: In this experiment, we investigate how the label distribution propagate across layers to understand the learning process from layer to layer in a
Revised: In this experiment, we investigate how the label distribution propagate across layers to understand the learning process from layer to layer in a

Original: DNN.
Revised: DNN.

Original: Note that we normalize the prediction of classiﬁer as the probability distribution ( i.e. , the sum of label prediction probabilities of one sample is 1.).
Revised: Note that we normalize the prediction of classiﬁer as the probability distribution ( i.e. , the sum of label prediction probabilities of one sample is 1.).

Original: Figure 4 shows that the label distribution of one sample propagates from the ﬁrst layer to the last layer.
Revised: Figure 4 shows that the label distribution of one sample propagates from the ﬁrst layer to the last layer.

Original: In the shallow layers, the label distribution is far away from the target distribution, then it can be close to the target distribution in the deep layers.
Revised: In the shallow layers, the label distribution is far away from the target distribution, then it can be close to the target distribution in the deep layers.

Original: Different from the decreasing tendency in Figure 3, the label distribution of one sample may not close to the target distribution progressively.
Revised: Different from the decreasing tendency in Figure 3, the label distribution of one sample may not close to the target distribution progressively.

Original: For example, in the ﬁrst row of Figure 4, the probability of the 10 -th class is large in the 0 -th layer, but goes down in the 3 -th layer.
Revised: For example, in the ﬁrst row of Figure 4, the probability of the 10 -th class is large in the 0 -th layer, but goes down in the 3 -th layer.

Original: 6.1. T HE EVALUATION OF ACROSS - LAYER ’ S BEHAVIORS
Revised: 6.1. T HE EVALUATION OF ACROSS - LAYER ’ S BEHAVIORS

Original: How to evaluate the quality of each layer?
Revised: How to evaluate the quality of each layer?

Original: Existing methods attempt to visualize the feature maps to evaluate the quality of each layer.
Revised: Existing methods attempt to visualize the feature maps to evaluate the quality of each layer.

Original: However, observing the feature maps is difﬁcult because of its high dimensionality and complexity.
Revised: However, observing the feature maps is difﬁcult because of its high dimensionality and complexity.

Original: Relying on  W -distance, we can evaluate the quality of each layer by measuring the ability of expression.
Revised: Relying on the W -distance, we can evaluate the quality of each layer by measuring the ability of expression.

Original: If  W -distance decreases rapidly in a layer, it suggests that the quality of this layer is good.
Revised: If the W -distance decreases rapidly in a layer, it suggests that the quality of this layer is good.

Original: Taking ResNet on VOC in Figure 3 as an example, 0 ∼ 5 layers have similar value of  W -distance, while 6 ∼ 8 layers have quite different value of  W -distance.
Revised: Taking ResNet on VOC in Figure 3 as an example, 0 ∼ 5 layers have similar value of the W -distance, while 6 ∼ 8 layers have quite different value of the W -distance.

Original: It means that these deep layers ( i.e. , 6 ∼ 8 ) are better than the shallow layers ( i.e. , 0 ∼ 5 ).
Revised: It means that these deep layers ( i.e. , 6 ∼ 8 ) are better than the shallow layers ( i.e. , 0 ∼ 5 ).

Original: 6.1. H OW TO EXPLOIT THE ACROSS - LAYER BEHAVIORS
Revised: 6.1. H OW TO EXPLOIT THE ACROSS - LAYER BEHAVIORS

Original: Does every sample contribute equally?
Revised: Does every sample contribute equally?

Original: In practice, a deep neural network construct more complex features progressively throughout the layers (Lee et al., 2011).
Revised: In practice, a deep neural network construct more complex features progressively throughout the layers (Lee et al., 2011).

Original: An interesting question arises: does the contribution of each sample vary across different layers and training iterations?
Revised: An interesting question arises: does the contribution of each sample vary across different layers and training iterations?

Original: To answer this question, we ﬁrst deﬁne the concept of sample difﬁculty in terms of  W -distance.
Revised: To answer this question, we ﬁrst deﬁne the concept of sample difﬁculty in terms of the W -distance.

Original: As shown in
Revised: As shown in

Original: Figure 5, the samples can be divided into three categories, including easy, hard and confused.
Revised: Figure 5, the samples can be divided into three categories, including easy, hard and confused.

Original: The intuition is that: 1) Easy samples should have small  W -distance in the ﬁrst few layers, i.e. , , they should be classiﬁed correctly with high conﬁdence in a shallow layer.
Revised: The intuition is that: 1) Easy samples should have small the W -distance in the ﬁrst few layers, i.e. , , they should be classiﬁed correctly with high conﬁdence in a shallow layer.

Original: Hard samples would have large  W -distance in a deeper layer, i.e. , , it cannot be resolved at all, or only near the last layer.
Revised: Hard samples would have large the W -distance in a deeper layer, i.e. , , it cannot be resolved at all, or only near the last layer.

Original: Confused samples have small  W -distance in a shallow layer, while have large  W -distance in a deep layer.
Revised: Confused samples have small the W -distance in a shallow layer, while have large the W -distance in a deep layer.

Original: It means that although these samples can be classiﬁed correctly in the shallow layer, they still become misclassiﬁed at the last layer.
Revised: It means that although these samples can be classiﬁed correctly in the shallow layer, they still become misclassiﬁed at the last layer.

Original: Considering the difﬁculty of samples would help to interpret the training of models and design the training loss, which may speed up convergence and improve the performance eventually.
Revised: Considering the difﬁculty of samples would help to interpret the training of models and design the training loss, which may speed up convergence and improve the performance eventually.

Original: We show more results in Section ?? in Supplementary materials.
Revised: We show more results in Section I in Supplementary materials.

Original: [Figure]
Revised: [Figure]

Original: networks often occur the over-thinking issue.
Revised: networks often occur the over-thinking issue.

Original: In Table ???
Revised: In Table ???

Original: , out of *% and *% of the samples for ResNet and VGG misclassiﬁes, respectively.
Revised: , out of *% and *% of the samples for ResNet and VGG misclassiﬁes, respectively.

Original: For these samples, there are *% samples are correctly classiﬁed in the *-th layer of ResNet, and *% samples are correctly classiﬁed in the *-th layer of VGG.
Revised: For these samples, there are *% samples are correctly classiﬁed in the *-th layer of ResNet, and *% samples are correctly classiﬁed in the *-th layer of VGG.

Original: Using this phenomenon, we try to improve the performance of classiﬁcation by early existing such confused samples.
Revised: Using this phenomenon, we try to improve the performance of classiﬁcation by early existing such confused samples.

Original: Different from the strategy of SDN ( ? ), how to early exist in a DNN for a multi-label task is more challenging.
Revised: Different from the strategy of SDN ( ? ), how to early exist in a DNN for a multi-label task is more challenging.

Original: Therefore, we design a new early-exist strategy for multi-label predictions.
Revised: Therefore, we design a new early-exist strategy for multi-label predictions.

Original: In Table 1, our proposed method consistently outperforms the other methods.
Revised: In Table 1, our proposed method consistently outperforms the other methods.

Original: 6.2 I NNER - LAYER B EHAVIOR E XPLORATION
Revised: 6.2 I NNER - LAYER B EHAVIOR E XPLORATION

Original: In this section, we study how the distributions evolute in one layer for different training iterations.
Revised: In this section, we study how the distributions evolute in one layer for different training iterations.

Original: We consider to use Wasserstein distance to measure the stability of distribution when training the neural network.
Revised: We consider to use Wasserstein distance to measure the stability of distribution when training the neural network.

Original: We propose theoretical and empirical analysis for the distribution stability in one layer.
Revised: We propose theoretical and empirical analysis for the distribution stability in one layer.

Original: How stable is the distribution in one layer?
Revised: How stable is the distribution in one layer?

Original: Recently, the stability of a deep neural network attracts extensive attention.
Revised: Recently, the stability of a deep neural network attracts extensive attention.

Original: Batch Normalization is proposed to stabilize the distributions of layer distribution.
Revised: Batch Normalization is proposed to stabilize the distributions of layer distribution.

Original: The popular belief is that success of Batch Normalization stems from controlling the stability of distribution during the training.
Revised: The popular belief is that success of Batch Normalization stems from controlling the stability of distribution during the training.

Original: However, we prove that the stability of distribution is hard to be guaranteed, the detail proofs are put in Supplementary.
Revised: However, we prove that the stability of distribution is hard to be guaranteed, the detail proofs are put in Supplementary.

Original: In this paper, we use W -distance to measure the discrepancy between two distributions in one layer across different training iterations.
Revised: In this paper, we use W -distance to measure the discrepancy between two distributions in one layer across different training iterations.

Original: Motivated by this, we further measure stability of label distribution in different training iterations.
Revised: Motivated by this, we further measure stability of label distribution in different training iterations.

Original: The conventional understanding of Batch Normalization suggests that
Revised: The conventional understanding of Batch Normalization suggests that

Original: W -distance between distributions in two successive iterations should decrease.
Revised: W -distance between distributions in two successive iterations should decrease.

Original: In Figure 7, the ﬁrst layer of ResNet or VGG is hard to decrease W -distance, indicating that the label distributions alway change in different iterations.
Revised: In Figure 7, the ﬁrst layer of ResNet or VGG is hard to decrease W -distance, indicating that the label distributions alway change in different iterations.

Original: How the distribution propagate across iterations?
Revised: How the distribution propagate across iterations?

Original: In this experiment, we investigate how distri- butions propagate across different training iterations.
Revised: In this experiment, we investigate how distri- butions propagate across different training iterations.

Original: The label distribution propagation is shown in Figure 5.
Revised: The label distribution propagation is shown in Figure 5.

Original: The label distribution in the ﬁrst layers of ResNet or VGG often has great ﬂuctuation in different training iterations because the supervision is insufﬁcient when the neural network is deep.
Revised: The label distribution in the ﬁrst layers of ResNet or VGG often has great ﬂuctuation in different training iterations because the supervision is insufﬁcient when the neural network is deep.

Original: When approaching the ﬁnal layer, the supervision is sufﬁcient to decrease W -distance.
Revised: When approaching the ﬁnal layer, the supervision is sufﬁcient to decrease W -distance.

Original: These experiments justify Theorem 2 that
Revised: These experiments justify Theorem 2 that

Original: W -distance can be decreased with sufﬁcient supervision.
Revised: W -distance can be decreased with sufﬁcient supervision.

Original: Why add auxiliary loss in some layer?
Revised: Why add auxiliary loss in some layer?

Original: When the neural network is very deep, shallow layers tend to receive insufﬁcient supervision.
Revised: When the neural network is very deep, shallow layers tend to receive insufﬁcient supervision.

Original: Recently, some methods are proposed to introduce auxiliary loss to improve the performance of DNNs ( ? ).
Revised: Recently, some methods are proposed to introduce auxiliary loss to improve the performance of DNNs ( ? ).

Original: However, why introduce loss in some layers and what an important role does it play in the training are not clear.
Revised: However, why introduce loss in some layers and what an important role does it play in the training are not clear.

Original: From Theorem 2, when adding the supervision in one intermediate layer, W -distance can be decreased across the training iterations.
Revised: From Theorem 2, when adding the supervision in one intermediate layer, W -distance can be decreased across the training iterations.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we have interpreted deep neural networks (DNNs) via understanding layer behaviors.
Revised: In this paper, we have interpreted deep neural networks (DNNs) via understanding layer behaviors.

Original: With the help of optimal transport theory, we propose a new analytical framework using Wasserstein
Revised: With the help of optimal transport theory, we propose a new analytical framework using Wasserstein

Original: How to improve classiﬁcation performance?
Revised: How to improve classiﬁcation performance?

Original: We will discuss how to exploit the behaviors of different layers to improve the model performance.
Revised: We will discuss how to exploit the behaviors of different layers to improve the model performance.

Original: In practice, we observe that in deep neural networks, some samples are correctly classiﬁed in the intermediate layers but misclassiﬁed in the last layer.
Revised: In practice, we observe that in deep neural networks, some samples are correctly classiﬁed in the intermediate layers but misclassiﬁed in the last layer.

Original: For example, 35.52% and 31.04% of the samples for ResNet and VGG are misclassiﬁed on the test set of VOC2007, respectively.
Revised: For example, 35.52% and 31.04% of the samples for ResNet and VGG are misclassiﬁed on the test set of VOC2007, respectively.

Original: For these samples, 5.96% and 7.76% samples are correctly classiﬁed in all intermediate layers of ResNet and VGG, respectively.
Revised: For these samples, 5.96% and 7.76% samples are correctly classiﬁed in all intermediate layers of ResNet and VGG, respectively.

Original: Based on this phenomenon, we can improve the performance by early exiting such confused samples.
Revised: Based on this phenomenon, we can improve the performance by early exiting such confused samples.

Original: Different from the strategy of
Revised: Different from the strategy of

Original: SDN (Kaya et al., 2019), how to early exit for multi-label classiﬁcation DNNs is very challenging.
Revised: SDN (Kaya et al., 2019), how to early exit for multi-label classiﬁcation DNNs is very challenging.

Original: Therefore, we design a new early-exits strategy for multi-label prediction.
Revised: Therefore, we design a new early-exits strategy for multi-label prediction.

Original: In Table 4, our proposed method consistently outperforms the baseline methods.
Revised: In Table 1, our proposed method consistently outperforms the baseline methods.

Original: We give more details about early-exits strategy and show more results in Section H in Supplementary materials.
Revised: We give more details about early-exits strategy and show more results in Section H in Supplementary materials.

Original: 6. S INGLE - LAYER B EHAVIOR E XPLORATION
Revised: 6. S INGLE - LAYER B EHAVIOR E XPLORATION

Original: In this section, we study how the distribution of an intermediate layer change during the training process.
Revised: In this section, we study how the distribution of an intermediate layer change during the training process.

Original: We consider to use W -distance to measure the stability of distribution.
Revised: We consider to use W -distance to measure the stability of distribution.

Original: We propose theoretical and empirical analysis for the distribution stability in one layer.
Revised: We propose theoretical and empirical analysis for the distribution stability in one layer.

Original: How the distribution propagate during updating DNNs?
Revised: How the distribution propagate during updating DNNs?

Original: In this experiment, we investigate how distributions propagate when training DNNs.
Revised: In this experiment, we investigate how distributions propagate when training DNNs.

Original: From Figure 6, the label distribution in the ﬁrst layers of
Revised: From Figure 6, the label distribution in the ﬁrst layers of

Original: ResNet or VGG often ﬂuctuates signiﬁcantly due to the limited discriminative power of very shallow layers.
Revised: ResNet or VGG often ﬂuctuates signiﬁcantly due to the limited discriminative power of very shallow layers.

Original: When approaching the last layer, the supervision is sufﬁcient to decrease  W -distance.
Revised: When approaching the last layer, the supervision is sufﬁcient to decrease the W -distance.

Original: These experiments justify Theorem 2 that  W -distance can be decreased with sufﬁcient supervision.
Revised: These experiments justify Theorem 2 that the W -distance can be decreased with sufﬁcient supervision.

Original: More results are shown in Section G in Supplementary materials.
Revised: More results are shown in Section G in Supplementary materials.

Original: How stable is the distribution in one layer?
Revised: How stable is the distribution in one layer?

Original: The training stability is a key problem in DNNs and can be largely addressed by the widely used Batch Normalization (BN).
Revised: The training stability is a key problem in DNNs and can be largely addressed by the widely used Batch Normalization (BN) (Ioffe & Szegedy, 2015).

Original: However, we prove that the stability of distribution is hard to be guaranteed even with BN (See proof in supplementary).
Revised: However, we prove that the stability of distribution is hard to be guaranteed even with BN (See proof in supplementary).

Original: In this paper, we use  W -distance to measure the discrepancy between two distributions in one layer across different training iterations.
Revised: In this paper, we use the W -distance to measure the discrepancy between the distribution in an iteration and the target distribution across different training iterations in the speciﬁc layer.

Original: Motivated by this, we further measure stability of label distribution during training DNNs.
Revised: Motivated by this, we further measure stability of label distribution during training DNNs.

Original: The conventional understanding of Batch Normalization suggests that  W -distancebetween distributions in two successive iterations should decrease.
Revised: The conventional understanding of BN suggests that the W -distance should decrease.

Original: Note that, from Figure 6, the ﬁrst layer of ResNet or VGG may ﬂuctuate during the training process, which indicates that deep layers often have better ability than shallow layers to express the target distribution.
Revised: Note that, from Figure 6, the ﬁrst layer of ResNet or VGG may ﬂuctuate during the training process, which indicates that deep layers often have better ability than shallow layers to express the target distribution.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we have interpreted deep neural networks (DNNs) via understanding layer behaviors.
Revised: In this paper, we have interpreted deep neural networks (DNNs) via understanding layer behaviors.

Original: With the help of optimal transport theory, we propose a new analytical frameworkusing Wasserstein distance to investigate the layer behaviors of a DNN.
Revised: With the help of optimal transport theory, we propose a uniﬁed teacher-student analysis framework to investigate the across-layer and single-layer behaviors of a DNN.

Original: Theoretically, we prove that i) theW-distance of across layers to the target distribution tends to decrease along thedepth.
Revised: Theoretically, we prove that i) the W -distance between the distribution of any layer and the target distribution decreases along the depth.

Original: ii) theW-distance of a speciﬁc layer to the target distribution tends todecrease along training iterations.iii)
Revised: ii) the W -distance of a speciﬁc layer between the distribution in an iteration and the target distribution decreases along training iterations.

Original: However, a deep layer is not always betterthan a shallow layer for some samples. Experiments on real-world datasets justify our theoretical ﬁnding.
Revised: Extensive experiments justify our theoretical ﬁndings. The proposed analytical framework can facilitate future researches to interpret a deep neural network.

Original: S
Revised: S

Original: UPPLEMENTARY ON “T OWARDS I NTERPRETING D EEP N EURAL N ETWORKS VIA U NDERSTANDING
Revised: UPPLEMENTARY ON “T OWARDS I NTERPRETING D EEP N EURAL N ETWORKS VIA U NDERSTANDING

Original: L AYER B
Revised: L AYER B

Original: EHAVIORS
Revised: EHAVIORS

Original: ” A P RELIMINARY
Revised: ” A P RELIMINARY

Original: Notation.
Revised: Notation.

Original: Speciﬁcally, we use bold lower-case letters ( e.g. , x ) to denote vectors, and bold uppercase letters ( e.g. , X ) to denote matrices.
Revised: Speciﬁcally, we use bold lower-case letters ( e.g. , x ) to denote vectors, and bold uppercase letters ( e.g. , X ) to denote matrices.

Original: We denoteby the superscript T the transpose of a vector ( e.g. , x T ) or matrix ( e.g. , X T ).
Revised: We denote the transpose of a vector ( e.g. , x T ) or matrix ( e.g. , X T ) by the superscript T .

Original: Let 1d :=
Revised: Let 1 :=

Original:  For two matrices A and B of the same size, their inner-product is (cid:104) A , B (cid:105) =
Revised: , 1] T . For two matrices A and B of the same size, their inner-product is (cid:104) A , B (cid:105) =

Original: tr( A T B ) .
Revised: tr( A T B ) .

Original: Let ∗ be the convolution operator.
Revised: Let ∗ be the convolution operator.

Original: Let (cid:107) · (cid:107) =
Revised: Let (cid:107) · (cid:107) =

Original: (cid:107) · (cid:107) 2 denote
Revised: (cid:107) · (cid:107) 2 denote

Original: Euclidean norm on vectors in R n .
Revised: Euclidean norm on vectors in R n .

Original: For a function f , let f ∨ :=
Revised: For a function f , let f ∨ :=

Original: f
Revised: f

Original: ( − x ) .
Revised: ( − x ) .

Original: Let ∆ f = (cid:80) ni
Revised: Let ∆ f = (cid:80) ni

Original: =1 ∂ 2 f ∂ 2 x i denote the Laplacian.
Revised: =1 ∂ 2 f ∂ 2 x i denote the Laplacian.

Original: Let B n be the unit ball in n dimension, i.e. , B n = { x ∈ R n : (cid:107) x
Revised: Let B n be the unit ball in n dimension, i.e. , B n = { x ∈ R n : (cid:107) x

Original: (cid:107)≤ 1 } .
Revised: (cid:107)≤ 1 } .

Original: For two sets A and
Revised: For two sets A and

Original: B and a scalar r . Let A + B = { x + y : x ∈A , y ∈B} and r A
Revised: B and a scalar r ∈ R , let A + B = { x + y : x ∈A , y ∈B} and r A

Original: = { r x : x ∈A} .
Revised: = { r x : x ∈A} .

Original: Deﬁnition 4 ( Lipschitz continuous )
Revised: Deﬁnition 4 ( Lipschitz continuous )

Original: A function ϕ : R m → R n is called L -Lipschitz continuous if there exists a real constant L ≥ 0 such that, for all x and y w.r.t.
Revised: A function ϕ : R m → R n is called L -Lipschitz continuous if there exists a real constant L ≥ 0 such that, for all x and y w.r.t.

Original: the L 2 norm, (cid:107) 
Revised: the L 2 norm, (cid:107) ϕ

Original: [Equation] y [Equation]
Revised: ( x ) − ϕ ( y ) (cid:107) 2 ≤ L (cid:107) x − y (cid:107) 2 .

Original: Deﬁnition 5 ( Fourier transform ) For ϕ ∈
Revised: Deﬁnition 5 ( Fourier transform ) For ϕ ∈

Original: L 1 ( R ) , Fourier transform of ϕ : R n → R is deﬁned as: (cid:98) ϕ
Revised: L 1 ( R ) , Fourier transform of ϕ : R n → R is deﬁned as: (cid:98) ϕ

Original: ( w ) : = (
Revised: ( w ) :=

Original: [Equation]
Revised: [Equation]

Original: For vector-valued functions ϕ : R n → R m , we conduct component-wise Fourier transform.
Revised: For vector-valued functions ϕ : R n → R m , we conduct component-wise Fourier transform.

Original: Let (cid:98) ϕ
Revised: Let (cid:98) ϕ

Original: ∨ ( x ) := (cid:98) ϕ
Revised: ∨ ( x ) := (cid:98) ϕ

Original: ( − x ) , the inverse Fourier transform is
Revised: ( − x ) , the inverse Fourier transform is

Original: [Equation]
Revised: [Equation]

Original: Deﬁnition 

Revised: Deﬁnition

Original: For a bounded set B , the norm of a function f : R n → R can be deﬁned on a set B , (cid:107) f (cid:107) ∗B :
Revised: For a bounded set B , the norm of a function f : R n → R can be deﬁned on a set B , (cid:107) f (cid:107) ∗B :

Original: = (cid:82) R n (cid:107) w (cid:107) B | ˆ f ( w ) | d w , where (cid:107) w (cid:107) B
Revised: = (cid:82) R n (cid:107) w (cid:107) B | ˆ f ( w ) | d w , where (cid:107) w (cid:107) B

Original: = sup x ∈B |(cid:104) w , x (cid:105)| .
Revised: = sup x ∈B |(cid:104) w , x (cid:105)| .

Original: B P ROOF OF T HEOREM
Revised: B P ROOF OF T HEOREM

Original: Deﬁnition 7 ( Barron function (Sonoda & Murata, 2019))
Revised: Deﬁnition 7 ( Barron function (Sonoda & Murata, 2019))

Original: The function ϕ is Barron function on B if ϕ satisﬁes Ω B ( C )=
Revised: The function ϕ is Barron function on B if ϕ satisﬁes Ω B ( C )=

Original: { ϕ : B→ R : ∃ φ, φ | B = ϕ, (cid:107) φ (cid:107) ∗B ≤ C, φ ∈F B } , where (cid:107) φ
Revised: { ϕ : B→ R : ∃ φ, φ | B = ϕ, (cid:107) φ (cid:107) ∗B ≤ C, φ ∈F B } , where (cid:107) φ

Original: (cid:107) ∗B :
Revised: (cid:107) ∗B :

Original: = (cid:82) R m (cid:107) w (cid:107) B | (cid:98) φ ( w ) | d w , where (cid:107) w (cid:107) B
Revised: = (cid:82) R m (cid:107) w (cid:107) B | (cid:98) φ ( w ) | d w , where (cid:107) w (cid:107) B

Original: =
Revised: =

Original: sup x ∈B |(cid:104) w , x (cid:105)| and (cid:98) φ is the Fourier transform of φ , and F B is the set of functions with the Fourier inversion formula holds on
Revised: sup x ∈B |(cid:104) w , x (cid:105)| and (cid:98) φ is the Fourier transform of φ , and F B is the set of functions with the Fourier inversion formula holds on

Original: [Equation]
Revised: [Equation]

Original: i.e.,
Revised: ,i.e.,

Original: [Equation]
Revised: [Equation]

Original: Deﬁnition 8 ( Sigmoidal function )
Revised: Deﬁnition 8 ( Sigmoidal function )

Original: A sigmoidal function is a bounded measurable function σ : R → R such that lim x →−∞ σ ( x )=

Revised: A sigmoidal function is a bounded measurable function σ : R → R such that lim x →−∞ σ ( x )=

Original: and lim x → + ∞ σ ( x )=1 .
Revised: and lim x → + ∞ σ ( x )=1 .

Original: Theorem 5 (Barron theorem (Barron, 1993)) Let B⊆ R n be a bounded set, and µ be any probability measure on B , ϕ ∈ Ω B ( C ) and f ( x )= (cid:80) ni
Revised: Theorem 5 (Barron theorem (Barron, 1993)) Let B⊆ R d be a bounded set, and µ be any probability measure on B , ϕ ∈ Ω B ( C ) and f ( x )= (cid:80) ni

Original: =1 c i σ ( (cid:104) w i , x (cid:105)
Revised: =1 c i σ ( (cid:104) w i , x (cid:105)

Original: + b i ) , where σ ( · ) be a sigmoidal function, then there exist w i ∈ R d , b i ∈ R , c i ∈ R with (cid:80) ni
Revised: + b i ) , where σ ( · ) be a sigmoidal function, then there exist w i ∈ R d , b i ∈ R , c i ∈ R with (cid:80) ni

Original: =1 | c i |≤ 2 C such that
Revised: =1 | c i |≤ 2 C such that

Original: [Equation]
Revised: [Equation]

Original: We extend the above theorem to the case of the composition of Barron functions, denoted as ϕ j : i 
Revised: We extend the above theorem to the case of the composition of Barron functions denoted as ϕ j : i :=

Original: [Equation] =
Revised: ϕ j ◦ ϕ

Original: [Equation]
Revised: j − 1 ◦ · · · ◦ ϕ i .

Original: Corollary 1 Let B be a bounded set, µ be any probability measure and the neural network f ( x )= (cid:80)
Revised: Corollary 1 Let B be a bounded set, µ be any probability measure and the neural network f ( x )= (cid:80)

Original: ni
Revised: ni

Original: =1 c i σ ( (cid:104) w i , x (cid:105)
Revised: =1 c i σ ( (cid:104) w i , x (cid:105)

Original: + b i ) , where σ ( · ) is a sigmoidal function.
Revised: + b i ) , where σ ( · ) is a sigmoidal function.

Original: If the composition of Barron functions is restricted in the set of Ω B ( C ) , then there exist w i ∈ R d , b i ∈ R , c i ∈ R with (cid:80) ni
Revised: If the composition of Barron functions is restricted in the set of Ω B ( C ) , then there exists w i ∈ R d , b i ∈ R , c i ∈ R with (cid:80) ni

Original: =1 | c i |≤ 2 C such that
Revised: =1 | c i |≤ 2 C such that

Original: [Equation]
Revised: [Equation]

Original: Proof
Revised: Proof

Original: Directly using Lemma 1 and Theorem 2 of (Barron, 1993) can complete the conclusion.
Revised: Directly using Lemma 1 and Theorem 2 of (Barron, 1993) can complete the conclusion.

Original: (cid:3)
Revised: (cid:3)

Original: Theorem 6 (Extend Barron theorem) Let B be a bounded set, µ be any probability measure and f ( x )= (cid:80) ni
Revised: Theorem 6 (Extend Barron theorem) Let B be a bounded set, µ be any probability measure and f ( x )= (cid:80) ni

Original: =1 c i σ ( (cid:104) w i , x (cid:105)
Revised: =1 c i σ ( (cid:104) w i , x (cid:105)

Original: + b i ) , where σ ( · ) be a sigmoidal function.
Revised: + b i ) , where σ ( · ) be a sigmoidal function.

Original: If support ( µ 0 ) ⊂K 0 and ϕ i ( K i − 1 ) ⊆K i , 1 ≤ i ≤ l , the function ϕ
Revised: If support ( µ 0 ) ⊂K 0 and ϕ i ( K i − 1 ) ⊆K i , 1 ≤ i ≤ l , the function ϕ

Original: i is log( l − 1) log ( l )+1 -Lipschitz
Revised: i is log( l − 1) log ( l )+1 -Lipschitz

Original: and Barron function, i.e., ϕ 1 ∈ Ω K 0 ( C 0 ) and
Revised: and Barron function, i.e., ϕ 1 ∈ Ω K 0 ( C 0 ) and

Original: ϕ i ∈
Revised: ϕ i ∈

Original: Ω K i − 

Revised: Ω K i −

Original: + s B mi − 1 ( C i ) , then there exists a neural network f with l hidden layers and S ⊂ R dsatisfying µ 0 ( S ) ≤ 1 −[Equation] ( l
Revised: + s B mi − 1 ( C i ) , then there exists a neural network f with l hidden layers and S ⊂ R dsatisfying µ 0 ( S ) ≤ 1 −

Original: [Equation]
Revised: [Equation]

Original: 2 so that
Revised: 2 so that

Original: [Equation]
Revised: [Equation]

Original: Proof
Revised: Proof

Original: For simplicity, let ϕ :=
Revised: For simplicity, let ϕ :=

Original: ϕ l :1 =
Revised: ϕ l :1 =

Original: ϕ l ◦ · · · ◦ ϕ 1 be the composition of Barron functions, especially ϕ 1 = (cid:101) ϕ
Revised: ϕ l ◦ · · · ◦ ϕ 1 be the composition of Barron functions, especially ϕ 1 = (cid:101) ϕ

Original: k :1 , where (cid:101) ϕ l is deﬁned in Corollary 1.
Revised: k :1 , where (cid:101) ϕ l is deﬁned in Corollary 1.

Original: Let f :=
Revised: Let f :=

Original: f l :1 =
Revised: f l :1 =

Original: f l ◦ f l − 1 ◦ · · · ◦ f 1 be the composition of k -layered neural network, where f 
Revised: f l ◦ f l − 1 ◦ · · · ◦ f 1 be the composition of k -layered neural network, where f i : R d i [Equation] 1 → R

Original: [Equation]
Revised: [Equation]

Original: 1 are functions deﬁned as
Revised: 1 are functions deﬁned as

Original: [Equation]
Revised: [Equation]

Original: where c ijk , b ijk ∈ R and w ijk ∈ R d
Revised: where c ijk , b ijk ∈ R and w ijk ∈ R d

Original: i − 1 are parameters, and r i is the number of nodes in the i -th layer.
Revised: i − 1 are parameters, and r i is the number of nodes in the i -th layer.

Original: Note that the function f i is represented by a neural network with one hidden layer and a linear output layer.
Revised: Note that the function f i is represented by a neural network with one hidden layer and a linear output layer.

Original: We prove the theorem by induction on l .
Revised: We prove the theorem by induction on l .

Original: i)
Revised: i)

Original: For l =1 , using assumptions of the support of initial distribution ( i.e. , support ( µ 0 ) ⊂K 0 ) and the deﬁnition of Barron function.
Revised: For l =1 , using assumptions of the support of initial distribution ( i.e. , support ( µ 0 ) ⊂K 0 ) and the deﬁnition of Barron function.

Original: Then, (cid:15) =(2 C 1 ) 2 /r 1 and (cid:80) r 1 i =1 | c i | ≤ 2 C 1 , we complete the results.
Revised: Then, (cid:15) =(2 C 1 ) 2 /r 1 and (cid:80) r 1 i =1 | c i | ≤ 2 C 1 , we complete the results.

Original: ii)
Revised: ii)

Original: For l> 1 , by the induction step, we assume the functions f 1 , . . .
Revised: For l> 1 , by the induction step, we assume the functions f 1 , . . .

Original: , f l − 1 satisfy the conclusion for (cid:101) ϕ 1 , . . .
Revised: , f l − 1 satisfy the conclusion for (cid:101) ϕ 1 , . . .

Original: , (cid:101) ϕ
Revised: , (cid:101) ϕ

Original: l − 1 .
Revised: l − 1 .

Original: Let S l − 1 be the set in the conclusion.
Revised: Let S l − 1 be the set in the conclusion.

Original: Applying Corollary 1 to f l to get that for each≤ j ≤ m j , for any µ supported on a set K (cid:48)
Revised: Applying Corollary 1 to f l to get that for each≤ j ≤ m j , for any µ supported on a set K (cid:48)

Original: l − 1 ⊆
Revised: l − 1 ⊆

Original: R m l − 1 and any r l ∈ N , there exists a neural net f l,j with 1 hidden layer with r l nodes such that
Revised: R m l − 1 and any r l ∈ N , there exists a neural net f l,j with 1 hidden layer with r l nodes such that

Original: [Equation]
Revised: [Equation]

Original: Note that Theorem 1 applies to any distribution µ supported on the set K (cid:48)
Revised: Note that Theorem 1 applies to any distribution µ supported on the set K (cid:48)

Original: l − 1 .
Revised: l − 1 .

Original: Let S l
Revised: Let S l

Original: [Equation]
Revised: [Equation]

Original: Applying Theorem 1 with
Revised: Applying Theorem 1 with

Original: K
Revised: K

Original: [Equation]
Revised: [Equation]

Revised: l , r

Original: [Equation]
Revised: [Equation]

Original: (cid:109) and µ = f l − 1:1# ( 1 S l µ 0 ) .
Revised: (cid:109) and µ = f l − 1:1# ( 1 S l µ 0 ) .

Original: We have that µ is supported on g l − 1:1 ( S l ) ⊆
Revised: We have that µ is supported on g l − 1:1 ( S l ) ⊆

Original: K l − 

Revised: K l −

Original: + s B m l − 1 = K (cid:48)
Revised: + s B m l − 1 = K (cid:48)

Original: l − 1 , and the function ϕ
Revised: l − 1 , and the function ϕ

Original: l is C l -Barron function on this set by the assumption.
Revised: l is C l -Barron function on this set by the assumption.

Original: The conclusion of Theorem 1 gives ( f l ) j such thatc
Revised: The conclusion of Theorem 1 gives ( f l ) j such that

Original: [Equation]
Revised: [Equation]

Original: For all elements of ϕ l and f l , we have
Revised: For all elements of ϕ l and f l , we have

Original: [Equation]
Revised: [Equation]

Original: We bound by the triangle inequality
Revised: We bound by the triangle inequality

Original: [Equation]
Revised: [Equation]

Original: [Table][Equation] [Table] [Equation]
Revised: [Table]

Original: [Table]

Original: [Equation][Table] [Equation] [Table]
Revised: [Equation]

Original: [Equation] [Table] [Equation] [Table]
Revised: [Table]

Original: [Equation][Table] [Equation] [Table]
Revised: [Equation]

Original: [Equation] [Table] [Equation] [Table]
Revised: [Table]

Original: [Equation][Table] [Equation] [Table]
Revised: [Equation]

Original: [Equation] [Table] [Equation] [Table]
Revised: [Table]

Original: [Equation][Table] [Equation] [Table]
Revised: [Equation]

Original: [Equation] [Table] [Equation] [Table]
Revised: [Table]

Original: [Equation][Table] [Equation] [Table]
Revised: [Equation]

Original: [Equation] [Table] [Equation] [Table]
Revised: [Table]

Original: [Equation][Table] [Equation] [Table]
Revised: [Equation]

Original: [Equation] [Table] [Equation] [Table]
Revised: [Table]

Original: [Equation][Table] [Equation] [Table]
Revised: [Equation]

Original: [Equation]

Original: [Table][Equation] [Table] [Equation]
Revised: [Table]

Original: [Table]

Original: [Equation][Table] [Equation] [Table]
Revised: [Equation]

Original: [Equation]

Original: [Table][Equation] [Table] [Equation]
Revised: [Table]

Original: log

Original: [Equation]
Revised: [Equation]

Original: where the last inequality holds by the assumption ( i.e. , Lip ( ϕ l )=
Revised: where the last inequality holds by the assumption ( i.e. , Lip ( ϕ l )=

Original: log( l − 1) log ( l )+1 ) and the induction hypothesis.
Revised: log( l − 1) log ( l )+1 ) and the induction hypothesis.

Original: As above analysis, we have
Revised: As above analysis, we have

Original: [Equation]
Revised: [Equation]

Original: by the induction hypothesis.
Revised: by the induction hypothesis.

Original: Also, ϕ l − 1:1 ∈ K l − 1 for all x ∈ support ( µ 0 ) by the assumption of the theorem.
Revised: Also, ϕ l − 1:1 ∈ K l − 1 for all x ∈ support ( µ 0 ) by the assumption of the theorem.

Original: Therefore, by Markov’s inequality and the induction hypothesis on
Revised: Therefore, by Markov’s inequality and the induction hypothesis on

Original: [Equation]
Revised: [Equation]

Original: Therefore,µ
Revised: Therefore,

Original: [Equation]
Revised: [Equation]

Original: the last equality hold by a fact that there exist a constant δ to satisfy µ 0 ( S l ) = 1 −
Revised: the last equality holds by a fact that there exists a constant δ to satisfy µ 0 ( S l ) = 1 −

Original: [Equation]
Revised: [Equation]

Revised: Based on the above conclusions, we prove Theorem 4 as follow.

Original: Theorem 4 Given a data distribution µ 0 and a function ϕ
Revised: Theorem 4 Given a data distribution µ 0 and a function ϕ

Original: i : R m i − 1 → R m i , and let L l = log( l )+1 ,
Revised: i : R m i − 1 → R m i , and let L l = log( l )+1 ,

Original: if support ( µ 0 ) ⊂K 0 and ϕ i ( K i − 1 ) ⊆K
Revised: if support ( µ 0 ) ⊂K 0 and ϕ i ( K i − 1 ) ⊆K

Original: i , 1 ≤ i ≤ l , the function ϕ
Revised: i , 1 ≤ i ≤ l , the function ϕ

Original: i is (cid:16) L l − 1 − 1 L l (cid:17) -Lipschitz and is a Barron function, i.e., ϕ 1 ∈ Ω K 0 ( C 0 ) and ϕ i ∈
Revised: i is (cid:16) L l − [Equation] (cid:17) -Lipschitz and is a Barron function, i.e., ϕ 1 ∈ Ω K 0 ( C 0 ) and ϕ i ∈

Original: Ω K i − 

Revised: Ω K i −

Original: + s B mi − 1 ( C i ) , then there exists a network f with l hidden layers with (cid:6) 4 C 2 l m l L 2 l − 1 L 2 l /(cid:15) 2 (cid:7) neurons on the i -th layer, 
Revised: + s B mi − 1 ( C i ) , then there exists a network f with l hidden layers with (cid:6) 4 C 2 l m l L 2 l − 1 L 2 l /(cid:15) 2 (cid:7) neurons on the i -th layer, W

Original: [Equation]
Revised: [Equation]

Original:  where D l is the diameter of the set K l , and (cid:15), δ, s> 0 are parameters.
Revised: ) where D l is the diameter of the set K l , and (cid:15), δ, s> 0 are parameters.

Original: Proof
Revised: Proof

Original: The functions f 1 , · · · , f l in Theorem 6 satisfy that
Revised: The functions f 1 , · · · , f l in Theorem 6 satisfy that

Original: [Equation]
Revised: [Equation]

Original: The range of f l = (( f l ) 1 , · · · , ( f l ) m l ) is contained in a set of diameter 2 C l √
Revised: The range of f l = (( f l ) 1 , · · · , ( f l ) m l ) is contained in a set of diameter 2 C l √

Original: m l because the activation in a neural network is ranged in [0 , 1] and the weights c ijk in Theorem 6 satisfy (cid:80) rk =

Revised: m l because the activation in a neural network is ranged in [0 , 1] and the weights c ijk in Theorem 6 satisfy (cid:80) rk =

Original: | c ijk | ≤ 2 C l .
Revised: | c ijk | ≤ 2 C l .

Original: Choose a constant vector k to minimize (cid:82) S l (cid:107)
Revised: Choose a constant vector k to minimize (cid:82) S l (cid:107)

Original: ϕ l :1 ( x ) − f l :1 − k (cid:107) 2 dµ 0 and replace f l with f l + k .
Revised: ϕ l :1 ( x ) − f l :1 − k (cid:107) 2 dµ 0 and replace f l with f l + k .

Original: Note that the range of f l and ϕ l necessarily overlap.
Revised: Note that the range of f l and ϕ l necessarily overlap.

Original: We still have (cid:82) S l (cid:107) ϕ l :1 ( x ) − f l :1 (cid:107) 2 dµ 0 ≤ (cid:15)
Revised: We still have (cid:82) S l (cid:107) ϕ l [Equation] 1 ( x ) − f l [Equation]

Revised: 1 (cid:107) 2 dµ 0 ≤

Original: [Equation]
Revised: [Equation]

Original: log
Revised: log

Original: [Equation]
Revised: [Equation]

Original: Moreover, we have
Revised: Moreover, we have

Original: [Equation]
Revised: [Equation]

Original: for x ∈ K 0 , where D l is the diameter of K l .
Revised: for x ∈ K 0 , where D l is the diameter of K l .

Original: Let S cl be a complementary of S l , we have
Revised: Let S cl be a complementary of S l , we have

Original: [Equation]
Revised: [Equation]

Original: then we have (cid:90)
Revised: then we have (cid:90)

Original: [Equation]
Revised: [Equation]

Original: For ( ϕ l :1 ( X ) , f l :1 ( X )) , X ∼ µ 0 deﬁne a coupling between the distributions.
Revised: For ( ϕ l :1 ( X ) , f l :1 ( X )) , X ∼ µ 0 deﬁne a coupling between the distributions.

Original: Based on the deﬁnition of W -distance, we have
Revised: Based on the deﬁnition of W -distance, we have

Original: [Equation]
Revised: [Equation]

Original: C P ROOFS OF L AYER B EHAVIOR A NALYSIS OF D EEP N EURAL N ETWORKS
Revised: C P ROOFS OF L AYER B EHAVIOR A NALYSIS OF D EEP N EURAL N ETWORKS

Original: Deﬁnition 9 ( Flow )
Revised: Deﬁnition 9 ( Flow )

Original: A ﬂow ϕ t is given by an ordinary differential equation (ODE), ˙ ϕ t ( x )= v t ( ϕ t ( x )) with a velocity ﬁeld v t when t> 0 , and ϕ 0 ( x )= x when t =0 .
Revised: A ﬂow ϕ t is given by an ordinary differential equation (ODE), ˙ ϕ t ( x )= v t ( ϕ t ( x )) with a velocity ﬁeld v t when t> 0 , and ϕ 0 ( x )= x when t =0 .

Original: Proposition 2 ( Continuity equation (Sonoda & Murata, 2019)) Let ϕ t be the ﬂow of an ODE with vector ﬁeld v t , then the distribution µ t evolves according to the continuity equation ∂ t µ t ( x )= −∇· [ µ t ( x ) v t ( x )] , x ∈ R m , t ≥ 0 , where ∇· denotes the divergence operator.
Revised: Proposition 2 ( Continuity equation (Sonoda & Murata, 2019)) Let ϕ t be the ﬂow of an ODE with vector ﬁeld v t , then the distribution µ t evolves according to the continuity equation ∂ t µ t ( x )= −∇· [ µ t ( x ) v t ( x )] , x ∈ R m , t ≥ 0 , where ∇· denotes the divergence operator.

Original: Theorem 7 ( Optimal transport map )
Revised: Theorem 7 ( Optimal transport map )

Original: The global minimum f ∗ t of L ( f ) is attained at
Revised: The global minimum f ∗ t of L ( f ) is attained at

Original: [Equation]
Revised: [Equation]

Original: where ∗ denotes the convolution operator.
Revised: where ∗ denotes the convolution operator.

Original: Proof
Revised: Proof

Original: The proof follows from the calculus of variations.
Revised: The proof follows from the calculus of variations.

Original: [Equation]
Revised: [Equation]

Original: where the second line holds by the calculus of variations, i.e. , x (cid:48)
Revised: where the second line holds by the calculus of variations, i.e. , x (cid:48)

Original: =
Revised: =

Original: x + ε .
Revised: x + ε .

Original: Then, for an arbitrary function h , the ﬁrst variation δ L [ h ] is given by
Revised: Then, for an arbitrary function h , the ﬁrst variation δ L ( h ) is given by

Original: [Equation]
Revised: [Equation]

Original: At a critical point f ∗ of L , δ L ( h ) = 0 for every h .
Revised: At a critical point f ∗ of L , δ L ( h ) = 0 for every h .

Original: Hence,
Revised: Hence,

Original: [Equation]
Revised: [Equation]

Original: by the fundamental lemma of calculus of variations for integrable functions, and we have
Revised: by the fundamental lemma of calculus of variations for integrable functions, and we have

Original: [Equation]
Revised: [Equation]

Original: Note that f ∗ attains the global minimum, because, for every function h ,
Revised: Note that f ∗ attains the global minimum, because, for every function h ,

Original: [Equation]
Revised: [Equation]

Original: Theorem 

Revised: Theorem

Original: If the target network is a residual unit f
Revised: If the target network is a residual unit f

Original: [Equation]
Revised: [Equation]

Original: ( (cid:101) x )= x
Revised: ( (cid:101) x )= x

Original: + σ ( (cid:101) x ) , where σ contains fully connected layer and activation function.
Revised: + σ ( (cid:101) x ) , where σ contains fully connected layer and activation function.

Original: When ν t := N ( 0 , t I ) , then the global minimum f ∗ t is
Revised: When ν t := N ( 0 , t I ) , then the global minimum f ∗ t is

Original: [Equation]
Revised: [Equation]

Original: Proof
Revised: Proof

Original: Using Stein’s identity (Liu, 1994), − t ∇ ν t ( ε ) =
Revised: Using Stein’s identity (Liu, 1994), − t ∇ ν t ( ε ) =

Original: ε ν t ( ε ) , which is known to hold only for
Revised: ε ν t ( ε ) , which is known to hold only for

Original: Gaussians.
Revised: Gaussians.

Original: [Equation]
Revised: [Equation]

Original: Theorem 2 Based on the equivalent condition in Theorem 2 (Belavkin, 2016)
Revised: Theorem 2 Based on the equivalent condition in Theorem 2 (Belavkin, 2016)

Original: [Equation]
Revised: [Equation]

Original: and let ∆ be the
Revised: and let ∆ be the

Original: Laplacian operator.
Revised: Laplacian operator.

Original: At the initial moment t →
Revised: At the initial moment t →

Original: [Equation]
Revised: [Equation]

Original: the pushforward f # µ t with Gaussian distribution satisﬁes the backward heat equation (BHE)
Revised: the pushforward f # µ t with Gaussian distribution satisﬁes the backward heat equation (BHE)

Original: [Equation]
Revised: [Equation]

Original: and evolves according to Wasserstein gradient ﬂow:
Revised: and evolves according to Wasserstein gradient ﬂow:

Original: [Equation]
Revised: [Equation]

Original: t
Revised: t

Original: [Equation]
Revised: [Equation]

Original: − gradW (
Revised: grad

Original: [Equation]
Revised: [Equation]

Original: 0 , µ) , x ∈ R m
Revised: µ

Original: [Equation]
Revised: [Equation]

Original: Proof
Revised: Proof

Original: The initial velocity vector is given by
Revised: The initial velocity vector is given by

Original: [Equation]
Revised: [Equation]

Original:  Hence, by substituting the score (13) in the continuity equation of Proposition [Equation] we have
Revised: [Equation] Hence, by substituting the score (13) in the continuity equation of Proposition 2, we have

Original: [Equation]
Revised: [Equation]

Original: We leave the proof of ∆ µ 0 ( x ) =
Revised: We leave the proof of ∆ µ 0 ( x ) =

Original: W  ( µ t , µ ) in Theorem 3.
Revised: W 2 ( µ t , µ ) in Theorem 3.

Original: [Equation]
Revised: [Equation]

Original: Theorem 3 Based on the equivalent condition in Theorem 2 (Belavkin, 2016), and when the noise distribution is a Gaussian distribution, then the pushforward measure µ t :=
Revised: Theorem 3 Based on the equivalent condition in Theorem 2 (Belavkin, 2016), and when the noise distribution is a Gaussian distribution, then the pushforward measure µ t :=

Original: ϕ t # µ 0 evolves according to Wasserstein gradient ﬂow as follows:
Revised: ϕ t # µ 0 evolves according to Wasserstein gradient ﬂow as follows:

Original: [Equation]
Revised: [Equation]

Original: Proof
Revised: Proof

Original: Let the potential function be equal to Kullback–Leibler divergence be KL ( µ t , µ )
Revised: Let the potential function be equal to Kullback–Leibler divergence be KL ( µ t , µ )

Original: = (cid:82) µ t ( x ) log µ t ( x ) µ ( x ) − µ t ( x ) + µ ( x ) d x , then V t
Revised: = (cid:82) µ t ( x ) log µ t ( x ) µ ( x ) − µ t ( x ) + µ ( x ) d x , then V t

Original: = − (log( µ t ) − log( µ )) , then
Revised: = − (log( µ t ) − log( µ )) , then

Original: [Equation]
Revised: [Equation]

Original: The above continuity also satisﬁes the case of Wasserstein distance, then using the equivalent condition in Theorem 2 (Belavkin, 2016), we have
Revised: The above continuity also satisﬁes the case of Wasserstein distance, then using the equivalent condition in Theorem 2 (Belavkin, 2016), we have

Original: [Equation]
Revised: [Equation]

Revised: We assume the distribution µ ( t ) l is Gaussian distribution.

Revised: The W -distance of two Gaussian distribution is deﬁned as follow.

Original: Proposition 3 (Gelbrich, 1990)
Revised: Proposition 3 (Gelbrich, 1990)

Original: Given two Gaussian distributions µ 1 ∼N ( u 1 , Σ 1 ) and µ 2 ∼N ( u 2 , Σ 2 ) , then Wasserstein distance between µ 1 and µ 2 is
Revised: Given two Gaussian distributions µ 1 ∼N ( u 1 , Σ 1 ) and µ 2 ∼N ( u 2 , Σ 2 ) , then Wasserstein distance between µ 1 and µ 2 is

Original: [Equation]
Revised: [Equation]

Original: Batch Normalization is proposed to stabilize the distributions of layer distribution by introducing the parameters γ t and β t in the t -th optimization step.
Revised: Batch Normalization is proposed to stabilize the distributions of layer distribution by introducing the parameters γ t and β t in the t -th optimization step.

Original: Let Γ ( t ) l = diag ( γ ( t ) l ) and Γ ( t +1)
Revised: Let Γ ( t ) l = diag ( γ ( t ) l ) and Γ ( t +1)

Original: l = diag ( γ ( t +1) l ) .
Revised: l = diag ( γ ( t +1) l ) .

Original: Proposition 4 Assume the distributions in two iterations in the l -th layer are Gaussian distributions, i.e., µ ( t ) l ∼N ( β ( t ) l , Γ ( t ) l ) and µ ( t +1) l ∼N ( β ( t +1) l , Γ ( t +1) l ) , then  W -distance between µ ( t ) l and µ (t +1) l is W 2 ( µ ( t ) l , µ ( t +1) l
Revised: Proposition 4 Assume the distributions in two iterations in the l -th layer are Gaussian distributions, i.e., µ ( t ) l ∼N ( β ( t ) l , Γ ( t ) l ) and µ ( t +1) l ∼N ( β ( t +1) l , Γ ( t +1) l ) , then the W -distance between µ ( t ) l and µ (

Original: [Equation]
Revised: [Equation]

Original: Proof
Revised: Proof

Original: Directly from 3, and let u 1 = β ( t ) l , u 2 = β ( t +1) l and Σ 1 = Γ ( t ) l , Σ 2 =
Revised: Directly from 3, and let u 1 = β ( t ) l , u 2 = β ( t +1) l and Σ 1 = Γ ( t ) l , Σ 2 =

Original: Γ ( t +1) l , we have 
Revised: Γ ( t +1) l , we have W

Original: [Equation]
Revised: [Equation]

Original: D O PTIMAL T RANSPORT
Revised: D O PTIMAL T RANSPORT

Original: Deﬁnition 10 (Optimal Transport (Villani, 2008))
Revised: Deﬁnition 10 (Optimal Transport (Villani, 2008))

Original: Given a cost function c : K×K→ R , the optimal transport distance measures the optimal plan to transport the mass from a probability measure (cid:98) µ to another probability measure µ :
Revised: Given a cost function c : K×K→ R , the optimal transport distance measures the optimal plan to transport the mass from a probability measure (cid:98) µ to another probability measure µ :

Original: [Equation]
Revised: [Equation]

Original: where Π( (cid:98) µ, µ ) is the set of joint probability measures π on K×K with the marginals (cid:98) µ and µ .
Revised: where Π( (cid:98) µ, µ ) is the set of joint probability measures π on K×K with the marginals (cid:98) µ and µ .

Original: In this paper, we consider the multi-label classiﬁcation problem, we ﬁrst introduce the deﬁnition of
Revised: In this paper, we consider the multi-label classiﬁcation problem. We ﬁrst introduce the deﬁnition of

Original: Wasserstein distance.
Revised: Wasserstein distance.

Original: For the case of probability measures, they are histograms in the simplex ∆ K .
Revised: For the case of probability measures, they are histograms in the simplex ∆ K .

Original: Deﬁnition 11 ( W -distance)
Revised: Deﬁnition 11 ( W -distance)

Original: Given an input x ∈X and any f : X→ ∆ K , let f ( x ) be the predicted label distribution (cid:98) µ , and let y be the target distribution µ , then  W -distance between (cid:98) µ and µ is[Equation]
Revised: Given an input x ∈X and any f : X→ ∆ K , let f ( x ) be the predicted label distribution (cid:98) µ , and let y be the target distribution µ , then the W -distance between (cid:98) µ and µ is

Original: [Equation]
Revised: [Equation]

Original:  where C ∈ R K × K + is the cost matrix whose the element can be deﬁned as C κ,κ (cid:48)
Revised: (16) where C ∈ R K × K + is the cost matrix whose the element can be deﬁned as C κ,κ (cid:48)

Original: =
Revised: =

Original: d p K ( κ, κ (cid:48) ) , and the set of couplings is composed of joint probability distributions with their marginals ˆ µ and µ , i.e., Π
Revised: d p K ( κ, κ (cid:48) ) , and the set of couplings is composed of joint probability distributions with their marginals ˆ µ and µ , i.e., Π

Original: [Equation]
Revised: [Equation]

Original: Entropic Wasserstein loss.
Revised: Entropic Wasserstein loss.

Original: Cuturi (2013) introduces an entropic regularization such that the nonconvex problem (2) can be turned to a strictly convex problem:
Revised: Cuturi (2013) introduces an entropic regularization such that the nonconvex problem (2) can be turned to a strictly convex problem:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where H ( T )= − (cid:80) κ,κ (cid:48)
Revised: where H ( T )= − (cid:80) κ,κ (cid:48)

Original: T κ,κ (cid:48)
Revised: T κ,κ (cid:48)

Original: (log( T κ,κ (cid:48) )
Revised: (log( T κ,κ (cid:48) )

Original: − 1) .
Revised: − 1) .

Original: With the help of the entropic regularization, we solve optimal transport problem efﬁciently using Sinkhorn-Knopp matrix scaling algorithm (Knight, 2008).
Revised: With the help of the entropic regularization, we solve optimal transport problem efﬁciently using Sinkhorn-Knopp matrix scaling algorithm (Knight, 2008).

Original: Proposition 5 The transport matrix T ∗ optimizing Problem (17) satisﬁes T ∗ =
Revised: Proposition 5 The transport matrix T ∗ optimizing Problem (17) satisﬁes T ∗ =

Original: diag ( u ) K diag ( v ) , where K = e − α M , u = e α a and v = e α b , where a and b are the Lagrange dual variables.
Revised: diag ( u ) K diag ( v ) , where K = e − α M , u = e α a and v = e α b , where a and b are the Lagrange dual variables.

Original: Theorem 8 (Genevay et al., 2017)Wasserstein sinkhorn loss between the predicted label distribution (cid:98) µ and the target label distribution µ is deﬁned as:
Revised: Theorem 8 (Genevay et al., 2017)Wasserstein sinkhorn loss between the predicted label distribution (cid:98) µ and the target label distribution µ is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: with the following limiting behavior as α →0 :
Revised: with the following limiting behavior as α →

Original: [Equation]
Revised: [Equation]

Original: Note that normalized Wasserstein loss is non-negative and 
Revised: Note that normalized Wasserstein loss is non-negative and ¯

Original: ¯ W  α ( (cid:98) µ, µ ) = 

Revised: W 2 α ( (cid:98) µ, µ ) =

Original: if and only is (cid:98) µ = µ .
Revised: if and only is (cid:98) µ = µ .

Original: In the quantiﬁcation, we use normalized Wasserstein loss to measure the divergence between distributions.
Revised: In the quantiﬁcation, we use normalized Wasserstein loss to measure the divergence between distributions.

Original: E I NTERMEDIATE LAYER
Revised: E I NTERMEDIATE LAYER

Original: We add auxiliary classiﬁers for the intermediate layers as mentioned below.
Revised: We add auxiliary classiﬁers for the intermediate layers as mentioned below.

Original: For ResNet-18, we truncate each block and add a classiﬁer after its feature map.
Revised: For ResNet-18, we truncate each block and add a classiﬁer after its feature map.

Original: We label these intermediate layers from layer0 to layer8.
Revised: We label these intermediate layers from 0 -th layer to 8 -th layer.

Original: The truncation of ResNet-50 is analogous to ResNet-18.
Revised: The truncation of ResNet-50 is analogous to ResNet-18.

Original: For VGG-16, we add a classiﬁer after each convolution layer and label them from layer0 to layer [Figure]
Revised: For VGG-16, we add a classiﬁer after each convolution layer and label them from 0 -th layer to 12 -th layer.

Original: F I NTER - LAYER DISTRIBUTION PROPAGATION
Revised: F A CROSS - LAYER DISTRIBUTION PROPAGATION

Original: In this section

Original: [Figure]

Original: we show morequalitative results for different datasets, measures and networks across differentinter layers.
Revised: We show more results for different datasets, measures and networks across different layers.

Original: Under review as a conference paper at ICLR

Original: [Figure]
Revised: [Figure]

Original: [Figure]

Original: training set.
Revised: different test set.

Original: [Table]
Revised: Speciﬁcally, we shufﬂe the VOC2007 dataset (including training set and test set) and divide it into training set and test set following the ratio of original dataset. We shufﬂe twice and deﬁne them as “trial_0” and “trial_1”, respectively. Besides, the “orig.” means that we use the original dataset. From Figure 9, different experiments consistently have the same decreasing tendency through the depth of a neural network.

Original: Under review as a conference paper at ICLR
Revised: Under review as a conference paper at ICLR

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [-W - d i s t a n c e VOC COCO 0 1 2 3 4 5 6 7 8 9 10 11 
-]
Revised: [-W - d i s t a n c e VOC COCO 0 1 2 3 4 5 6 7 8 9 10 11 
-][Figure]

Original: (VGG
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: VOC COCO
Revised: 16 (right) on test set. The tendency of distribution propagation on test set is the same as the training set(Figure 3), suggesting that the distribution propagation is irrelevant to the property of the dataset.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: ) on training set.

Original:  Although the tendency of Chebyshev distance is analogous to Wasserstein distance,
Revised: [Figure] Although the tendency of Chebyshev distance is analogous to Wasserstein distance,

Original: Chebyshev distance ignores any metric structure.
Revised: Chebyshev distance ignores any metric structure (Frogner et al., 2015).

Original: Therefore, we use Wasserstein distance to quantify the discrepancy of label distributions.
Revised: Therefore, we use Wasserstein distance to quantify the discrepancy of label distributions.

Original: Under review as a conference paper at ICLR
Revised: Under review as a conference paper at ICLR

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure][Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: ideal result V OC 1.17% 0.44% 0.69% 0.71% 0.55% 0.50% 0.53% 0.85% 0.53% 64.48% 70.44% MS-COCO 0.49% 0.43% 0.47% 0.28% 0.30% 0.31% 0.37% 0.78% 0.50% 27.33% 31.26%
Revised: R e s N e t (

Original: [Figure]
Revised: [Figure]

Original: Under review as a conference paper at ICLR
Revised: O C )

Original: [Figure] [Figure] [Figure]{+epoch=1   20 epoch=30   
+}
Revised: {+epoch=1   20 epoch=30   
+}

Original: [Figure] [Figure] [Figure]   20 0 0.5 V G G ( V O C )
Revised: epoch=70   20 epoch=100   20 target   20 0 0.5 V GG ( V O C )

Original: epoch=1   20 epoch=30   

Revised: epoch=1   20 epoch=30

Original: epoch=70   20 epoch=100  20 target   80 0 0.

Revised: epoch=70   20 epoch=100  20 target   80 0 0.

Original: R e s N e t ( C O C O )
Revised: R e s N e t ( C O C O )

Original: epoch=1   80 epoch=30   

Revised: epoch=1   80 epoch=30

Original: epoch=70   80 epoch=100   80 target   80 # label 0 0.5 V GG ( C O C O )
Revised: epoch=70   80 epoch=100   80 target   80 # label 0 0.5 V GG ( C O C O )

Original: epoch=1   80 # label epoch=30   80 # label epoch=70   80 # label epoch=100   
Revised: epoch=1   80 # label epoch=30   80 # label epoch=70   80 # label epoch=100

Original: 80 # label target
Revised: 80 # label target

Revised: [Figure]

Original: ideal result
Revised: ideal result

Original: [Figure]
Revised: [Figure]

Original: [Table][Table] [Table] [Table]
Revised: [Table]

Original: [Table] [Table] [Table] [Table]
Revised: MS

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]
Revised: [Table]

Original: 48% 70.44% MS-COCO

Original: [Table] [Table] [Figure][Table]
Revised: [Figure]

Original: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: Under review as a conference paper at ICLR
Revised: Under review as a conference paper at ICLR

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure][Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: 20 0 0.5 R e s N e t ( V O C )
Revised: 20 0 0.5 R e s N e t ( V O C )

Original: epoch=1   20 epoch=30   

Revised: epoch=1   20 epoch=30

Original: epoch=70   20 epoch=100   20 target   20 0 0.5 V GG ( V O C ) {+epoch=1   20 epoch=30   
 epoch=70   20 epoch=100  20 target   80 0 0.
 R e s N e t ( C O C O )+}
Revised: epoch=70   20 epoch=100   20 target   20 0 0.5 V GG ( V O C ) {+epoch=1   20 epoch=30   
 epoch=70   20 epoch=100  20 target   80 0 0.
 R e s N e t ( C O C O )+}

Original: [-epoch=
-]{+epoch=1   80 epoch=30   
 epoch=70   80 epoch=100   80 target   80 # label 0 0.5 V GG ( C O C O ) epoch=1   80 # label epoch=30   80 # label epoch=70   80 # label epoch=100   80 # label target layer          ideal result VOC 1.17% 0. [Table]+}
Revised: [-epoch=
-]{+epoch=1   80 epoch=30   
 epoch=70   80 epoch=100   80 target   80 # label 0 0.5 V GG ( C O C O ) epoch=1   80 # label epoch=30   80 # label epoch=70   80 # label epoch=100   80 # label target layer          ideal result VOC 1.17% 0. [Table]+}

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: % MS

Revised: [Figure]

Revised: COCO

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: Under review as a conference paper at ICLR

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: 1   20 epoch

Revised: [Figure]

Original: {+30   20 epoch=70   20 epoch=
+}
Revised: {+30   20 epoch=70   20 epoch=
+}

Revised: [Figure]

Revised: 20 0 0.5 V GG ( V O C )

Original: {+epoch=1   20 epoch=30   
+}
Revised: {+epoch=1   20 epoch=30   
+}

Revised: epoch=70   20 epoch=

Revised: [Figure]

Revised: [Figure]

Original: {+target   80 0 0.
+}
Revised: {+target   80 0 0.
+}

Revised: R e s N e

Revised: [Figure]

Original: [Figure] {+O ) epoch=1   80 epoch=30   
 epoch=70   80 epoch=
+}
Revised: [Figure] {+O ) epoch=1   80 epoch=30   
 epoch=70   80 epoch=
+}

Original: [Figure] [Figure] 
Revised: [Figure] [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original:  epoch=

Revised: ) epoch=

Original: [Figure]
Revised: [Figure]

Original: [Figure] 
Revised: [Figure] [Figure] [Figure] [Figure]

Original: [Figure] 
Revised: [Figure] [Figure] [Figure] [Figure]

Revised: [Table]

Original: [Figure] 
Revised: [Figure] [Table] [Figure] [Table]

Original: [Figure] 
Revised: [Figure] [Table] [Figure] [Table]

Original: [Figure] 
Revised: [Figure] [Table] [Figure] [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: G I NNER - LAYER DISTRIBUTION PROPAGATION
Revised: G S INGLE - LAYER DISTRIBUTION PROPAGATION

Original: In this part, we show morequalitative results about the distributions evolving in one layer when training.
Revised: We present more results about the single-layer distribution propagation when training DNNs.

Original: more...
Revised: We calculate the Wasserstein distance between the prediction distribution of a selected epoch i and the next epoch i + 1 and show in Figure 15. Analogous to Figure 4, Figure 16 shows that the label distribution of one sample propagates from the ﬁrst epoch to the last epoch.

Original: Under review as a conference paper at ICLR 2020        9 layer (ResNet) 0.00 0.25 0.50 0.75 1.00{+
 [Figure]+} J S - d i v e r g e n c e
Revised: Under review as a conference paper at ICLR {+
 [Figure]+} J S - d i v e r g e n c e

Original: VOC
Revised: VOC

Original: COCO 0 1 2 3 4 5 6 7 8 9 10 11 12 13 layer (VGG) 0.00 0.25 0.50 0.75 1.
Revised: COCO [Figure]

Original: VOC
Revised: VOC

Original: COCO
Revised: COCO

Original: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 layer (ResNet) 0.0 0.5 1.0 1.5 W - d i s t a n c e
Revised: .0 1.5 W - d i s t a n c e

Original: VOC
Revised: VOC

Original: COCO 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 layer (ResNet) 0.0 0.5 1.0 1.
Revised: COCO 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 layer (ResNet) [Figure] [Figure]

Original: VOC
Revised: VOC

Original: COCOtraining set ( and test set
Revised: COCO

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: different layers of ResNet-50 on training set.

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure] [Figure] at
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure] [Figure]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure] [Figure] [Figure]
Revised: Under review

Original: [Figure]
Revised: [Figure]

Original: [Table] [Figure]
Revised: [Figure] [Figure]

Revised: c

Revised: [Figure]

Revised: COCO

Revised: [Figure]

Revised: COCO

Revised: [Figure]

Revised: [Figure]

Original:  H E ARLY - EXITS STRATEGY
Revised: [Figure] [Figure] [Table] H E ARLY - EXITS STRATEGY

Original: Deep neural networks often occur the over-thinking issue, that is, the samples are correctly classiﬁed in the intermediate layer but misclassiﬁed in the last layer.
Revised: Deep neural networks often occur the over-thinking issue, that is, the samples are correctly classiﬁed in the intermediate layer but misclassiﬁed in the last layer.

Original: We call these samples as confused samples.
Revised: We call these samples as confused samples.

Original: xx
Revised: From Table 2 and

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: the accuracy of ResNet-18 (VOC) and VGG-16 (VOC) are 64.48% and 68.96%, respectively. In other words, out of 35.52% and 31.04% of the samples for ResNet-18 and VGGare misclassiﬁed on the test set of VOC2007, respectively.

Original: Speciﬁcally, we add auxiliary classiﬁers for some intermediate layers to get the prediction probability distributions corresponding to the selected layers.
Revised: For these misclassiﬁed samples, 1.17% are actually correctly classiﬁed at 0 -th layer, 0.44% are at 1 -th layer and so on.

Original: For each selected layer
Revised: Ideally, if we early exit these confused samples, the cumulative accuracy of ResNet-18 is 70.44% on VOC dataset [Table] [Table]

Original: [Table]
Revised: [Table]

Revised: Early-exits strategy.

Original: we propose a simple probability mechanism to exit confused samples.
Revised: We propose a simple probability mechanism to exit confused samples.

Original: We deem a  predictionconﬁdent if its probability exceeds not only the ﬁxed parameter but also the hyperparameter. For this prediction, We also adjust the ratio between the number of exceeding the ﬁxed parameter and  the number of exceeding the hyperparameter.
Revised: Speciﬁcally, we add an auxiliary classiﬁer for a selected intermediate layer to get the prediction probability distribution. Each auxiliary classiﬁer contains fully connected layer and Sigmoid function. For each sample, we denote the number of exceeding the threshold p = 0 .

Original: The ﬁxed parameter is set to be 0.5 due to using a Sigmoid function.
Revised: 5 as N . We also denote the number of exceeding another threshold q as n . We denote the ratio as γ and γ = n/N .

Original: The hyperparameter and ratio  are range from 0.5 to 1.
Revised: The threshold q and ratio γ are range from 0.5 to 1.

Original: We search for the best values of them to improve classiﬁcation performance.
Revised: We search for the best values of them to improve classiﬁcation performance and show the results on Table

Revised: [Table]

Original: [Table]
Revised: [Table]

Original: I M ORE QUALITATIVE RESULTS
Revised: I M ORE QUANTITATIVE RESULTS

Original: We show more qualitative result on easy, hard and confused samples.
Revised: We demonstrate more results on easy, hard and confused samples.

Original: For one sample, we show the image in the upper left, distribution corresponding to the selected layer in the upper right and the distribution propagation across different layers in the bottom.
Revised: For each sample, we show the image in the upper left, distribution corresponding to the selected layer in the upper right and the distribution propagation across different layers in the bottom.