Title: On the Expressive Power of Deep Neural Networks
Authors: Maithra Raghu
Ben Poole
Jon Kleinberg
Surya Ganguli
Jascha Sohl-Dickstein
Venue: ICLR 2017
Keywords: Theory
Deep learning

Abstract: We study the expressive power of deep neural networks before and after
training. Considering neural nets after random initialization, we show that
three natural measures of expressivity all display an exponential dependence
on the depth of the network. We prove, theoretically and experimentally,
that all of these measures are in fact related to a fourth quantity, trajectory
length. This quantity grows exponentially in the depth of the network, and
is responsible for the depth sensitivity observed. These results translate
to consequences for networks during and after training. They suggest that
parameters earlier in a network have greater influence on its expressive power
– in particular, given a layer, its influence on expressivity is determined by
the remaining depth of the network after that layer. This is verified with
experiments on MNIST and CIFAR-10. We also explore the effect of training
on the input-output map, and find that it trades off between the stability
and expressivity of the input-output map.

TL;DR: Derives and explains the exponential depth sensitivity of different expressivity measures for deep neural networks, and explores consequences during and after training. 

Review discussion:
Kindly explain ``circular trajectory is chosen between two random vectors''.
Also ``non-zero magnitude perpendicular to x(t)''. Is this saying that the curve is not a ray from the origin?
In Theorem 1, random neural network means a fixed network with random weights?
The one dimensional trajectory is meant finite in length, differentiable?
Thank you very much for your questions! We will endeavor to make these terms clearer in the paper.
Circular trajectory: we pick points x_0, x_1 in the high dimensional space, and look at the curve x_0 cos(theta) + x_1 sin(theta) as theta varies between 0 and pi/2.
Non-zero magnitude perpendicular to x(t): Yes, exactly (we will add this to the statement to make it clearer.) To perform the analysis used in the theorem, we focus on an independent component by looking at the perpendicular, so for this proof, we need some non-zero perpendicular component.
Random neural network: In theorem 1, the network architecture (depth, width) is fixed, and we draw the weights from the initialization distribution (so weights are random). Note that the statement of theorem 1 is in expectation, and the expectation is taken over the weights (i.e. taken over the initialization distribution for the weights.)
The one dimensional trajectory: we are looking at finite length trajectories (as we want to e.g. interpolate between different data points), but we don't require the trajectory to be differentiable.
Hope this is helpful!
your papers proposes a proxy for function complexity (which you call expressivity)
You then study how function complexity changes with the number of layers and with training.
In several of you experiments, you seem to change the weight variance a lot.
- Why are the weight variances in your experiments much larger than what would usually be found at the beginning or the end of training a real, useful network ?
- One experiments (Fig 2) test several variance values values, another uses sigma2 = 8, another sigma2=2, then you switch to sigma1, then sigma2=16 and finally sigma2=3. These values seem very arbitrary, what is their significance ?
Page 2, what do you mean by "Furthermore, architectures are often compared via ‘hardcoded’ weight values – a specific function that can be represented efficiently by one architecture is shown to only be inefficiently approximated by another."
Thank you for your comments and questions.
The paper summary is correct (though our motivation was more to study expressivity via function complexity than the other way around), and we have three measures of functional complexity that we study for different architectures, before, during and after training.
Regarding the choice of weight variances: we broadly had two different types of experiments:
1) Experiments on the network after random initialization, which we used to verify that the theoretical results also held empirically.
2) Experiments studying the network during and after training to study how the results for random networks apply during and after training
As the theory (Theorem 1) predicts behavior for all values of sigma, the first type of experiments (Figures 1, 2) were carried out for many different values of sigma. Other experiments in the first type (Figure 4, 5) were run for many different values of sigma, but for ease of display, one was chosen (e.g. sigma2 = 8) to be the plot in the paper. The results also hold for all sigma in the exponential regime. We would be happy to include more/different plots if this would be helpful!
For the second kind of experiments, again, many different values of sigma were tested, and one was chosen for display in the paper. For Figures 6,7 we wanted to demonstrate that remaining depth is an important consideration even for smaller sigma, and for figures 8, 9 , we chose sigma2 = 3 and sigma2 = 16 as representatives of the two different regimes for training. Other weight variances could also be chosen here.
Page 2: Suppose there are two neural networks, f1 and f2. A common way to compare them is to pick a specific set of values for the weights of f1, and show that for those *specific* values, f2 doesn't have weight values that let it approximate f1 well. From this type of result however, it's hard to conclude that f1 is better than f2 just because *one* function of f1 (which is often artificially constructed) is not able to be approximated well by f2. This was part of the motivation to start with random initialization.
Summary of the paper:
Authors study in this paper quantities related to the expressivity of neural networks.The analysis is done for a random network. authors define the ‘trajectory length’ of a one dimensional trajectory as the length of the trajectory as the points (in a m- dimensional space) are embedded by layers of the network. They provide growth factors as function of hidden units k, and number of layers d.  the growth factor is exponential in the number of layers. Authors relates this trajectory length to authors quantities : ‘transitions’,’activation patterns ’ and ‘Dichotomies’.
As a consequence of this study authors suggest that training only  earlier layers in the network  leads higher accuracy then just training later layers. Experiments are presented on MNIST and CIFAR10.
Clarity:
The  paper is a little hard to follow, since  the motivations are not clear in the introduction and the definitions across the paper are not clear.
Novelty:
Studying the trajectory length as function of transforming the data by a multilayer network is   new and interesting idea. The relation to transition numbers is in term of the growth factor, and not as a quantity to quantity relationship.
Hence it is hard to understand what are the implications.
Significance:
The geometry of the input set (of dimension m)  shows up only weakly in the activation patterns analysis.  The trajectory study should tell us how the network organizes the input set. As observed in the experiments the network becomes contractive/selective as we train the network. It would be interesting to study those phenomenas using this trajectory length , as a measure for disentangling nuisance factors ( such as invariances etc.). In the supervised setting the network need not to be contractive every where , so it needs to be selective to the class label, a  theoretical study of the selectivity and contraction using the trajectory length would be more appealing.
Detailed comments:
Theorem 1:
- As raised by reviewer one the definition of a one dimensional input trajectory is missing.
- What does theorem 1 tells us about the design and the architecture to use in neural networks as promised in the introduction is not clear. The connection to transitions in Theorem 2 is rather weak.
Theorem 2:
- in the proof of theorem 2 it not clear what is meant by T and t. Notations are confusing, the expectation is taken with respect to which weight: is it W_{d+1} or (W_{d+1} and W_{d})? I understand you don't want to overload notation but maybe E_{d+1} can help keeping track. I don't see how the recursion is applied if T and t in it, have different definitions. seems T_{d+1} for you is a random variable and t_{d} is fixed. Are you fixing W_d and then looking at W_{d+1} as  random?
- In the same proof:  the recursion  is for d>1  ? your analysis is for W \in R^{k\times k}, you don't not study the W \in \mathbb{R}^{k\times m}. In this case you can not assume assume that |z^(0)|=1.
- should d=1, be analyzed alone to know how it scales with m?
Theorem 4 in main text:
- Is the proof missing? or Theorem 4 in the main text is Theorem 6 in the appendix?
Figures 8 and 9:
- the trajectory length reduction in the training isn't that just the network becoming contractive to enable mapping the training points to the labels? See for instance  on contraction in deep networks https://arxiv.org/pdf/1601.04920.pdf
- How much the plot depends on the shape of the trajectory? have you tried other then circular trajectory?
- In these plots the 2 mnist points had same label ? or different label?  both cases should be studied, to see the tradeoff between contraction and selectivity to the class label.
Thank you for the review! We will take the comments into account and endeavour to make the text even clearer.
A quick comment about motivation: our goal in this work improve interpretability in deep neural networks through a better understanding of neural network expressivity.
In particular, we look at different "diagnostics" (transitions/activation patterns/dichotomies) for measuring the expressiveness of different neural network architectures, and their practical consequences. The surprising fact that three natural measures of expressiveness are related by *direct* proportion (see below) to trajectory length suggests consequences on remaining depth (earlier parameters are more important to fit the final function) and a trade off between expressivity and stability during training due to initialization choices.
Responses inline to other specific comments below:
Trajectory Length: We will add this as a definition before Theorem 1. We take a 1-d trajectory to be a 1-d curve in the high dimensional space, and we measure the length  -- https://en.wikipedia.org/wiki/Arc_length -- of its images in the latent spaces.
Comment: "As a consequence of this study authors suggest"
Response: In addition to the results suggesting that training earlier layers is important (as the network is especially sensitive to perturbations in the earlier layers) we also examine how different initializations cause the network to trade off between expressivity and stability (figures 8,9.) Further investigation into the "optimal" (least trajectory growth/shrinkage required) could result in faster convergence.
Comment: "The relation to transition numbers is in term of the growth factor, and not as a quantity to quantity relationship"
Response: The relation *is* a quantity to quantity relationship -- confusion might have been caused as due to the direct proportionality relationship, the growth rates are also naturally related. The *direct* proportionality between transitions and trajectory length intuited by the paragraph below Observation 1, and then demonstrated empirically by Figure 2.
Figure 2 shows trajectory length and transitions for networks of different depths, widths and initialization (this might be made clearer by plotting each result as a separate point, and we are sorry if that line plot might be a source of confusion.) We find that the number of transitions are equal to the trajectory length (the quantity by quantity relationship) divided by a constant (approximately sigma_w/sqrt(k)).
Figure 10 also shows a direct proportion between transitions and dichotomies, and so the take away conclusion is that measuring trajectory length growth for different architectures immediately indicates the approximate values of other expressivity measures on those architectures.
Comment: "The geometry of the input set (of dimension m)  shows up only weakly in the activation patterns analysis."
Response: The activation pattern analysis is a general result, looking at any high dimensional input space (R^m, some m) and giving a novel result on how the hidden layers of the network *hierarchically* carve up the space into convex polytopes. The input dimension is an important part, as the upper bound on the number of regions is shown to be asymptotically tight by the construction in https://arxiv.org/abs/1402.1869
Comment: "What does theorem 1 tells us about the design and the architecture to use in neural networks as promised in the introduction is not clear"
Response: Theorem 1 proves a growth rate for trajectory length, and as trajectory length is directly proportional to the other measures of expressivity, it suggests that a deeper network is more expressive (and therefore has inherent capacity to fit data better) than a shallow network. This method of analysis helps generalize the benefit of depth described in e.g. https://arxiv.org/abs/1512.03965, https://arxiv.org/abs/1402.1869, etc
Comment: "The connection to transitions in Theorem 2 is rather weak"
Response: Theorem 2 has a precise connection to the number of transitions one could expect in a network of a certain depth and width in the large weight initialization regime. The proof (in more detail below) shows that the bound given by Theorem 1 on trajectory length is *exact* (up to a constant) in the number of transitions for such a network.
Comment: Comments on proof of Theorem 2
Response: Thank you for the constructive comments! We will state notation beforehand to make this proof clearer. As noted, T is a random variable (for layer d+1) and t is a fixed value (of number of transitions in the previous layer.) The expectation is taken with respect to the weights right before the neurons in layer d (so W^(d)) as that determines the number of neuron transitions in layer d+1, and we get the expected number of transitions in layer d+1 given the number of transitions in layer d.
The recursion is indeed for d+1 > 1 (so d the first hidden layer onwards), and we will make that clearer. (This is what enables us to assume that the input is saturated, and has absolute value 1.)
For the input layer, where W \in \mathbb{R}^{k\times m}, note that the initialization scale is slightly different -- as is standard in practice, the initialization is N(0, sigma_w^2/input_dim). In the second hidden layer on, the input dim is k, but in the first hidden layer, the input dim is = m, and so this difference is taken into account.
Comment: Proof of Theorem 4
Response: Thank you for the comment, we will make this clearer. Theorem 4 can be derived quickly after Theorem 6 is proved, and the connection is thus stated immediately after Theorem 6, but could be made clearer! In more detail:
We show (theorem 6) that k hyperplanes in R^m result in r(k,m) regions, which is <= O(k^m) (sum of binomial coefficients).
So, the first hidden layer of the deep neural network carves up input space into r(k,m) regions <= O(k^m).
Consider the second hidden layer  -- this has a different hyperplane arrangement for *every* region created by the first hidden layer (which has at most r(k,m) regions).
So, in total, there are r(k,m)*r(k,m) (number of regions created by first layer*number of regions created by second layer in each region of first layer.).
Continuing in this way, for a network with n hidden layers, there are at most r(k,m)*n <= O(k^mn) regions total, giving the result in Theorem 4.
Thanks again for the comments, and we will update the paper making this clearer!
Comment: Figures 8 and 9
Response: Thank you for the related link! Actually, it is not a straightforward contraction that happens, because plotting the weight norms during training shows that the weight norms don't change *too* much. We suspect the network is changing the exact point in parameter space to result in less of an expansion.
Other trajectories besides circular were tried (and the theory holds for most trajectories) and similar results were observed. We can include more plots if this would help?
Both cases (same/different labels) were tried, with similar results.
Thanks again for the detailed comments, and we will incorporate them in the next version to address them!
In response to Reviewer2's helpful comments, we've submitted a revision with easier to read proofs of Theorems 2, 4.
Thank you very much for the constructive comments!
SUMMARY
This paper studies the expressive power of deep neural networks under various related measures of expressivity.
It discusses how these measures relate to the `trajectory length', which is shown to depend exponentially on the depth of the network, in expectation (at least experimentally, at an intuitive level, or theoretically under certain assumptions).
The paper also emphasises the importance of the weights in the earlier layers of the network, as these have a larger influence on the represented classes of functions, and demonstrates this in an experimental setting.
PROS
The paper further advances on topics related to the expressive power of feedforward neural networks with piecewise linear activation functions, in particular elaborating on the relations between various points of view.
CONS
The paper further advances and elaborates on interesting topics, but to my appraisal it does not contribute significantly new aspects to the discussion.
COMMENTS
- The paper is a bit long (especially the appendix) and seems to have been written a bit in a rush.
Overall the main points are presented clearly, but the results and conclusions could be clearer about the assumptions / experimental vs theoretical nature.
The connection to previous works could also be clearer.
- On page 2 one finds the statement ``Furthermore, architectures are often compared via ‘hardcoded’ weight values -- a specific function that can be represented efficiently by one architecture is shown to only be inefficiently approximated by another.''
This is partially true, but it neglects important parts of the discussion conducted in the cited papers.
In particular, the paper [Montufar, Pascanu, Cho, Bengio 2014] discusses not one hard coded function, but classes of functions with a given number of linear regions.
That paper shows that deep networks generically* produce functions with at least a given number of linear regions, while shallow networks never do.
* Generically meaning that, after fixing the number of parameters, any function represented by the network, for parameter values form an open, positive -measure, neighbourhood, belongs to the class of functions which have at least a certain number of linear regions.
In particular, such statements can be directly interpreted in terms of networks with random weights.
- One of the measures for expressivity discussed in the present paper is the number of Dichotomies. In statistical learning theory, this notion is used to define the VC-dimension. In that context, a high value is associated with a high statistical complexity, meaning that picking a good hypothesis requires more data.
- On page 2 one finds the statement ``We discover and prove the underlying reason for this – all three measures are directly proportional to a fourth quantity, trajectory length.''
The expected trajectory length increasing exponentially with depth can be interpreted as the increase (or decrease) in the scale by a composition of the form a*...*a x, which scales the inputs by a^d. Such a scaling by itself certainly is not an underlying cause for an increase in the number of dichotomies or activation patterns or transitions. Here it seems that at least the assumptions on the considered types of trajectories also play an important role.
This is probably related to another observation from page 4: ``if the variance of the bias is comparatively too large...
then we no longer see exponential growth.''
OTHER SPECIFIC COMMENTS
In Theorem 1
- Here it would be good to be more specific about ``random neural network'', i.e., fixed connectivity structure with random weights, and also about the kind of one-dimensional trajectory, i.e., finite in length, closed, differentiable almost everywhere, etc.
- The notation ``g \geq O(f)'' used in the theorem reads literally as |g| \geq \leq k |f| for some k>0, for large enough arguments. It could also be read as g being not smaller than some function that is bounded above by f, which holds for instance whenever g\geq 0.
For expressing asymptotic lower bounds one can use the notation \Omega (see https://en.wikipedia.org/wiki/Big_O_notation).
- It would be helpful to mention that the expectation is being taken with respect to the network weights and that these are normally distributed with variance \sigma.
- Theorem 2. Here it would be good to be more specific about the kind of sign transitions. Is this about transitions at any units of the network, or about sign transitions at the scalar output of the entire network.
- Theorem 3 is quite trivial.
The bijection between transitions and activation patterns is not clear.
Take a regular n-gon in the plane and a circle that crosses each edge twice.
This makes 2n transitions but only n+1 activation patterns.
- Theorem 4.
Where is the proof of this statement?
How does this relate to the simple fact that each activation pattern corresponds to the vector indicating the units that are `active'?
MINOR COMMENTS
- The names of the theorems (e.g. ``Bound on ...'' in Theorem 1) could be separated more clearly from the statements, for instance using bold font, a dot, or parentheses.
- On page 4, in Latex one can use \gg for the `much larger' symbol.
- On page 4, explain the notation \delta z_\orth.
- On page 4, explain that ``latent image'' refers to the image in the last layer.
- Why are there no error bars in Figure 2?
- On page 5 explain that the hyperplane is in the last hidden layer.
- On page 5, ``is transitioning for any input''. This is not clearly stated, since a transition takes place at a point in a trajectory of inputs, not for a single input.
- The y-axis labels in Figure 1 (c) and (d) are too small.
- Why are there no error bars in Figure 1 (a) and (b)? The caption could at least mention that shown are the averages over experiments.
- In Figure 4 (b) the curves are occluded by the labels.
- The numbering of results is confusing. In the Appendix some numbers are repeated with the main part and some are missing.
- On page 19. Theorem 6. As far as I remember Stanley also provides an elementary proof of case with hyperplanes in general position. Many other works also provide elementary proofs using the same induction arguments in what is known as the sweep hyperplane method.
Thank you for your careful reading, and specific feedback! We are working to improve the paper based on your suggestions. Some responses to specific points follow below:
***"In particular, the paper [Montufar, Pascanu, Cho, Bengio 2014] discusses not one hard coded function, but classes of functions with a given number of linear regions.
That paper shows that deep networks generically* produce functions with at least a given number of linear regions, while shallow networks never do. "
This is untrue. In [Montufar et al, 2014], in Lemma 2 they put a lower bound on the *maximum* number of linear regions that can be achieved by adjusting weights. They also discuss in the text how weights can be perturbed, even for a network with an exponential number of linear regions, without changing the number of linear regions. Nowhere do they establish bounds on the expected number of linear regions for generic weights.
***"One of the measures for expressivity discussed in the present paper is the number of Dichotomies. In statistical learning theory, this notion is used to define the VC-dimension. In that context, a high value is associated with a high statistical complexity, meaning that picking a good hypothesis requires more data."
Thank you for the connection. We actually discuss this in Appendix D.1, and will improve the reference to this discussion in the text. As you note, the literature on VC dimension typically takes a negative view of complexity, and treats it as a measure of how badly a class of functions could overfit to the training data. In this paper we take an opposite perspective, and instead treat complexity as a measure of how powerful and flexible a class of functions is, which is a view adopted from the combinatorial literature on the Sauer-Shelah lemma.
***"On page 2 one finds the statement ``We discover and prove the underlying reason for this – all three measures are directly proportional to a fourth quantity, trajectory length.''
The expected trajectory length increasing exponentially with depth can be interpreted as the increase (or decrease) in the scale by a composition of the form a*...*a x, which scales the inputs by a^d. Such a scaling by itself certainly is not an underlying cause for an increase in the number of dichotomies or activation patterns or transitions. Here it seems that at least the assumptions on the considered types of trajectories also play an important role.
This is probably related to another observation from page 4: ``if the variance of the bias is comparatively too large...
then we no longer see exponential growth.''"
Interpreting the increase in trajectory length as the result of a composition a*a*a..*x is important, but only part of the whole picture. As we examine hard tanh, there is another  competing factor, which is whether neurons are saturated or not -- if neurons are saturated, then we cannot expect a perturbation to experience exponential growth and hence the trajectory itself will not grow.
From the outset, it is unclear which of these will win out.
But it turns out that even with saturation effects, the composition effects are enough to cause trajectory growth.
The reviewer is correct in that the increase of transitions is not due to this alone -- it also has to do with autocorrelation length, which also decreases exponentially (explored further in https://arxiv.org/abs/1606.05340). The only (mild) condition we need on the trajectory for this to be true is that the tangent vector must have a perpendicular component to the current point. (So for the proof to go through, it can't be a line from the origin.)
***"In Theorem 1
- Here it would be good to be more specific about ``random neural network'', i.e., fixed connectivity structure with random weights, and also about the kind of one-dimensional trajectory, i.e., finite in length, closed, differentiable almost everywhere, etc.
- The notation ``g \geq O(f)'' used in the theorem reads literally as |g| \geq \leq k |f| for some k>0, for large enough arguments. It could also be read as g being not smaller than some function that is bounded above by f, which holds for instance whenever g\geq 0.
For expressing asymptotic lower bounds one can use the notation \Omega (see https://en.wikipedia.org/wiki/Big_O_notation).
- It would be helpful to mention that the expectation is being taken with respect to the network weights and that these are normally distributed with variance \sigma."
Thank you for these suggestions! We are making all of these changes (differentiable everywhere is not required.)
***"- Theorem 2. Here it would be good to be more specific about the kind of sign transitions. Is this about transitions at any units of the network, or about sign transitions at the scalar output of the entire network. "
The theorem as stated is for the expected number of transition of the readout neuron in an n-layer network. The same relationship holds for any neuron in the network, as a function of that neuron's depth. We are clarifying this in the text.
***"- Theorem 3 is quite trivial. "
Note that Theorem 3 states that activation patterns of the *entire network* subdivide input space into convex polytopes! See Figure 3 for an illustration of this. We have found in conversation that our colleagues find it obvious that a single layer in a deep network subdivides the layer immediately below into convex polytopes, but are typically surprised to discover that the pattern of activity across all layers of the network subdivides input space into convex polytopes.
***"The bijection between transitions and activation patterns is not clear.
Take a regular n-gon in the plane and a circle that crosses each edge twice.
This makes 2n transitions but only n+1 activation patterns."
This was stated and proved as only true for affine trajectories in the appendix but we will clarify this further in the text -- thank you for pointing it out!
***"- Theorem 4.
Where is the proof of this statement?
How does this relate to the simple fact that each activation pattern corresponds to the vector indicating the units that are `active'? "
The proof of theorem 4 follows after the proof of theorem 6 in the appendix -- apologies that this was not more clearly marked before, but theorem 4 follows readily from the proof of theorem 3 and the statement of theorem 6.
This theorem gives an asymptotically tight bound on how many such unique vectors we can expect from a real neural network. The naive bound would be 2^(number of neurons) - so in our case, 2^(kn + m).
But all of these combinations are not possible. E.g. if we trace a line in our input space and look at the on off pattern of the first hidden layer, each unit can only switch from on to off (or vice versa), *once*.
So we get k different patterns, not 2^k.
MINOR COMMENTS
"On page 19. Theorem 6. As far as I remember Stanley also provides an elementary proof of case with hyperplanes in general position. Many other works also provide elementary proofs using the same induction arguments in what is known as the sweep hyperplane method."
Stanley's proof method uses intersection posets to prove Zaslavsky's theorem a special case of which gives the bound in general position. At the time of writing, the authors were unable to find an elementary proof of this theorem.
We will make all the other changes you suggest in MINOR COMMENTS. Thank you again for your careful reading and actionable suggestions!
Kindly note that a property holds for `generic weights' when this property still holds after perturbing a particular choice of the weights, which is precisely what you write.
Roughly speaking this means that the property holds in a region of parameters which has a positive volume.
The actual size of this region will of course depend on how one measures the space, as does the expectation value.
Measuring the space of parameters with respect to a normal distribution is one possibility which also can be interpreted in terms of perturbations around the mean.
Apologies for the confusion-- of course perturbing after a particular choice of weights means that we have a set of measure > 0 (some epsilon ball say) for which this property holds.
But this still requires that one lands in that epsilon ball to get that number of linear regions, and it is unclear how many such epsilon balls there are (discounting the permutations of the existing construction) and how "close" they are to the typical starting point (after random initialization) of a neural network.
Our use of the word "generic" was to indicate that we do away with having to think about such a specific construction, and show that even by drawing weights iid centered around 0, we *still* end up with exponential growth with depth.
I think this is important, so I want to contribute an additional followup.
In our paper we use generic to mean that a property applies for "typical" or "average case" parameters. We then define "average case" for our specific results -- usually as an expectation over deep networks with weights drawn from a Gaussian.
You are defining generic to mean within an epsilon ball of hand chosen parameters. Under this definition of generic, I agree that your assesment of previous work is correct.
However, this usage of generic is nearly the opposite of its colloquial usage.
For instance, using this modified definition of generic: I have a very specific image, of an adorable puppy. If you slightly perturb the pixels in this image, it will still be of my puppy.
Therefore, "this image is of my puppy" is a statement that holds for "generic" images. If you generate a new "generic" image, you can happily expect it to also have this property, and be of my puppy.
We are rather interested in the generic (now meaning "typical") behavior of neural networks, since we are interested in predicting properties of neural networks that can be expected to occur in practice after initialization and during training, rather than just in carefully constructed scenarios. Continuing with the image example, under this definition some generic properties of images would be their power-law falloff in fourier magnitude with frequency, and the existence of oriented edges.
Thank you again for taking the time to carefully review our work.
This paper presents a theoretical and empirical approach to the problem of understanding the expressivity of deep networks.
Random networks (deep networks with random Gaussian weights, hard tanh or ReLU activation) are studied according to several criterions: number of neutron transitions, activation patterns, dichotomies and trajectory length.
There doesn't seem to be a solid justification for why the newly introduced measures of expressivity really measure expressivity.
For instance the trajectory length seems a very discutable measure of expressivity. The only justification given for why it should be a good measure of expressivity is proportionality with other measures of expressivity in the specific case of random networks.
The paper is too obscure and too long. The work may have some interesting ideas but it does not seem to be properly replaced in context.
Some findings seem trivial.
detailed comments
p2
"Much of the work examining achievable functions relies on unrealistic architectural assumptions such as layers being exponentially wide"
I don’t think so. In "Deep Belief Networks are Compact Universal Approximators" by Leroux et al., proof is given that deep but narrow feed-forward neural networks with sigmoidal units can represent any Boolean expression i.e. A neural network with 2n−1 + 1 layers of n units (with n the number of input neutron).
“Comparing architectures in such a fashion limits the generality of the conclusions”
To my knowledge much of the previous work has focused on mathematical proof, and has led to very general conclusions on the representative power of deep networks (one example being Leroux et al again).
It is much harder to generalise the approach you propose, based on random networks which are not used in practice.
“[we study] a family of networks arising in practice: the behaviour of networks after random initialisation”
These networks arise in practice as an intermediate step that is not used to perform computations; this means that the representative power of such intermediate networks is a priori irrelevant. You would need to justify why it is not.
“results on random networks provide natural baselines to compare trained networks with”
random networks are not “natural” for the study of expressivity of deep networks. It is not clear how the representative power of random networks (what kind of random networks seems an important question here) is linked to the representative power of (i) of the whole class of networks or (ii) the class of networks after training. Those two classes of networks are the ones we would a priori care about and you would need to justify why the study of random networks helps in understanding either (i) or (ii).
p5
“As FW is a random neural network […] it would suggest that points far enough away from each other would have independent signs, i.e. a direct proportionality between the length of z(n)(t) and the number of times it crosses the decision boundary.”
As you say, it seems that proportionality of the two measures depends on the network being random. This seems to invalidate generalisation to other networks, i.e. if the networks are not random, one would assume that path lengths are not proportional.
p6
the expressivity w.r.t. remaining depth seems a trivial concerns, completely equivalent to the expressivity w.r.t. depth. This makes the remark in figure 5 that the number of achievable dichotomies only depends *only* on the number of layers above the layer swept seem trivial
p7
in figure 6 a network width of 100 for MNIST seems much too small.
Accordingly performance is very poor and it is difficult to generalise the results to relevant situations.

Meta-review:
role: PC
data: comment: While the reviewers saw some value in your contribution, there were also serious issues, so the paper does not reach the acceptance threshold.

Decision: Reject