Title: The Incredible Shrinking Neural Network: New Perspectives on Learning Representations Through The Lens of Pruning
Authors: Nikolas Wolfe
Aditya Sharma
Lukas Drude
Bhiksha Raj
Venue: ICLR 2017
Keywords: Theory
Deep learning

Abstract: How much can pruning algorithms teach us about the fundamentals of learning representations in neural networks? A lot, it turns out. Neural network model compression has become a topic of great interest in recent years, and many different techniques have been proposed to address this problem. In general, this is motivated by the idea that smaller models typically lead to better generalization. At the same time, the decision of what to prune and when to prune necessarily forces us to confront our assumptions about how neural networks actually learn to represent patterns in data. In this work we set out to test several long-held hypotheses about neural network learning representations and numerical approaches to pruning. To accomplish this we first reviewed the historical literature and derived a novel algorithm to prune whole neurons (as opposed to the traditional method of pruning weights) from optimally trained networks using a second-order Taylor method. We then set about testing the performance of our algorithm and analyzing the quality of the decisions it made. As a baseline for comparison we used a first-order Taylor method based on the Skeletonization algorithm and an exhaustive brute-force serial pruning algorithm. Our proposed algorithm worked well compared to a first-order method, but not nearly as well as the brute-force method. Our error analysis led us to question the validity of many widely-held assumptions behind pruning algorithms in general and the trade-offs we often make in the interest of reducing computational complexity. We discovered that there is a straightforward way, however expensive, to serially prune 40-70\% of the neurons in a trained network with minimal effect on the learning representation and without any re-training. 

TL;DR: Pruning algorithms reveal fundamental insights into neural network learning representations

Review discussion:
The page limit are 8 pages, you have 23.
Hi,
Thank you for your question. It is mentioned in the call for papers (http://www.iclr.cc/doku.php?id=iclr2017:callforpapers) that there is no strict page limit enforced. 8 pages (without References) seems to be the recommended length but is not seemingly a hard limit and that's the impression the authors were under while writing the manuscript. You will notice that due to the bulk of experiments carried out and subsequent discussion on the results, it is very difficult to stay within the recommended page limit.
That said, the authors are more than happy for feedback on how the reviewers and the readers think the length can be shortened and will try their best to incorporate that feedback.
Please do let us know if we can answer any further questions.
Thanks.
The paper describes and analyses a more advanced method for pruning neural networks. The analysis seems quite solid but after a first read I am left with the following three comments that I hope the authors can address:
1) I find the writing and the figures excessively long and fairly repetitive. E.g. all figures show the gain from 0 to 10, but I am not sure what I should learn from gains > 1 (other then sanity checks)?
At least that part of the figure, which takes up 90% of the space, should be thoroughly reduced.
Plus, the take-away gist of the paper is quite bounded, in my opinion not justifying a paper of more then eight pages.
2) Your title includes “new perspectives on learning representations through the lens on pruning”, and it is indeed an interesting observation that large parts of the neurons can be pruned and the representation is thus not evenly spread.
However, this has been observed earlier (e.g. by brute-force pruning), so I am not sure how your methods expands on this. In the conclusions you argue that your results suggest that Mozer was right in his dualist view, in particular that many neurons simply cancel each other. You seem to base this conclusion on the “bumpy” error landscape you get for successive pruning with your method, but if at all the “smooth” results from the brute-force pruning suggests that this is rather an artefact of your method then any insight into the neural representations.
3) Currently I feel your paper is not so much about representations but about a more efficient method for pruning.
However, all results you show are “toy examples” in the sense that they can be solved with brute-force pruning; no applications on realistically sized networks (e.g. VGG) are shown.
Thank you for your questions! These are all very important issues to address, and it's good you asked.
First off, I hope you are looking at the latest revision of the paper.
Responses are in-line...
1) I find the writing and the figures excessively long and fairly repetitive. E.g. all figures show the gain from 0 to 10, but I am not sure what I should learn from gains > 1 (other then sanity checks)?
At least that part of the figure, which takes up 90% of the space, should be thoroughly reduced.
Plus, the take-away gist of the paper is quite bounded, in my opinion not justifying a paper of more then eight pages.
>> Re: Excessively long / repetitive: We would not feel comfortable asserting the conclusions we have drawn here without providing a record of at least a few corroborating experiments. Had we shown the results on only one experiment, the conclusion could be limited to the peculiarities of that one experiment / dataset, but we show that this result holds across several different experiment types and datasets. Each experiment demonstrates a different aspect of the phenomenon we describe. For one, we show that the second and first order methods seldom behave predictably. One is typically better than the other but by no means do these trends hold, and there are examples where the trend does not hold.
Moreover, the brute force method seems to work in all cases and this suggests that there is in fact a way to prune optimally in a serial fashion without re-training, even if computationally expensive, and we argue this should be explored further because this is a very appealing result.
Next, we show that this phenomenon holds for both regression and classification problems.
Finally, we show that there is a completely unique phenomenon which arises when the starting network has not been optimized on the training objective.
Re: Gain values: The purpose of showing the gain values higher than 1.0 is two-fold: 1.) to give the reader a sense of the overall shape of the error surface as seen by individual hidden units, and 2.) to show the way in which the network output is sensitive to the perturbation of their incoming and outgoing weights.
For example: If I showed you that for a given hidden unit between gain values 0 and 1.0, the curvature of the error surface was flat, you might argue that this particular neuron was not important to the network and could be pruned.
But if the curvature is STILL flat all the way out past a gain value of 10, that is clearly significant: That means that this neuron's output could be amplified 10-fold, and the network output would remain unchanged. Under no circumstances could a network with a well-distributed learning representation contain 40% hidden units whose outputs could simply be cranked up to 10x their original value with zero effect on the output.
But what if the slope increases or decreases past 1.0? Should the neuron's importance be amplified or should it be pruned outright? We ran many experiments in our research and we selected to show the information past 1.0 because we feel it is crucial to understanding the intrinsic nature the error surface with respect to each neuron. Taking a closer look allows you to challenge your own perceptions about what individual neurons are really doing, and what back-propagation really learns. To our knowledge, no other paper shows such plots, so we found them fascinating, and we are sorry you do not share our enthusiasm. :-)
Re: Length of the paper: We took pains in the paper to distinguish the work we're doing from simply presenting a new algorithm and its results. We are offering an extensive evaluation of a phenomenon that, from our survey of the literature, is not well understood. If you have any suggestions as to how to cut down the paper or what sections / information to cut, we are all ears.
2) Your title includes “new perspectives on learning representations through the lens on pruning”, and it is indeed an interesting observation that large parts of the neurons can be pruned and the representation is thus not evenly spread.
However, this has been observed earlier (e.g. by brute-force pruning), so I am not sure how your methods expands on this. In the conclusions you argue that your results suggest that Mozer was right in his dualist view, in particular that many neurons simply cancel each other. You seem to base this conclusion on the “bumpy” error landscape you get for successive pruning with your method, but if at all the “smooth” results from the brute-force pruning suggests that this is rather an artefact of your method then any insight into the neural representations.
>> We want to stress that we are not presenting this paper as a benchmarking study of our algorithm. The brute force approach is neither a previously discussed algorithm, nor are we suggesting that we are improving upon it. We used the brute force method to form a baseline. The second order method we derive -- though novel in its own right -- draws largely on historical literature, and we are not the first to propose using Taylor methods to inform pruning decisions. The crux of our conclusion, really, is that such methods are -- and have always been -- flawed, though existing pruning algorithms compensate for this flaw through generous amounts re-training. Empirically, we observe that first and second order methods provide a poor approximation of the error surface with respect to individual network parameters.
Furthermore, we do not have to accept the conclusion that re-training is a necessary part of pruning because a brute force search reveals that neurons can in fact be pruned from trained networks in a piecemeal fashion with no retraining and minimal adverse effect on the overall performance of the network. This would be impossible if neurons did not belong to the distinct classes we describe.
The conclusions we draw regarding the dualism of neuron roles is not based on the shape of the first and second order method curves. It is drawn holistically looking at a brute force method's ability to prune half a network with no re-training or impact on performance, and from observing that clearly certain groups of neurons must be pruned together to minimize impact on the network output. If the learning representation were distributed in some fashion over all (or even most) neurons, it would be impossible to observe this phenomenon, because each sequential removal would remove some component of the learning representation.
But this is not what we observe in practice.
3) Currently I feel your paper is not so much about representations but about a more efficient method for pruning.
However, all results you show are “toy examples” in the sense that they can be solved with brute-force pruning; no applications on realistically sized networks (e.g. VGG) are shown.
>> In multiple sections of the paper we have tried to make clear that we are not trying to "sell" a new algorithm or pruning method. We are using pruning methods to begin an investigation of the nature of learning representations in neural networks. Our derived algorithm is an improvement over existing neuron-based pruning methods, but it turns out that you can do a lot better by just using a brute-force search and no re-training. The experiments we ran are on the well-studied and understood MNIST dataset, which is a standard benchmark for research science, not a "toy example." Furthermore, there is no theoretical reason to suppose that the phenomena we observe here are limited only to these experiments. A "realistically sized network" does not magically spread the learning representation over all neurons. Neural network architectures are typically designed heuristically, i.e. they are over-parameterized "by design" in practice so as to avoid inhibiting learning.
Whenever this is the case, what we are asserting with this paper is that the network will NOT learn a representation which seeks to spread itself over the parameters provided and/or use the full parameter space to enrich its representation. It will instead learn a minimal network implicitly, and the superfluous and redundant parameters will learn a complex interdependence function to cancel each other out. This is a fundamental property of neural networks in general.
Thanks for your enlightening response which will be very helpful for the full review of the paper!
One of our observations is that first and second order approximations do not really provide a consistently reliable estimate of the contribution of a network parameter to the overall training objective. An unstated corollary of this is that pruning methods that use first (or even second order) criteria to select parameters to prune prior to retraining may not fare much better than those that select these at random.
I am trying to understand the necessity of the appendix in light of modern technologies.  Can you please clarify: with the pervasive availability and ease of automatic differentiation, are there additional insights to be gained from this appendix and this detailed computation, and if so, what?  I appreciate the genuine effort to calculate this derivative, and I'm simply trying to understand if there is a more subtle point here.
Thank you for your question. We have taken every effort to ensure that our work is easliy readable and understandable for anyone who is new to the field.
While the available technologies might make it very easy to calculate these derivatives, we did not want any part of our work to feel like a black box for someone who might be interested in implementing our ideas and improving them. This was the major motivation behind adding the entire derivation of the equations used in the main body of the paper as an appendix. The intermediate equations in this derivation will give the readers a better insight into how we used the network structure to evaluate quadratic gradients and they can directly use these intermediate equations if needed for any future ideas.
We live in a time where research papers focus way more on results than on explaining how they were achieved and set up. We strongly believe that talking more about the implementational details of the methods used (at least the math behind them) and making manuscripts easy to read will inspire more people to take ideas forward.
That said, if deemed necessary by the reviewers, we can surely remove the appendix in the final version of the paper.
To add to what has already been noted: In our survey of the existing literature, we constantly struggled with inconsistent notation. In many cases, complicated terms are left unexplained, or are simply stated without mentioning the non-trivial amount of work required to implement them in code.
For example, in Yann LeCun's original Optimal Brain Damage paper, the weight salience metric is simple but the rule for computing the second derivative with respect to weights is not very clear. One could not trivially implement it from reading the paper, in other words. In deriving our method, we set out to provide steps which could be literally translated into code. Our index notation, layering notation and ordering of terms are always consistent such that you do not need to re-do the work in order to implement this yourself.
You might want to implement second order neuron pruning with some of the more obvious tweaks such as re-training. This enables you to do that. Or, you might want to implement second order back-propagation, which is most certainly not available in most standard toolkits. The historical literature on second-order back-propagation methods such as Scott Fahlman's quickprop method is actually quite promising, even if it has been forgotten in recent years. By most accounts it should be faster than first-order, or standard back-propagation.
Of course, deriving the second-order weight update rule is not perfectly straightforward, but a fairly simple extension of the work we have provided. The derivative terms are there, they just need to be extended to perform the weight update rule.
When we developed this document we realized it would be a rather useful reference for the research community. We didn't extend it for back-propagation weight updates because that is outside the scope of the paper, but to our knowledge, a simple-to-read reference like this does not exist, though not-very-easy-to-read ones are certainly out there (e.g. http://page.mi.fu-berlin.de/rojas/pub/second_order_backpropagation1993.pdf).
So, we decided to just leave it in. :-)
For code reference, you can take a look at our toolkit here: https://github.com/nikwolfe7/neural-network
The second order pruning method implemented for single neurons is located here: https://github.com/nikwolfe7/neural-network/blob/master/src/mlsp/cs/cmu/edu/dnn/elements/GainSwitchNeuron.java
See the implementation of the backward() method.
We developed this toolkit more or less entirely for the sake of the paper. Enjoy!
First Revision: 27 November, 2016. Added Section 4.5: Investigation of Pruning Performance with Imperfect Starting Conditions. We discuss the impact of pruning sub-optimally trained networks in this section by specifically analyzing performance of networks classifying the digits 0, 1 and 2 of the MNIST database.
Here are answers to some common questions the authors have been asked about the current work in the past by readers of the manuscript. We hope these will help clarify any other questions our reviewers/readers might have.
Q: Why doesn't the paper present numerical comparision to state-of-the-art/recent pruning techniques?
A: Under certain motivational assumptions, it is understandable to demand benchmarking comparisons against state-of-the-art methods, but this may be missing the fundamental purpose of the present research. Our investigation is intended less to propose a competing alternative to existing pruning techniques and more to shed light on the limitations of generally accepted approaches to pruning and the degree to which increased numbers of parameters affect learning representations in neural networks. The paper does talk about most, if not all popoular pruning techniques out there.
In fact, we examined the literature for numerical methods to approximate the importance of network elements, and the widely-cited 1st & 2nd order techniques proposed by Mozer, LeCun, Hassibi, Stork, et al. provided our initial inspiration. This is the jumping off point for our research in terms of key insights.
Q: The idea of using Taylor series approximations seems interesting but not really effective.
A: It is not effective when used as a pruning technique but it is VERY effective to test out the effectiveness of existing pruning techniques, which is what we do here. We have mentioned it multiple times in the paper that the motivation behind this work is NOT to propose a new pruning technique that will outperform all other techniques out there but to tap into learning representations to see how effective our established techniques are when seen from the perspective of representations. The Taylor series approximations play an important role here.
A lot of pruning techniques out there use 2nd Order error gradients and assume that using them is the most effective way to prune networks. We have conclusively proved using the Taylor series that this is very much not the case. Our results with the brute-force method show us that there is a much larger extent to which networks can be pruned. This makes for a great starting-off point for future research to find methods that can produce similar results.
Q: Why did you decide in favor of sigmoid activation functions instead of something more recent and more popular like ReLUs?
A: As mentioned above, the main contribution of this work is to demonstrate the feasibility of pruning entire neurons from trained networks, and offer novel insight on learning representations. We use Taylor methods to approximate the results achieved by the brute-force method but this is not an ideal solution to the problem, as we discuss. The 2nd order approximation technique will not work for ReLU networks because ReLUs do not have a 2nd derivative, unless we use the soft-plus function as a continuous approximation.
Furthermore, due to the fact that we are approximating the error surface of a network element with respect to the output using a parabola, if there is no useful parabola to approximate this relationship, then the method breaks down. The derivatives of the activation function are simply parameters of the Taylor series. It doesn’t cease to be a parabolic approximation or become more effective if we use a different doubly-differentiable activation function.
Q: Why carry out your experiments on the MNIST dataset and not go for a larger and more practical image dataset?
A: All experiments were necessarily carried out on optimally trained networks (not counting Section 4.5, which specifically examines non-optimally trained networks), so there is no way to improve them. We derived the algorithm assuming the well-studied sigmoid activation function.
Furthermore, the MNIST dataset is a de-facto standard for demonstrating the potential of new techniques. A different dataset, task, activation function, or network architecture will not change the trends we see in the results but could make the results less interpretable.
Q: The best setting is Iterative Re-ranking with Brute Force removal which is too expensive.
A: The brute-force method is highly parallelizable, so time complexity is not necessarily a deal-breaker. Our focus is the proof of concept, and we intend to investigate potential speedups in future work.
Also, since pruning is anyways a single step carried out after the training process is over (which usually takes orders of magnitude more time), this is potentially acceptable.
The paper introduces a new pruning method for neural networks based on the second-order Taylor expansion and compares the results against a first-order method and brute-force pruning. It performs experiments of the three methods on several toy examples - including a two-layer network on MNIST - and shows that the second-order method behaves much worse then the brute-force baseline.
In addition, from the success of the brute-force pruning the authors conclude that the hypothesis of Mozer et al - that neurons either contribute to performance or cancel out the effect of other neurons - is probably correct.
The authors put in considerable effort to explain all details of the paper clearly and at length, so the content of the paper is accessible even to people novel to pruning methods.
Additionally, the authors have very carefully answered all questions that were coming up through the pre-review and have been very responsive.
My major criticism is that the paper lacks focus, does not have a concrete conclusion and does not explain what it adds to the literature. To make this apparent, I here summarise each paragraph of the conclusion section:
Paragraph 1: We do not benchmark / Pruning methods do not fare well against brute-force baseline / Some evidence for hypothesis of Mozer & Smolensky, but further investigation needed
Paragraph 2: Introduced 2nd order Taylor method / Does not fare well against baseline
Paragraph 3: Re-training may help but is not fair
Paragraph 4: Brute-force can prune 40-70% in shallow networks
Paragraph 5: Brute-force less effective in deep networks
Paragraph 6: Not all neurons contribute equally to performance of network
The title of the paper and answers of the authors to the pre-review questions seemed to strongly suggest that the paper is not about the new second-order method, is not about benchmarking pruning algorithms but is instead about the learnt representations.
But only two or three sentences in the conclusion, and no sentence in the part on results in the abstract, even refers to neural representations. In an answer to the pre-review questions the authors stated:
> Furthermore, we do not have to accept the conclusion that re-training is a necessary part of pruning because a brute force search reveals that neurons can in fact be
> pruned from trained networks in a piecemeal fashion with no retraining and minimal adverse effect on the overall performance of the network. This would be
> impossible if neurons did not belong to the distinct classes we describe."
But this can already be concluded from the 2nd order method, which has a similar characteristic and is based on other 2nd order methods (not shown here). What is the motivation to introduce a new 2nd order method here?
In addition, some other minor conclusions about representations - in particular the cancellation effect - might be based on side-effects of the greedy serial pruning method. Optimally, one would need to consider all the different ways of pruning (which, of course, scales exponentially with the number of neurons and is computationally infeasible).
Notably, the authors do consider this limitation in the context of conventional pruning methods in the conclusions: "Third, we assumed that pruning could be done in a serial fashion [...]. We found that all of these assumptions are deeply flawed in the sense that the true relevance of a neuron can only be partially approximated [...] at certain stages of the pruning process".
But the brute-force pruning process is also serial - why is that not a problem?
All in all it is unclear to me what the paper adds: there are little conclusions regarding the learnt representations nor is there sufficient benchmarking against state-of-the-art pruning methods. I would suggest to focus the paper in the following way: first, use a state-of-the-art pruning method from the literature (that works without re-training) or do not use any other pruning methods besides brute-force (depending on whether you want to compare pruning methods against brute-force, or want to learn something about the learnt representations). In this way you need to write little about this second-order tuning methods, and readers are not so easily confused about the purpose of this paper (plus it will be considerably shorter!).
Then concentrate on 2-layer MNIST and a deeper CIFAR10 network.
Further focus the paper by adding an itemised list of the exact contributions that you make, and streamline the paper accordingly. These measures could strongly boost the impact of your work but will require a major revision.
PS: I think the confusion starts with the following sentence in the abstract: "In this work we set out to test several long-held hypothesis about neural network learning representations and numerical approaches to pruning." Both aspects are pretty orthogonal, but are completely mixed up in the paper.
The authors would like to thank the reviewer for taking time out to review the paper and for your kind comments. The review is definitely enlightening and while the authors agree to some of the concerns raised, they would like to respectfully disagree on some. The authors added a set of Frequently Asked Questions a few days ago, which answer most, if not all the concerns raised here.
>> My major criticism is that the paper lacks focus, does not have a concrete conclusion and does not explain what it adds to the literature.
We agree that we can do a better job telling a story, and will probably do so in the next version of this paper. The length of the paper, coupled with the fact that there are multiple key points the authors are trying to make might have come across as the paper lacking focus, and we apologize for that. The authors have been very clear about the focus of the paper, the gist of which is that we are not trying to propose a new method for pruning, but are trying to establish why existing methods are inherently flawed and how a look inside the learning representations can help us identify that.
Most pruning techniques implicitly make certain assumptions about learning representations which we show in our work to not necessarily be true, hence leading us to wondering if these new findings can be used to come up with better pruning results, the possibility of which is demonstrated by the Brute Force method. This issue has been addressed in the FAQ section.
As far as the Conclusion section goes, the authors agree that they can do a much better job of clearly explaining how this work contributes to the existing literature, the short version of which would be that this work offers a directly opposing approach to much of the long-established beliefs when it comes to pruning techniques. We will work on improving the Conclusion section.
>> But only two or three sentences in the conclusion, and no sentence in the part on results in the abstract, even refers to neural representations.
This is simply not true. The reviewer seems to have clearly missed something here. We have constantly referred back to how our results shed light on learning representations throughout the text, including in the Abstract where we mentioned that pruning can be done without affecting the learning representations, which is an assumption most techniques make but fail to come through on, as demonstrated by the 2nd order Taylor Series experiments.
>> What is the motivation to introduce a new 2nd order method here?
This has been answered in the FAQs. The 2nd order method plays an important role in pointing out flaws in existing techniques and hence is essential to be discussed at length. We are sorry that the reviewer missed the point and will take care to more explicitly state this in future versions.
>> But the brute-force pruning process is also serial - why is that not a problem?
The Brute Force method is only used to demonstrate that it is in fact, possible to prune networks without affecting the learning representations and that it is possible to achieve much better results than quadratic methods, which a lot of established techniques currently use.
>> [..] nor is there sufficient benchmarking against state-of-the-art pruning methods.
We have answered this concern in the FAQs already. All papers out there cannot be about outperforming existing algorithms, this one at least, is about pointing out serious flaws in them. We are not concerned about exactly by what amount these state-of-the-art methods are inefficient, we are interested in pointing out what inherent assumptions make them inefficient.
>> Then concentrate on 2-layer MNIST and a deeper CIFAR10 network.
This again has been answered in the FAQs. The size of the network is not important here due to reasons explained in the FAQs.
I did enjoy reading some of the introductions and background, in particular that of reminding readers of popular papers from the late 1980s and early 1990s. The idea of the proposal is straight forward: remove neurons based on the estimated change in the loss function from the packpropagation estimate with either first or second order backpropagation. The results are as expected that the first order method is worse then the second order method which in turn is worse than the brute force method.
However, there are many reasons why I think that this work is not appropriate for ICLR. For one, there is now a much stronger comprehension of weight decay algorithms and their relation to Bayesian priors which has not been mentioned at all. I would think that any work in this regime would require at least some comments about this.
Furthermore, there are many statements in the text that are not necessarily true, in particular in light of deep networks with modern regularization methods.
For example, the authors state that the most accurate method is what they call brute-force.
However, this assumes that the effects of each neurons are independent which might not be the case.
So the serial order of removal is not necessarily the best.
I also still think that this paper is unnecessarily long and the idea and the results could have been delivered in a much compressed way. I also don’t think just writing a Q&A section is not enough, and the points should be included in the paper.
The authors have put forward a sincere effort to investigate the "fundamental nature of learning representations in neural networks", a topic of great interest and importance to our field.  They propose to do this via a few simplistic pruning algorithms, to essentially monitor performance decay as a function of unit pruning.  This is an interesting idea and one that could potentially be instructive, though in total I don't think that has been achieved here.
First, I find the introduction of pruning lengthy and not particularly novel or surprising.
For example, Fig 1 is not necessary, nor is most of the preamble section 3.3.0.  The pruning algorithms themselves are sensible (though overly simplistic) approaches, which of course would not matter if they were effective in addressing the question.
However, in looking for contributions this paper makes, an interesting, pithy, or novel take on pruning is not one of them, in my opinion.
Second, and most relevant to my overall rating, Section 4 does not get deeper than scratching the surface.  The figures do not offer much beyond the expected decay in performance as a percentage of neurons removed or gain value.  The experiments themselves are not particularly deep, covering a toy problem and MNIST, which does not convince me that I can draw lessons to the broader story of neural networks more generally.
Third, there is no essential algorithmic, architectural, or mathematical insight, which I expect out of all but the most heavily experimental papers.

Meta-review:
role: PC
data: comment: The paper does not seem to have enough novelty, and the contribution is not clear enough due to presentation issues.

Decision: Reject