Title: A SIMPLE YET EFFECTIVE METHOD TO PRUNE DENSE LAYERS OF NEURAL NETWORKS

Abstract: Neural networks are usually over-parameterized with significant redundancy in the number of required neurons which results in unnecessary computation and memory usage at inference time. One common approach to address this issue is to prune these big networks by removing extra neurons and parameters while maintaining the accuracy. In this paper, we propose NoiseOut, a fully automated pruning algorithm based on the correlation between activations of neurons in the hidden layers. We prove that adding additional output neurons with entirely random targets results into a higher correlation between neurons which makes pruning by NoiseOut even more efficient. Finally, we test our method on various networks and datasets. These experiments exhibit high pruning rates while maintaining the accuracy of the original network.

1. INTRODUCTION:

Neural networks and deep learning recently achieved state-of-the-art solutions to many problems in computer vision (Krizhevsky et al. (2012) ; He et al. (2015) ), speech recognition (Graves et al. (2013) ), natural language processing (Mikolov et al. (2013) ) and reinforcement learning (Silver et al. (2016) ). Using large and oversized networks in these tasks is a common practice. Such oversized networks can easily overfit on the training dataset while having poor generalization on the testing data (Sabo & Yu (2008) ). A rule of thumb for obtaining useful generalization is to use the smallest number of parameters that can fit the training data (Reed (1993) ). Unfortunately, this optimal size is not usually obvious and therefore the size of the neural networks is determined by a few rules-of-thumb (Heaton (2008) ) which do not guarantee an optimal size for a given problem. One common approach to overcome overfitting is to choose an over-sized network and then apply regularization (Ng (2004) ) and Dropout (Srivastava et al. (2014) ). However, these techniques do not reduce the number of parameters and therefore do not resolve the high demand of resources at test time.

Another method is to start with an oversized network and then use pruning algorithms to remove redundant parameters while maintaining the network's accuracy (Augasta & Kathirvalavakumar (2013) ). These methods need to estimate the upper-bound size of a network, a task for which there are adequate estimation methods (Xing & Hu (2009) ). If the size of a neural network is bigger than what is necessary, in theory, it should be possible to remove some of the extra neurons without affecting its accuracy. To achieve this goal, the pruning algorithm should find neurons which once removed result into no additional prediction errors. However, this may not be as easy as it sounds since all the neurons contribute to the final prediction and removing them usually leads to error.

It is easy to demonstrate this problem by fitting an oversized network on a toy dataset. Figure 1 shows a two-dimensional toy dataset which contains two linearly separable classes. Hence, only one hidden neuron in a two-layer perceptron should be enough to classify this data and any network with more than one neuron (such as the network in Figure 2-a ) is an oversized network and can be pruned. However, there is no guarantee that removing one of the hidden neurons will maintain the network's performance. As shown in the example in Figure 1 , removing any of the hidden neurons results into a more compact, but under-performing network. Therefore, a more complicated process is required for pruning neural networks without accuracy loss. Our goal in this paper is two-fold. First, we introduce NoiseOut, a pruning method based on the correlation between activations of the neurons. Second, we propose an approach which enforces the higher correlation between activations of neurons. Since the effectiveness of NoiseOut hinges on high correlations between neuron activations, the combination of these two methods facilitates more aggressive pruning.

2. RELATED WORK:

Optimal Brain Damage (LeCun et al. (1989) ) and Optimal Brain Surgeon (Hassibi & Stork (1993) ) prune networks based on the Hessian of the loss function. It is shown that such pruning is more effective and more accurate than earlier magnitude-based pruning such as weight decay (Hanson & Pratt (1989) ). However, the necessary second-order derivatives require additional computational resources.

Recently, replacing the fully connected layers with other types of layers has been utilized to reduced the number of parameters in a neural network. Deep fried convnets (Yang et al. (2015) ) replaces these layers with kernel methods while the GoogLenet (Szegedy et al. (2015) ) and Network in Network architecture (Lin et al. (2013) ) replace them with global average pooling. Alternatively, Han et al. (2015) proposed a pruning method which learns the import connections between neurons, pruning the unimportant connections, and then retraining the remaining sparse network.

Besides pruning, other approaches have been proposed to reduce the computation and memory requirements of neural networks. HashNets ( Chen et al. (2015) ) reduce the storage requirement of neural networks by randomly grouping weights into hash buckets. These techniques can be combined with pruning algorithms to achieve even better performance. As an example, Deep Compression ( Han et al. (2016) ) proposed a three stage pipeline: pruning, trained quantization, and Huffman coding, to reduce the storage requirement of neural networks.

3. PROPOSED METHOD:

In this section, we describe the details of the proposed method called NoiseOut. First, we show how this method can prune a single neuron and then how it can prune a full network, one neuron at a time.

3.1. PRUNING A SINGLE NEURON:

The key idea in NoiseOut is to remove one of the two neurons with strongly correlated activations. The main rationale behind this pruning is to keep the signals inside the network as close to the original network as possible. To demonstrate this, assume there exists u, v, l such that |ρ(h

(l) u , h (l) v ) = 1 ⇒ h (l) u = αh (l) v + β, where h (l)

i is the activation of i th neuron in the l th layer. By definition:

h (l+1) k := i h (l) i w (l) i,k + b (l+1) k = i =u,v h (l) i w (l) i,k + h (l) u w (l) u,k + h (l) v w (l) v,k + b (l+1) k = i =u,v h (l) i w (l) i,k + αh (l) v w (l) u,k + βw (l) u,k + h (l) v w (l) v,k + b (l+1) k = i =u,v h (l) i w (l) i,k + h (l) v αw (l) u,k + w v,k + βw (l) u,k + b (l+1) k (1)

This means that neuron u can be removed without affecting any of the neurons in the next layer, simply by adjusting the weights of v and neurons' biases. Note that max i,j,i =j ρ(h

EQUATION

j ) = 1 is an ideal case. In this ideal scenario, removing one of the neurons results into no change in accuracy since the final output of the network will stay the same. In non-ideal cases, when the highest correlated neurons are not strongly correlated, merging them into one neuron may alter the accuracy. However, continuing the training after the merge may compensate for this loss. If this does not happen, it means that the removed neuron was necessary to achieve the target accuracy and the algorithm cannot compress the network any further without accuracy degradation.

NoiseOut follows the same logic to prune a single neuron using the following steps:

1. For each i, j, l calculate ρ h (l) i , h (l) j 2. Find u, v, l = arg max u,v,l,u =v ρ h (l) u , h (l) v 3. Calculate α, β := arg min α,β h (l) u -αh (l) v -β 4.

Remove neuron u in layer l 5. For each neuron k in layer l + 1:

-Update the weight w (l) v,k = w (l) v,k + αw (l) u,k -Update the bias b (l+1) k = b (l+1) k + βw (l) u 3.2 ENCOURAGING CORRELATION BETWEEN NEURONS

The key element for successful pruning of neural networks using NoiseOut is the strong correlation between activation of the neurons. Essentially, a higher correlation between these activations means more efficient pruning. However, there is no guarantee that back-propagation results in correlated activations in a hidden layer. In this section, we propose a method to encourage higher correlation by adding additional output nodes, called noise outputs. The targets for noise outputs will randomly change in each iteration based on a predefined random distribution. We show that adding noise outputs to the output layer intensifies the correlation between activation in the hidden layers which will subsequently make the pruning task more effective.

3.2.1. EFFECT OF ADDING NOISE OUTPUTS TO THE NETWORK:

To demonstrate the effect of adding noise outputs to the network, let us reconsider the toy example described previously in Figure 1 , this time with some additional noise outputs in the output layer as shown in Figure 2-b . The result of training this network has been shown in Figure 3 . As seen in this figure, the activation of the two hidden neurons has become highly correlated, and each hidden unit converged to the optimal discriminant by itself. This means that either of the extra neurons in the hidden layer can be removed without loss of accuracy. This claim can be proved formally as well. The key to this proof is that neural networks are deterministic at inference time. In other words, the network generates a constant output for a constant input. For noise output, since the target is independent of the input, the training algorithm finds an optimal constant value that minimizes the expected error for every input. This objective can be presented as:

EQUATION

where W is the adjustable parameters (i.e. weights), X is the input, Y is the target, Y i is the target of noisy outputs at iteration i, C is the cost function, m is the number of iterations. f i (X; W ) and fi (X; W ) are the outputs of the neural network in the original and noise outputs respectively, at iteration i. Note that Y i changes in each iteration according to a random distribution P Ŷ .

The first part of Equation 2 represents the common objective of training a neural network, while the second part has been added because of the noise outputs. It is possible to adjust the effect of noise outputs based on P Ŷ . For instance by adding more noise outputs (in the case of Binomial distribution) or adjusting the variance (for Gaussian distribution). Another way of this adjustment is introducing a new multiplier for the second part of the cost. Although Y i changes in each iteration, the constant value that the network infers for any given input would be the same e.g. θ, due to the independent of P Ŷ from X. Therefore: fm (X; W ) = J 1×n θ where J 1×n is the matrix of ones of size 1 × n (n is the number of samples in the dataset). The actual value of θ can be estimated for any given network architecture. As an example, let W (1) , W (2) and W (2) be the weights of the first hidden layer, the output layer and the noisy neurons in a 2-2-1 MLP network respectively (Figure 2-b ). Assuming linear activation in the hidden neurons and mean square error (MSE) as the cost function, Equation 2 can be rewritten as: Figure 4 : Effect of noise outputs to the correlation of neurons activation in the hidden layers. The top row shows the correlation of two hidden neurons in the network of Figure 2 while the bottom row is the correlation between the two neurons on the first hidden layer of a 6 layer MLP (2-2-2-2-2-2-1). The left column represents the mean correlation during training of the network for 100 times. In right column, yellow is the distribution of correlations at the end of the training and small red line shows the median. As it can be seen in these graphs, adding noise outputs improves the correlation between neurons in the hidden layer.

EQUATION

In this particular case, θ can be calculated using derivatives of the cost functions in Equation 3:

∂C ∂θ = ∂ ∂θ f (X; W ) -Y 2 + 1 2 f (X; W ) -θ 2 = ∂ ∂θ 1 2 f (X; W ) -θ) 2 = θ -f (X; W ) = 0 ⇒ θ = E f (X; W ) = E P Ŷ (4)

This means that in this particular network with MSE as the cost function, the final error will be minimized when the network outputs the expected value of targets in noise outputs E P Ŷ for any given input.

To demonstrate how outputting a constant value affects the weights of a network, let's consider the network in Figure 2-b . In this case, the output of noisy output will be:

f (X; W ) = XW (1) W (2) = h (1) 1 w (2) 1,1 + h (1) 2 w (2) 2,1 = θ ⇒ h (1) 1 = 1 w (2) 1,1 (θ -h (1) 2 w 2,1 ) ⇒ ρ h (1) 1 ,h (1) 2 = sgn -w (2) 2,1 w (2) 1,1 ⇒ ρ h (1) 1 ,h (1) 2 = 1 (5)

Equation 6 means that the activation of the hidden neurons has a correlation of 1 or -1. For more than two neurons it can be shown that the output of one neuron will be a linear combination of other neurons which means the claim still holds.

The same results can be achieved empirically. Since the output of the noise outputs will converge to E P Ŷ , it seems that there may not be any difference between different random distributions with the same expected value. Therefore, we tested different random distributions for E P Ŷ with the same E P Y N , on the network shown in Figure 2-b . These noise distributions are as follows:

Algorithm 1 NoiseOut for pruning hidden layers in neural networks 1: procedure TRAIN(X, Y ) X is input, Y is expected output 2:

W ← initialize_weights()

3:

for each iteration do 4:

Y N ← generate_random_noise() generate random expected values 5:

Y ← concatenate(Y, Y N ) 6: W ← back_prop(X, Y ) 7:

while cost(W ) ≤ threshold do 8:

A, B ← f ind_most_correlated_neurons(W, X)

9:

α, β ← estimate_parameters(W, X, A, B)

10:

W ← remove_neuron(W, A)

11:

W ← adjust_weights(W , B, α, β)

12:

W ← W 13:

return W -Gaussian P Ŷ (x) = N (0.1, 0.4) Normal distribution with mean of 0.1 and standard deviation of 0.4. This noise distribution is appropriate for regression tasks with MSE cost.

-Binomial P Ŷ (x) = B(1, 0.1) Binomial distribution with 1 trial and success probability of 0.1. We chose binomial distribution since it generates random classification labels and is appropriate for networks that have to produce binary labels.

-Constant P Ŷ (x) = δ(x -0.1) In this case, the target of the noise outputs is the constant value of 0.1. This is used as an expected-value "shortcut" so that we can examine a stochastic vs. a deterministic approach.

-No_Noise no noise output for comparison.

As it can be seen in the top row Figure 4 , in a regular network with no noise unit (shown as No_Noise), the correlation between the output of hidden neurons ρ h (1)

1 ,h

(1) 2

does not go higher than 0.8, while in the existence of a noise output this value approaches to one, rather quickly. This means that the two hidden neuron are outputting just a different scale of the same value for any given input. In this case, NoiseOut easily prunes one of the two neurons.

The same technique can be applied to correlate the activation of the hidden neurons in networks with more than two layers. The bottom row of Figure 4 shows the correlation between the activation of two hidden neurons in the first layer of a six layer MLP (2-2-2-2-2-2-1). As it can be seen in this figure, adding noise outputs helped the neurons to achieve higher correlation compared to a network with no noise output. Binomial noise acts chaotic at the beginning due to its sudden change of expected values in the noise outputs while Gaussian noise improved the correlation the best in these experiments.

3.3. PRUNING A FULLY CONNECTED NETWORK:

Algorithm 1 shows the final NoiseOut algorithm. For the sake of readability, this algorithm has been shown for networks with only one hidden layer. But the same algorithm can be applied to networks with more that one hidden layer by performing the same pruning on all the hidden layers independently. It can also be applied to convolutional neural networks that use dense layers, in which we often see over 90% of the network parameters (Cheng et al. (2015) ). for this criteria to match the original accuracy; therefore all the compressed network have the same accuracy as the original network. For each experiment, different random distributions have been used for P Ŷ , to demonstrate the difference in practice.

4.1. PRUNING FULLY CONNECTED AND CONVOLUTIONAL NETWORKS:

Table 1 and Table 2 show the result of pruning Lenet-300-100 and Lenet-5 (LeCun et al. (1998) ) on MNIST dataset. Lenet-300-100 is a fully connected network with two hidden layers, with 300 and 100 neurons each, while Lenet-5 is a convolutional network with two convolutional layers and one dense layer. These networks achieve 3.05% and 0.95% error rate on MNIST respectively (LeCun et al. (1998) ). Note that in Lenet-5 over 98% of parameters are in the dense layer and pruning them can decrease the model size significantly.

As it can be seen in these tables, NoiseOut removed over %95 of parameters with no accuracy degradation. Astonishingly, pruned Lenet-5 can achieve 0.95% error rate with only 3 neurons in the hidden layer which reduce the total number of weights in Lenet-5 by a factor of 44. Figure 6 demonstrates the output of these 3 neurons. This graph has been generated by outputting the activation of hidden layer neurons for 1000 examples randomly selected from MNIST dataset. Then, the data has been sorted by the target class. As it can be seen in this figure, the three neurons in the hidden layer efficiently encoded the output of convolutional layers to the expected ten classes. Obviously, these values can be utilized by the softmax layer to perform the final classification.

To test the effect of pruning of deeper architectures, we prune the network described in Table 4 on SVHN data set.This model which has over 1 million parameters, achieves 93.39% and 93.84% accuracy on training set and test set respectively. As it can be seen in Table 3 , NoiseOut pruned more than 85% of the parameters from the base model while maintaining the accuracy. x axis represents the total number of parameters in the pruned network (including weights in the convolutional layers), while y axis shows the accuracy of the model on test and training dataset.

4.2. EFFECT OF NOISEOUT ON TEST ACCURACY:

To explore the effect of NoiseOut on the test accuracy, we pruned Lenet-300-100 and Lenet-5 on MNIST with multiple accuracy thresholds, using Gaussian distribution as the target for noise outputs.

In each one of these experiments, we measured both the training and test accuracy. As expected, the results which have been shown in Figure 5 , indicate that lower accuracy thresholds result into more pruned parameters. However, the gap between training and testing threshold stays the same. This shows that pruning the network using NoiseOut does not lead to overfitting.

4.3. RELATION TO DROPOUT AND REGULARIZATION:

The key point in successful pruning with NoiseOut is a higher correlation between neurons. This goal might seem to be in contradiction with techniques designed to avoid overfitting such as Dropout and Regularization. To investigate this, we pruned Lenet-5 in the presence and absence of these features and demonstrated the results in Figure 7 . As it can be seen in this figure, Dropout helps the pruning process significantly, while L2-regularizations causes more variance. It seems that preventing co-adaptation of neurons using Dropout also intensifies the correlation between them, which helps NoiseOut to remove even more redundant neurons without accuracy loss. 

5. CONCLUSION:

In this paper, we have presented NoiseOut, a simple but effective pruning method to reduce the number of parameters in the dense layers of neural networks by removing neurons with correlated activation during training. We showed how adding noise outputs to the network could increase the correlation between neurons in the hidden layer and hence result to more efficient pruning. The experimental results on different networks and various datasets validate this approach, achieving significant compression rates without loss of accuracy.

:

This algorithm is simply repeating the process of removing a single neuron, as described in the previous section. The pruning ends when the accuracy of the network drops below some given threshold. Note that the pruning process is happening while training.4 EXPERIMENTSTo illustrate the generality of our method we test it on a core set of common network architectures, including fully connected networks and convolutional neural networks with dense layers. In all of these experiments, the only stop criteria is the accuracy decay of the model. We set the threshold

Figures and Tables:

Figure 1: Effect of pruning on accuracy. Bold line represents discriminator learned by a 2-2-1 MLP (Figure 2-a) on a toy data set. Dash line and dotted line show the results after pruning one of the hidden neurons. As it can be seen removing a hidden neuron will result into accuracy drop.( )

Figure 3: Comparison between discriminators learned with and without noise neurons. As it can be seen with noise neurons the activation of hidden neurons is more correlated; a) discriminators defined by each hidden neuron in Figure 2-a; b) discriminators defined by each hidden neuron in Figure 2-b; c) final discriminator after pruning one hidden neuron in Figure 2-b.

Figure5: Pruning Lenet-300-100 and Lenet-5 on MNIST data set with various accuracy thresholds.x axis represents the total number of parameters in the pruned network (including weights in the convolutional layers), while y axis shows the accuracy of the model on test and training dataset.

Figure 6: Activation of neurons in a pruned Lenet-5 with only 3 neurons left in the dense layer. The x-axis has been populated by 100 random samples from each class of MNIST, sorted by class. y-axis shows the neuron ID. Note that tanh is the activation function in the hidden layer.

Figure 7: Effect of Dropout and L2-regularization on NoiseOut. The y-axis represents the number of remaining neurons in the dense layer. Note that more than one neuron can be removed in each epoch. In each curve, the bold line is the median of 10 runs and colored background demonstrates the standard deviation.

Results of pruning Lenet-300-100 on MNIST. The accuracy of all the compressed networks are the same as the original network.

Table content: <table><tr><td>Method</td><td>Noise Outpus</td><td>Layer 1 Neurons</td><td>Layer 2 Neurons</td><td>Parameters</td><td>Removed Parameters</td><td>Compression Rate</td></tr><tr><td>Ground Truth</td><td>-</td><td>300</td><td>100</td><td>266610</td><td>-</td><td>-</td></tr><tr><td>No_Noise</td><td>-</td><td>23</td><td>14</td><td>15989</td><td>94.00%</td><td>16.67</td></tr><tr><td>Gaussian</td><td>512</td><td>20</td><td>9</td><td>15927</td><td>94.02%</td><td>16.73</td></tr><tr><td>Constant</td><td>512</td><td>20</td><td>7</td><td>15105</td><td>94.33%</td><td>17.65</td></tr><tr><td>Binomial</td><td>512</td><td>19</td><td>6</td><td>11225</td><td>95.78%</td><td>23.75</td></tr><tr><td>No_Noise</td><td>-</td><td>13</td><td>12</td><td>10503</td><td>96.06%</td><td>20.89</td></tr><tr><td>Gaussian</td><td>1024</td><td>16</td><td>7</td><td>12759</td><td>95.21%</td><td>18.58</td></tr><tr><td>Constant</td><td>1024</td><td>18</td><td>7</td><td>14343</td><td>94.62%</td><td>17.61</td></tr><tr><td>Binomial</td><td>1024</td><td>19</td><td>7</td><td>15135</td><td>94.32%</td><td>25.38</td></tr></table>

Pruning Lenet-5 on MNIST. The accuracy of all the compressed networks are the same as the original network.

Table content: <table><tr><td>Method</td><td>Noise Neurons</td><td>Dense Layer Neurons</td><td>Parameters</td><td>Removed Parameters</td><td>Compression Rate</td></tr><tr><td>Ground Truth</td><td>-</td><td>512</td><td>605546</td><td>-</td><td>-</td></tr><tr><td>No_Noise</td><td>-</td><td>313</td><td>374109</td><td>38.21%</td><td>1.61</td></tr><tr><td>Gaussian</td><td>512</td><td>3</td><td>13579</td><td>97.75%</td><td>44.59</td></tr><tr><td>Constant</td><td>512</td><td>33</td><td>48469</td><td>91.99%</td><td>12.49</td></tr><tr><td>Binomial</td><td>512</td><td>26</td><td>40328</td><td>93.34%</td><td>15.01</td></tr></table>

Pruning the reference network in Table4, on SVHN dataset.

Table content: <table><tr><td/><td/><td/><td/><td/><td/><td colspan="2">Table 4: Base model architecture for SVHN</td></tr><tr><td/><td/><td/><td/><td/><td/><td colspan="2">with 1236250 parameters.</td></tr><tr><td/><td/><td/><td/><td/><td/><td/><td>Layer</td><td>Filters Kernel Weights</td></tr><tr><td/><td/><td/><td/><td/><td/><td/><td>conv1</td><td>32</td><td>3 × 3</td><td>896</td></tr><tr><td/><td/><td/><td/><td/><td/><td/><td>conv2</td><td>32</td><td>3 × 3</td><td>9246</td></tr><tr><td colspan="2">Method</td><td colspan="2">Dense Layer Neurons</td><td colspan="3">Parameters Removed Parameters</td><td>conv3 pool1 conv4</td><td>32 -48</td><td>3 × 3 2 × 2 3 × 3 13872 9246 -</td></tr><tr><td colspan="2">Ground Truth No_Noise Gaussian Constant Bionomial</td><td colspan="2">1024 132 4 25 17</td><td colspan="2">1236250 313030 180550 202285 194005</td><td>-74.67% 85.39% 83.63% 84.30%</td><td>conv5 conv6 pool2 conv7 conv8 conv9</td><td>48 48 -64 64 64</td><td>3 × 3 20784 3 × 3 20784 2 × 2 -3 × 3 27712 3 × 3 36928 3 × 3 36928</td></tr><tr><td/><td/><td/><td/><td/><td/><td/><td>pool3</td><td>-</td><td>2 × 2</td><td>-</td></tr><tr><td/><td/><td/><td/><td/><td/><td/><td>dense</td><td>1024</td><td>-</td><td>1049600</td></tr><tr><td/><td/><td/><td/><td/><td/><td/><td>softmax 10</td><td>-</td><td>10250</td></tr><tr><td/><td colspan="6">Pruning results of LeNet-300-100 on MNIST</td><td>1.005</td><td>Pruning results of LeNet-5 on MNIST</td></tr><tr><td/><td>1.00</td><td/><td/><td/><td/><td/><td>1.000</td></tr><tr><td/><td/><td/><td/><td/><td/><td/><td>0.995</td></tr><tr><td/><td>0.98</td><td/><td/><td/><td/><td/></tr><tr><td/><td/><td/><td/><td/><td/><td/><td>0.990</td></tr><tr><td>Accuracy</td><td>0.96</td><td/><td/><td/><td/><td>Accuracy</td><td>0.985</td></tr><tr><td/><td/><td/><td/><td/><td/><td/><td>0.980</td></tr><tr><td/><td>0.94</td><td/><td/><td/><td/><td/></tr><tr><td/><td/><td/><td/><td/><td/><td/><td>0.975</td></tr><tr><td/><td>0.92</td><td/><td/><td/><td colspan="2">Training Testing</td><td>0.970</td><td>Training Testing</td></tr><tr><td/><td>10000</td><td>15000</td><td colspan="2">20000 Parameters</td><td>25000</td><td>30000</td><td>Parameters 12000 14000 16000 18000 20000 22000 24000 0.965</td></tr></table>

Peer review (iclr2017):
The paper proposes to prune a neural network by removing neurons whose operation is highly correlated with other neurons. The idea is nice and somewhat novel - most pruning methods concentrate on removal of individual weights, however I haven*t done a through research on this topic. However, the experimental and theoretical justification of this method need to be improved before publication: 1. Experiments. The authors do not report accuracy degradation while pruning in the tables, laconically stating that the networks did not degrade. This is not convincing. The only details are given in Figure 5, however this Figure disagrees with Table 2: in the Table, the number of parameters ranges from 40k-600k, while the Figure pictures the range 12k-24k. Unless more details are provided, simply claiming that a network can remove 50% neurons with no number on the degradation of accuracy is not convincing. 2. Theory. The proofs do not match the experimental conditions and make unreasonable assumptions. The proofs show that in the absence of biases a network with a constant output will have two correlated neurons that generate the output offset. However, this is exactly why networks have biases and doesn*t explain why noise injection helps (the proof suggests that all should be fine with deterministic auxiliary neuron). My interpretation is that the noisy output injects gradient noise (see e.g. the concurrent ICLR submission https://openreview.net/forum?id=rkjZ2Pcxe). As such the proof muddies the picture more than it helps in understanding what is happening. Verdict: Reject and resubmit. The pruning idea has potential, however its efficiency must be more soundly demonstrated (please provide network accuracies at various pruning levels, the method removes one neuron at a time, this allows making of nice plots) rather than laconically stating that a degradation on mnist from 97% accuracy to 92% is not significant (Figure 5.). Please provide Figures and Tables that agree with the text in terms of numbers provided.