Paper:
Neural networks are usually over-parameterized with significant redundancy in the number of required neurons which results in unnecessary computation and memory usage at inference time. One common approach to address this issue is to prune these big networks by removing extra neurons and parameters while maintaining the accuracy. In this paper, we propose NoiseOut, a fully automated pruning algorithm based on the correlation between activations of neurons in the hidden layers. We prove that adding additional output neurons with entirely random targets results into a higher correlation between neurons which makes pruning by NoiseOut even more efficient. Finally, we test our method on various networks and datasets. These experiments exhibit high pruning rates while maintaining the accuracy of the original network.
Neural networks are usually over-parameterized with significant redundancy in the number of required neurons which results in unnecessary computation and memory usage at inference time. One common approach to address this issue is to prune these big networks by removing extra neurons and parameters while maintaining the accuracy. In this paper, we propose NoiseOut, a fully automated pruning algorithm based on the correlation between activations of neurons in the hidden layers. We prove that adding additional output neurons with entirely random targets results into a higher correlation between neurons which makes pruning by NoiseOut even more efficient. Finally, we test our method on various networks and datasets. These experiments exhibit high pruning rates while maintaining the accuracy of the original network.
1 INTRODUCTION
Neural networks and deep learning recently achieved state-of-the-art solutions to many problems in computer vision (Krizhevsky et al. (2012); He et al. (2015)), speech recognition (Graves et al. (2013)), natural language processing (Mikolov et al. (2013)) and reinforcement learning (Silver et al. (2016)). Using large and oversized networks in these tasks is a common practice. Such oversized networks can easily overfit on the training dataset while having poor generalization on the testing data (Sabo & Yu (2008)). A rule of thumb for obtaining useful generalization is to use the smallest number of parameters that can fit the training data (Reed (1993)). Unfortunately, this optimal size is not usually obvious and therefore the size of the neural networks is determined by a few rules-of-thumb (Heaton (2008)) which do not guarantee an optimal size for a given problem. One common approach to overcome overfitting is to choose an over-sized network and then apply regularization (Ng (2004)) and Dropout (Srivastava et al. (2014)). However, these techniques do not reduce the number of parameters and therefore do not resolve the high demand of resources at test time.
Another method is to start with an oversized network and then use pruning algorithms to remove redundant parameters while maintaining the network’s accuracy (Augasta & Kathirvalavakumar (2013)). These methods need to estimate the upper-bound size of a network, a task for which there are adequate estimation methods (Xing & Hu (2009)). If the size of a neural network is bigger than what is necessary, in theory, it should be possible to remove some of the extra neurons without affecting its accuracy. To achieve this goal, the pruning algorithm should find neurons which once removed result into no additional prediction errors. However, this may not be as easy as it sounds since all the neurons contribute to the final prediction and removing them usually leads to error.
It is easy to demonstrate this problem by fitting an oversized network on a toy dataset. Figure 1 shows a two-dimensional toy dataset which contains two linearly separable classes. Hence, only one hidden neuron in a two-layer perceptron should be enough to classify this data and any network with more than one neuron (such as the network in Figure 2-a) is an oversized network and can be pruned. However, there is no guarantee that removing one of the hidden neurons will maintain the network’s performance. As shown in the example in Figure 1, removing any of the hidden neurons results into a more compact, but under-performing network. Therefore, a more complicated process is required for pruning neural networks without accuracy loss.
Our goal in this paper is two-fold. First, we introduce NoiseOut, a pruning method based on the correlation between activations of the neurons. Second, we propose an approach which enforces the higher correlation between activations of neurons. Since the effectiveness of NoiseOut hinges on high correlations between neuron activations, the combination of these two methods facilitates more aggressive pruning.
2 RELATED WORK
Optimal Brain Damage (LeCun et al. (1989)) and Optimal Brain Surgeon (Hassibi & Stork (1993)) prune networks based on the Hessian of the loss function. It is shown that such pruning is more effective and more accurate than earlier magnitude-based pruning such as weight decay (Hanson & Pratt (1989)). However, the necessary second-order derivatives require additional computational resources.
Recently, replacing the fully connected layers with other types of layers has been utilized to reduced the number of parameters in a neural network. Deep fried convnets (Yang et al. (2015)) replaces these layers with kernel methods while the GoogLenet (Szegedy et al. (2015)) and Network in Network architecture (Lin et al. (2013)) replace them with global average pooling. Alternatively, Han et al. (2015) proposed a pruning method which learns the import connections between neurons, pruning the unimportant connections, and then retraining the remaining sparse network.
Besides pruning, other approaches have been proposed to reduce the computation and memory requirements of neural networks. HashNets( Chen et al. (2015)) reduce the storage requirement of neural networks by randomly grouping weights into hash buckets. These techniques can be combined with pruning algorithms to achieve even better performance. As an example, Deep Compression( Han et al. (2016)) proposed a three stage pipeline: pruning, trained quantization, and Huffman coding, to reduce the storage requirement of neural networks.
3 PROPOSED METHOD
In this section, we describe the details of the proposed method called NoiseOut. First, we show how this method can prune a single neuron and then how it can prune a full network, one neuron at a time.
3.1 PRUNING A SINGLE NEURON
The key idea in NoiseOut is to remove one of the two neurons with strongly correlated activations. The main rationale behind this pruning is to keep the signals inside the network as close to the original network as possible. To demonstrate this, assume there exists u, v, l such that |ρ(h(l)u , h(l)v ) ∣∣ = 1⇒ h (l) u = αh (l) v + β, where h (l) i is the activation of i th neuron in the lth layer. By definition:
h (l+1) k := ∑ i h (l) i w (l) i,k + b (l+1) k = ∑ i6=u,v h (l) i w (l) i,k + h (l) u w (l) u,k + h (l) v w (l) v,k + b (l+1) k
= ∑ i 6=u,v h (l) i w (l) i,k + αh (l) v w (l) u,k + βw (l) u,k + h (l) v w (l) v,k + b (l+1) k
= ∑ i 6=u,v h (l) i w (l) i,k + h (l) v ( αw (l) u,k + wv,k ) + ( βw (l) u,k + b (l+1) k ) (1)
This means that neuron u can be removed without affecting any of the neurons in the next layer, simply by adjusting the weights of v and neurons’ biases. Note that max
i,j,i 6=j ∣∣ρ(h(l)i , h(l)j )∣∣ = 1 is an ideal case. In this ideal scenario, removing one of the neurons results into no change in accuracy since the final output of the network will stay the same. In non-ideal cases, when the highest correlated neurons are not strongly correlated, merging them into one neuron may alter the accuracy. However, continuing the training after the merge may compensate for this loss. If this does not happen, it means that the removed neuron was necessary to achieve the target accuracy and the algorithm cannot compress the network any further without accuracy degradation.
NoiseOut follows the same logic to prune a single neuron using the following steps:
1. For each i, j, l calculate ρ ( h (l) i , h (l) j ) 2. Find u, v, l = argmax
u,v,l,u6=v ∣∣ρ(h(l)u , h(l)v )∣∣ 3. Calculate α, β := argmin
α,β
( h (l) u − αh(l)v − β ) 4. Remove neuron u in layer l 5. For each neuron k in layer l + 1:
- Update the weight w(l)v,k = w (l) v,k + αw (l) u,k
- Update the bias b(l+1)k = b (l+1) k + βw (l) u
3.2 ENCOURAGING CORRELATION BETWEEN NEURONS
The key element for successful pruning of neural networks using NoiseOut is the strong correlation between activation of the neurons. Essentially, a higher correlation between these activations means more efficient pruning. However, there is no guarantee that back-propagation results in correlated activations in a hidden layer. In this section, we propose a method to encourage higher correlation by adding additional output nodes, called noise outputs. The targets for noise outputs will randomly change in each iteration based on a predefined random distribution. We show that adding noise outputs to the output layer intensifies the correlation between activation in the hidden layers which will subsequently make the pruning task more effective.
3.2.1 EFFECT OF ADDING NOISE OUTPUTS TO THE NETWORK
To demonstrate the effect of adding noise outputs to the network, let us reconsider the toy example described previously in Figure 1, this time with some additional noise outputs in the output layer as shown in Figure 2-b. The result of training this network has been shown in Figure 3. As seen in this figure, the activation of the two hidden neurons has become highly correlated, and each hidden unit converged to the optimal discriminant by itself. This means that either of the extra neurons in the hidden layer can be removed without loss of accuracy.
This claim can be proved formally as well. The key to this proof is that neural networks are deterministic at inference time. In other words, the network generates a constant output for a constant input. For noise output, since the target is independent of the input, the training algorithm finds an optimal constant value that minimizes the expected error for every input. This objective can be presented as:
min W
C ( f(X;W ), [ Y, Ŷ ]) = min
W
( C ( fm(X;W ), Y ) + m∑ i=0 C ( f̂i(X;W ), Ŷi )) (2)
where W is the adjustable parameters (i.e. weights), X is the input, Y is the target, Ŷi is the target of noisy outputs at iteration i, C is the cost function, m is the number of iterations. fi(X;W ) and f̂i(X;W ) are the outputs of the neural network in the original and noise outputs respectively, at iteration i. Note that Ŷi changes in each iteration according to a random distribution PŶ .
The first part of Equation 2 represents the common objective of training a neural network, while the second part has been added because of the noise outputs. It is possible to adjust the effect of noise outputs based on PŶ . For instance by adding more noise outputs (in the case of Binomial distribution) or adjusting the variance (for Gaussian distribution). Another way of this adjustment is introducing a new multiplier for the second part of the cost. Although Ŷi changes in each iteration, the constant value that the network infers for any given input would be the same e.g. θ, due to the independent of PŶ from X . Therefore:
f̂m(X;W ) = J1×nθ
where J1×n is the matrix of ones of size 1×n (n is the number of samples in the dataset). The actual value of θ can be estimated for any given network architecture. As an example, let W (1), W (2) and Ŵ (2) be the weights of the first hidden layer, the output layer and the noisy neurons in a 2-2-1 MLP network respectively (Figure 2-b). Assuming linear activation in the hidden neurons and mean square error (MSE) as the cost function, Equation 2 can be rewritten as:
min W C = min W
C ( f(X;W ), [ Y, Ŷi ]) = min W (1),W (2) 1 2 (f(X;W )− Y )2 + min
W (1),Ŵ (2) m∑ i=0 1 2 ( f̂i(X;W )− Ŷi )2 = min W (1),W (2) ( f(X;W )− Y )2 + min W (1),Ŵ (2) 1 2 ( f̂(X;W )− θ )2 (3)
In this particular case, θ can be calculated using derivatives of the cost functions in Equation 3:
∂C ∂θ = ∂ ∂θ
[( f(X;W )− Y )2 + 1
2
( f̂(X;W )− θ )2] = ∂
∂θ [1 2 ( f̂(X;W )− θ)2 ] = ( θ − f̂(X;W ) ) = 0
⇒ θ = E [ f̂(X;W ) ] = E [ PŶ ]
(4)
This means that in this particular network with MSE as the cost function, the final error will be minimized when the network outputs the expected value of targets in noise outputs ( E [ PŶ ])
for any given input.
To demonstrate how outputting a constant value affects the weights of a network, let’s consider the network in Figure 2-b. In this case, the output of noisy output will be:
f̂(X;W ) = XW (1)Ŵ (2) = h (1) 1 w (2) 1,1 + h (1) 2 w (2) 2,1 = θ
⇒ h(1)1 = 1
w (2) 1,1
(θ − h(1)2 w2,1)
⇒ ρ h (1) 1 ,h (1) 2 = sgn (−w(2)2,1 w
(2) 1,1
) ⇒ ∣∣ρ h (1) 1 ,h (1) 2 ∣∣ = 1 (5) Equation 6 means that the activation of the hidden neurons has a correlation of 1 or -1. For more than two neurons it can be shown that the output of one neuron will be a linear combination of other neurons which means the claim still holds. The same results can be achieved empirically. Since the output of the noise outputs will converge to E [ PŶ ] , it seems that there may not be any difference between different random distributions with
the same expected value. Therefore, we tested different random distributions for E [ PŶ ]
with the same E [ PYN ] , on the network shown in Figure 2-b. These noise distributions are as follows:
Algorithm 1 NoiseOut for pruning hidden layers in neural networks 1: procedure TRAIN(X,Y ) . X is input, Y is expected output 2: W ← initialize_weights() 3: for each iteration do 4: YN ← generate_random_noise() . generate random expected values 5: Y ′ ← concatenate(Y, YN ) 6: W ← back_prop(X,Y ′) 7: while cost(W ) ≤ threshold do 8: A,B ← find_most_correlated_neurons(W,X) 9: α, β ← estimate_parameters(W,X,A,B) 10: W ′ ← remove_neuron(W,A) 11: W ′ ← adjust_weights(W ′, B, α, β) 12: W ←W ′ 13: return W
- Gaussian PŶ (x) = N (0.1, 0.4) Normal distribution with mean of 0.1 and standard deviation of 0.4. This noise distribution is appropriate for regression tasks with MSE cost.
- Binomial PŶ (x) = B(1, 0.1) Binomial distribution with 1 trial and success probability of 0.1. We chose binomial distribution since it generates random classification labels and is appropriate for networks that have to produce binary labels.
- Constant PŶ (x) = δ(x− 0.1) In this case, the target of the noise outputs is the constant value of 0.1. This is used as an expected-value “shortcut" so that we can examine a stochastic vs. a deterministic approach.
- No_Noise no noise output for comparison.
As it can be seen in the top row Figure 4, in a regular network with no noise unit (shown as No_Noise), the correlation between the output of hidden neurons ∣∣ρ h (1) 1 ,h (1) 2 ∣∣ does not go higher than 0.8, while in the existence of a noise output this value approaches to one, rather quickly. This means that the two hidden neuron are outputting just a different scale of the same value for any given input. In this case, NoiseOut easily prunes one of the two neurons.
The same technique can be applied to correlate the activation of the hidden neurons in networks with more than two layers. The bottom row of Figure 4 shows the correlation between the activation of two hidden neurons in the first layer of a six layer MLP (2-2-2-2-2-2-1). As it can be seen in this figure, adding noise outputs helped the neurons to achieve higher correlation compared to a network with no noise output. Binomial noise acts chaotic at the beginning due to its sudden change of expected values in the noise outputs while Gaussian noise improved the correlation the best in these experiments.
3.3 PRUNING A FULLY CONNECTED NETWORK
Algorithm 1 shows the final NoiseOut algorithm. For the sake of readability, this algorithm has been shown for networks with only one hidden layer. But the same algorithm can be applied to networks with more that one hidden layer by performing the same pruning on all the hidden layers independently. It can also be applied to convolutional neural networks that use dense layers, in which we often see over 90% of the network parameters (Cheng et al. (2015)).
This algorithm is simply repeating the process of removing a single neuron, as described in the previous section. The pruning ends when the accuracy of the network drops below some given threshold. Note that the pruning process is happening while training.
4 EXPERIMENTS
To illustrate the generality of our method we test it on a core set of common network architectures, including fully connected networks and convolutional neural networks with dense layers. In all of these experiments, the only stop criteria is the accuracy decay of the model. We set the threshold
for this criteria to match the original accuracy; therefore all the compressed network have the same accuracy as the original network. For each experiment, different random distributions have been used for PŶ , to demonstrate the difference in practice.
4.1 PRUNING FULLY CONNECTED AND CONVOLUTIONAL NETWORKS
Table 1 and Table 2 show the result of pruning Lenet-300-100 and Lenet-5 (LeCun et al. (1998)) on MNIST dataset. Lenet-300-100 is a fully connected network with two hidden layers, with 300 and 100 neurons each, while Lenet-5 is a convolutional network with two convolutional layers and one dense layer. These networks achieve 3.05% and 0.95% error rate on MNIST respectively (LeCun et al. (1998)). Note that in Lenet-5 over 98% of parameters are in the dense layer and pruning them can decrease the model size significantly.
As it can be seen in these tables, NoiseOut removed over %95 of parameters with no accuracy degradation. Astonishingly, pruned Lenet-5 can achieve 0.95% error rate with only 3 neurons in the hidden layer which reduce the total number of weights in Lenet-5 by a factor of 44. Figure 6 demonstrates the output of these 3 neurons. This graph has been generated by outputting the activation of hidden layer neurons for 1000 examples randomly selected from MNIST dataset. Then, the data has been sorted by the target class. As it can be seen in this figure, the three neurons in the hidden layer efficiently encoded the output of convolutional layers to the expected ten classes. Obviously, these values can be utilized by the softmax layer to perform the final classification.
To test the effect of pruning of deeper architectures, we prune the network described in Table 4 on SVHN data set.This model which has over 1 million parameters, achieves 93.39% and 93.84% accuracy on training set and test set respectively. As it can be seen in Table 3, NoiseOut pruned more than 85% of the parameters from the base model while maintaining the accuracy.
Table 3: Pruning the reference network in Table 4, on SVHN dataset.
Method Dense Layer
Neurons Parameters Removed Parameters
Ground Truth 1024 1236250 -
No_Noise 132 313030 74.67% Gaussian 4 180550 85.39% Constant 25 202285 83.63% Bionomial 17 194005 84.30%
Table 4: Base model architecture for SVHN with 1236250 parameters.
Layer Filters Kernel Weights conv1 32 3× 3 896 conv2 32 3× 3 9246 conv3 32 3× 3 9246 pool1 - 2× 2 - conv4 48 3× 3 13872 conv5 48 3× 3 20784 conv6 48 3× 3 20784 pool2 - 2× 2 - conv7 64 3× 3 27712 conv8 64 3× 3 36928 conv9 64 3× 3 36928 pool3 - 2× 2 - dense 1024 - 1049600 softmax 10 - 10250
4.2 EFFECT OF NOISEOUT ON TEST ACCURACY
To explore the effect of NoiseOut on the test accuracy, we pruned Lenet-300-100 and Lenet-5 on MNIST with multiple accuracy thresholds, using Gaussian distribution as the target for noise outputs. In each one of these experiments, we measured both the training and test accuracy. As expected, the results which have been shown in Figure 5, indicate that lower accuracy thresholds result into more pruned parameters. However, the gap between training and testing threshold stays the same. This shows that pruning the network using NoiseOut does not lead to overfitting.
4.3 RELATION TO DROPOUT AND REGULARIZATION
The key point in successful pruning with NoiseOut is a higher correlation between neurons. This goal might seem to be in contradiction with techniques designed to avoid overfitting such as Dropout and Regularization. To investigate this, we pruned Lenet-5 in the presence and absence of these features and demonstrated the results in Figure 7. As it can be seen in this figure, Dropout helps the pruning process significantly, while L2-regularizations causes more variance. It seems that preventing co-adaptation of neurons using Dropout also intensifies the correlation between them, which helps NoiseOut to remove even more redundant neurons without accuracy loss.
5 CONCLUSION
In this paper, we have presented NoiseOut, a simple but effective pruning method to reduce the number of parameters in the dense layers of neural networks by removing neurons with correlated activation during training. We showed how adding noise outputs to the network could increase the correlation between neurons in the hidden layer and hence result to more efficient pruning. The experimental results on different networks and various datasets validate this approach, achieving significant compression rates without loss of accuracy.

Review:
This paper proposes and tests two ideas. (1) a method of pruning networks by identifying highly correlated neuron pairs, pruning one of the pair, and then modifying downstream weights to compensate for the removal (which works well if the removed neurons were highly correlated). (2) a method, dubbed NoiseOut, for increasing neuron correlation by adding auxiliary noise target outputs to the network during training.
The first idea (1) is fairly straightforward, and it is not clear if it has been tried before. It does seem to work.
The second idea (2) is of unclear value and seems to this reviewer that it may merely add a regularizing effect. Comments in this direction:
 - In Fig 4 (right), the constant and Gaussian treatments seem to produce the same effect in both networks, right? And the Binomial effect seems the same as No_Noise. If this is true, can we conclude that the NoiseOut targets are simply serving to regularize the network, that is, to reduce its capacity slightly?
 - To show whether this effect is true, one would need to compare to other methods of reducing the network capacity, for example: by reducing the number of neurons, by applying L2 regularization of various values, or by applying Dropout of various strengths. Fig 7 makes an attempt at this direction, but critically misses several comparison treatments: “Pruned without any regularization”, “Pruned with only L2”, and “Pruned with only DropOut”. Have these experiments been run? Can their results be included and used to produce plots like Fig 5 and Fig 7?
Without these comparisons, it seems impossible to conclude that NoiseOut does anything but provide similar regularization to DropOut or L2.
The combined ideas (1) + (2) DO produce a considerable reduction in parameters, but sadly the experiments and exposition are somewhat too lacking to really understand what is going on. With a little more work the paper could be quite interesting, but as is it should probably not be accepted.
Additional comments:
 - Section 4 states: “In all of these experiments, the only stop criteria is the accuracy decay of the model. We set the threshold for this criteria to match the original accuracy; therefore all the compressed network have the same accuracy as the original network.” Is this accuracy the train accuracy or test accuracy? If train, then test accuracy needs to be shown (how much test performance is lost when pruning?). If test, then this would typically be referred to as “cheating” and so the choice needs to be very clearly stated and then defended.
 - Lowercase rho is used to indicate correlation but this is never actually specified, which is confusing for. Just state once that it indicates correlation.
 - How do these results compare to other pruning methods? No numerical comparison is attempted.