## 6 Conclusions and Future Work  
In this work, we demonstrate that, when using data-driven measures of neuron importance, the rank orderings of the output channels of a given layer strongly depend on the sparsity level of the preceding layer. In Section 3, we show that as we increase the sparsity of the input activations for a given layer, the correlation trends toward zero when comparing: (1) the rank order of its output channels using input activations with no sparsity; versus (2) the rank order of its output channels when the input activations have sparsity \(s>0\). Informed by this observation, we propose a data-driven nonparametric, magnitude-based channel pruning algorithm that works in a greedy manner based on the activations of the previous sparsified layer, as detailed in Section 4. For the purpose of learning low-precision structured subnetworks, we demonstrate that our proposed algorithm works effectively in combination with statistics-based uniform quantization. As demonstrated in Section 5, our joint layerwise channel pruning and uniform quantization framework yields increased performance per memory footprint over existing solutions \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline
**Method** & **Network** & \(\mathbf{N_{W}}\) & \(\mathbf{N_{A}}\) & \(\mathbf{s_{W}}\) & \(\mathbf{s_{A}}\) & **Baseline Acc** & **Accuracy** & **Weights (Mb)** & **Activations (Mb)** & **PD (Acc/Mb)** \\ \hline \([\)55] & ResNet-32 & 8 & – & 77.8\% & – & 92.58 & 92.64 (+0.06) & 0.83 & 20.19 & 4.41 \\ \hline \([\)56] & DenseNet-76 & 2 & – & 54\% & – & 92.19 & 91.17 (\(-\)1.02) & 0.68 & 141.26 & 0.64 \\ \hline \multirow{6}{*}{\([\)38]} & VGG-19 & – & – & 95\% & – & 93.50 & 93.34 (\(-\)0.16) & 16.03 & 19.40 & 2.63 \\  & PreResNet-110 & – & – & 95\% & – & 95.04 & 92.35 (\(-\)2.69) & 1.38 & 67.50 & 1.34 \\  & DenseNet-100 & – & – & 95\% & – & 95.24 & 94.19 (\(-\)1.05) & 0.97 & 213.37 & 0.44 \\  & VGG-19 & – & – & 70\% & 70\% & 93.5 & 93.60 (\(+\)0.1) & 70.70 & 5.31 & 1.23 \\  & PreResNet-164 & – & – & 60\% & 60\% & 95.04 & 94.23 (\(-\)0.81) & 16.67 & 40.21 & 1.66 \\  & DenseNet-40 & – & – & 60\% & 60\% & 94.10 & 93.87 (\(-\)0.23) & 1.60 & 22.97 & 3.82 \\ \hline \multirow{6}{*}{\([\)57]} & DenseNet-40 & – & – & 60\% & 60\% & 94.11 & 93.16 (\(-\)0.95) & 1.60 & 22.97 & 3.79 \\  & ResNet-20 & – & – & 38\% & 38\% & 92.01 & 91.66 (\(-\)0.35) & 2.70 & 7.96 & 8.59 \\  & ResNet-56 & – & – & 45\% & 45\% & 93.04 & 92.26 (\(-\)0.78) & 7.53 & 19.18 & 3.45 \\  & ResNet-110 & – & – & 63\% & 63\% & 93.21 & 92.96 (\(-\)0.25) & 10.25 & 25.12 & 2.63 \\ \hline \([\)58] & VGG-16 & – & – & 78.8\% & 78.8\% & 93.40 & 91.50 (\(-\)1.90) & 49.96 & 3.75 & 1.70 \\ \hline \multirow{6}{*}{\([\)59]} & VGG16-C & – & – & 95\% & – & 93.51 & 93.00 (\(-\)0.51) & 11.78 & 17.70 & 3.15 \\  & WRN-22-8 & – & – & 95\% & – & 95.74 & 95.07 (\(-\)0.67) & 13.73 & 115.34 & 0.74 \\ \hline \([\)51] & ResNet-20 & 1.9 & – & 54\% & – & 91.29 & 91.15 (\(-\)0.14) & 0.24 & 12.85 & 6.97 \\ \hline \([\)12] & VGG-7 & 4.8 & 5.4 & – & – & 93.05 & 93.23 (\(+\)0.18) & 43.85 & 3.27 & 1.98 \\ \hline \([\)60] & MobileNet & 8 & 8 & – & – & 91.31 & 90.59 (\(-\)0.72) & 25.74 & 13.17 & 2.33 \\ \hline \([\)61] & ResNet-32 & 8 & – & 87.5\% & – & 92.58 & 92.57 (\(-\)0.01) & 0.47 & 20.19 & 4.48 \\ \hline \multirow{6}{*}{Ours} & \multirow{6}{*}{ResNet-32} & \multirow{6}{*}{8} & 8 & 25\% & 25\% & \multicolumn{2}{c}{92.53 –} & 2.80 & 7.57 & **8.92** \\  & & 8 & 8 & 40\% & 40\% & 92.58 & 91.77 (\(-\)0.81) & 2.24 & 6.06 & **11.06** \\ \cline{1-1}  & & 8 & 8 & 50\% & 50\% & 90.16 (\(-\)2.42) & 1.87 & 5.05 & **13.04** \\ \cline{1-1}  & & 4 & 4 & 50\% & 50\% & 87.30 (\(-\)5.28) & 0.93 & 2.52 & **25.25** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: We demonstrate the superior performance per memory footprint (i.e., performance density) of our framework when compared to existing image classification solutions trained on CIFAR10.