## 5 Experiments
### Evaluating Pruning Schedules and Importance Measures
As such, this class of algorithms uses rankings \(\mathbf{r}_{\mathbf{h}_{i}|s_{t}}\) to heuristically guide pruning. Furthermore, one-shot pruning schedules approximate the neuron importance with respect to the target sparsity \(s_{T}\) using estimates at step 0, when the input activations have no sparsity. As such, this class of algorithms uses rankings \(\mathbf{r}_{\mathbf{h}_{i}}\) to heuristically guide pruning. In contrast, our layerwise pruning schedule directly uses the neuron importance measure with respect to the target sparsity \(s_{T}\), where we use \(\mathbf{r}_{\mathbf{h}_{i}|s_{T}}\) to heuristically guide pruning. As the sparsity target \(s_{T}\) increases to more extreme levels, there is a gradual decorrelation between \(\mathbf{r}_{\mathbf{h}_{i}}\) and \(\mathbf{r}_{\mathbf{h}_{i}|s_{T}}\), as shown in Section 3. Our results given in Table 1 show that directly using \(\mathbf{r}_{\mathbf{h}_{i}|s_{T}}\) within our greedy layerwise framework increases the effectiveness of data-driven rankings. We conjecture this is because our greedy layerwise channel pruning algorithm allows an unpruned layer to freely adjust to abrupt shifts in its input activation distribution caused by pruning the channels of its preceding layer. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & & \multicolumn{2}{c}{**Classification**} & \multirow{2}{*}{**Segmentation (mIOU)**} & \multirow{2}{*}{
\begin{tabular}{c} **Style Transfer** \\ **(FID)** \\ \end{tabular} } \\ \cline{3-3} \cline{5-6}  & & **DenseNet121** & & & **MobileNetV2** & & **UNet** & **FRNet** & **CycleGAN** \\ \hline Baseline & Full Model & 78.70 & 68.31 & 61.69 & 66.32 & 47.17 \\ \hline \multirow{3}{*}{50\% Sparsity} & From Scratch & 76.75 & 65.36 & 58.90 & 61.84 & 50.39 \\  & One-Shot & 78.40 & 67.39 & **60.59** & 64.53 & 49.43 \\  & Stepwise & 77.14 & **68.05** & 58.13 & 64.90 & 52.83 \\  & Layerwise (Ours) & **78.77** & 67.58 & 60.57 & **65.52** & **48.06** \\ \hline \multirow{3}{*}{75\% Sparsity} & From Scratch & 73.07 & 56.66 & **56.71** & 59.29 & 59.55 \\  & One-Shot & 76.40 & 56.16 & 52.77 & 58.71 & 57.07 \\ \cline{1-1}  & Stepwise & 72.55 & 60.40 & 51.05 & 56.59 & 65.67 \\ \cline{1-1}  & Layerwise (Ours) & **76.44** & **60.71** & 56.48 & **61.14** & **55.48** \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Comparing pruning schedules across discriminative and generative tasks. With higher levels of sparsity, our greedy layerwise channel pruning algorithm performs better than existing baselines across both discriminative and generative tasks. Note that for top-1 accuracy and mIOU, higher is better, but for FID, lower is better. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & & \multicolumn{2}{c}{**Classification**} & \multirow{2}{*}{**Segmentation (mIOU)**} & \multirow{2}{*}{
\begin{tabular}{c} **Style Transfer** \\ **(FID)** \\ \end{tabular} } \\ \cline{3-3} \cline{5-6}  & & **DenseNet121** & & & **MobileNetV2** & **UNet** & **FRNet** & **CycleGAN** \\ \hline Baseline & Full Model & 78.70 & 68.31 & 61.69 & 66.32 & 47.17 \\ \hline \multirow{3}{*}{50\% Sparsity} & Layerwise (A) & **78.77** & **67.58** & **60.57** & 65.52 & **48.06** \\  & Layerwise (W) & 77.88 & 67.56 & 59.41 & **66.16** & 52.52 \\ \hline \multirow{3}{*}{75\% Sparsity} & Layerwise (A) & **76.44** & **60.71** & **56.48** & **61.14** & **55.48** \\  & Layerwise (W) & 73.53 & 60.62 & 54.38 & 60.18 & 67.97 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparing data-free vs. data-driven neuron importance measures across discriminative and generative tasks using our greedy layerwise channel pruning algorithm, we observe that the mean \(\ell_{1}\)-norm of the output activations (i.e., “Layerwise (A)”) performs better than the \(\ell_{1}\)-norm of channel weights (i.e., “Layerwise (W)”).