IPPRO: Importance-based Pruning with PRojective Offset for Magnitude-indifferent Structural Pruning

## Abstract

Abstract With the growth of demand on neural network compression methods, the structured pruning methods including importance-based approach are actively studied. The magnitude importance and many correlated modern importance criteria often limit the capacity of pruning decision, since the filters with larger magnitudes are not likely to be pruned if the smaller one didn’t, even if it is redundant. In this paper, we propose a novel pruning strategy to challenge this dominating effect of magnitude and provide fair chance to each filter to be pruned, by placing it on projective space. After that, we observe the gradient descent movement whether the filters move toward the origin or not, to measure how the filter is likely to be pruned. This measurement is used to construct PROscore, a novel importance score for IPPRO, a novel importance-based structured pruning with magnitude-indifference. Our evaluation results shows that the proposed importance criteria using the projective space achieves near-lossless pruning by reducing the performance drop in pruning, with promising performance after the finetuning. Our work debunks the “size-matters” myth in pruning and expands the frontier of importance-based pruning both theoretically and empirically.

## 1 Introduction

Deep learning has achieved remarkable success in various application domains at the cost of ever-increasing computation and memory requirements.
Overparametrization of neural networks is beneficial for convergence and generalization power of deep learning, but burdens resource-constrained platforms such as smartphones and edge devices.
Motivated by this need for model compression, neural network pruning became a quintessential tool to curtail the computation and memory requirement of deep learning models deng2020model.
In particular, structured pruning he2023structured emerged as a key pruning technique, as its explicit channel or filter-based pruning aligns well with modern computing architecture optimization and translates to significantly reduced memory footprint and FLOPs.

Regularization-based structured pruning uses regularizers such as Lasso liu2017slim; li2017pruning or Group Lasso wen2016ssl; liu2015sparse to decide filters to prune.
Importance-based structured pruning ranks and prunes the filters with least importance scores.
Although distinct, both approaches are closely related; regularization and importance measures often rely on each other for smarter pruning.
For example, a common theme for both regularization-based and importance-based structural pruning is the magnitude of a filter, typically measured as the norm of each channel’s weight tensor luo2017thinet.

Figure: Figure 1: IPPRO (Importance-based Pruning with PRojective Offset) outperforms state-of-the-art pruning methods and expands the Pareto frontier on Cityscape dataset using Deeplabv3.
Refer to caption: x1.png

Despite its intuitiveness, pruning based on filter magnitudes can be deceiving He_2019_CVPR; liu2018rethinking: imagine scaling a layer up and inversely scaling down the next layer – this procedure can arbitrarily perturb the filter magnitudes while preserving the model output.
More refined structured pruning methods incorporate other criteria, such as gradient information via Taylor expansion molchanov2019importance or geometric similarity of filter clusters he2019filter.
However, even these efforts have not yet disentangled magnitude-norm based artifacts from their filter score metric of functional importance in structured pruning.

We address this limitation by proposing IPPRO, a novel magnitude-invariant importance-based structured pruning algorithm.
Our key contribution is the construction of PROscore, a novel importance score with magnitude invariance established from projective geometry.
To gain scale invariance, IPPRO maps filters to real projective space by augmenting each filter with its vector norm, and then computes PROscore from the filters’ dynamics under gradient flow, which completely decouples the filter weight vector from its magnitude.

IPPRO is conceptually simple, straightforward to implement, model architecture agnostic.
As CNN-based vision models are the workhorses for on-device low-resource inference tasks, we validate IPPRO’s practical effectiveness on diverse vision benchmarks with its consistent performance even in high-pruning regimes and no fine-tuning scenarios.
We formally define the theoretical foundations of PROscore in [Section 3](https://arxiv.org/html/2507.14171v2#S3), describe implementation of IPPRO in [Section 4](https://arxiv.org/html/2507.14171v2#S4), and present its comprehensive empirical evaluation in [Section 5](https://arxiv.org/html/2507.14171v2#S5).

## 2 Related Works

### 2.1 Structured Pruning using Magnitudes

Deep Neural Networks (DNNs) are often overparameterized, leading to unnecessary computational costs. Pruning addresses this by removing unimportant weights or structures. Early methods like magnitude-based pruning han2015deep eliminate weights with small absolute values, achieving model compression with minimal accuracy loss. However, such approaches may overlook a weight’s actual impact on performance.

To improve pruning effectiveness, gradient-based methods blalock2020state; molchanov2019importance consider how changes in weights affect the loss function, enabling more informed decisions. Yet, determining optimal pruning ratios per layer is still challenging, as uniform pruning often leads to suboptimal results. Studies luo2017thinet; liu2017learning show that layers vary in pruning sensitivity, motivating adaptive approaches. Ye et al. ye2023performance proposed the Sequential Greedy Pruning Strategy (SGPS), which iteratively removes weights based on layer-wise importance, achieving more balanced and performance-aware pruning.

### 2.2 Importance-based Pruning

Importance-based pruning evaluates filter contribution to remove redundancy in CNNs. Norm-based methods like $L_{1}$-norm filters2016pruning and $L_{2}$-norm he2018soft prune low-magnitude filters, while others use filter similarity. For example, he2019filter uses the Geometric Median to remove closely clustered filters, and yvinec2021red; yvinec2022red++ apply Scalar Hashing and Input-wise Splitting to detect and prune similar filters based on input relevance.

Statistical and structural approaches further enhance pruning. wang2019cop uses Pearson Correlation to remove highly similar filters and applies Layer-wise Max-Normalization for cross-layer comparison. wang2021convolutional treats filters as graph nodes to detect redundancy via structural properties. Recently, 10484493 introduced a torque-inspired method that weights filters by distance from a pivot, capturing spatial structure and offering a simple yet effective pruning strategy.

Figure: (a) ResNet50 - layer2.1.conv2
Refer to caption: x2.png

## 3 Methods

Let $C$ be number of prunable channels in a neural network model, and $F_{1},\cdots,F_{C}\in\mathbb{R}^{N}$ be the filters which correspond to the channels.
Each filter $F_{i}$ is a vector, which has two distinct types of geometric information: magnitude and direction.
We aim to design a novel pruning methodology that challenges the “size matters” myth in pruning, where ‘size’ is the magnitude of a prunable filter, as this myth limits the potential of magnitude-based pruning.

Figure: (a) Visualization of $B_{\pi/4}(0)$ and $embed(F)$ in $\mathbb{R}^{N+1}$
Refer to caption: x5.png

In [Section 3.1](https://arxiv.org/html/2507.14171v2#S3.SS1), we introduce the concept of projective geometry, and define the mapping function $embed$ which maps the filters to the projective space $\mathbb{RP^{N}}$.
After placing the filters to $\mathbb{RP}^{N}$, in [Section 3.2](https://arxiv.org/html/2507.14171v2#S3.SS2) we explain the distance in projective space given by an angular distance, and define the proposed importance using the angular distance.

### 3.1 Embedding Filters into Projective Space

To extract the directional information of a filter, which can complement its magnitude, we place the filter on a projective space with proper mapping $embed$, and specify the projective space which we utilized.
In algebraic geometry, the (real) projective space $\mathbb{RP}^{N}$ is defined as quotient of $\mathbb{R}^{N+1}-\{0\}$ under equivalence relation $x\sim cx$ for all positive real constant $c$, comprised of lines in $\mathbb{R}^{N+1}$ passing through origin.
The element of $\mathbb{RP}^{N}$ can be expressed as $[v]=[v_{0}:\cdots:v_{N}]$ where

$$ $\forall c\neq 0,[v_{0}:\cdots:v_{N}]=[cv_{0}:\cdots:cv_{N}].$ (1) $$

The point $[v]$ of projective space $\mathbb{PR}^{N}$ exactly corresponds to the line in $\mathbb{R}^{N+1}$ whose direction vector is parallel to $v$ and passing through 0.

Considering the filter as a vector in $\mathbb{R}^{N}$, we can find the embedding function which maps filter $F_{i}=(F_{i1},\cdots,F_{iN})\neq 0$ to $\mathbb{RP}^{N}$ by following function:

$$ $embed:(F_{i1},\cdots,F_{iN})\mapsto[\|F_{i}\|:F_{i1}:\cdots:F_{iN}].$ (2) $$

In [Fig. 3(a)](https://arxiv.org/html/2507.14171v2#S3.F3.sf1), we visualize how the filter $F$ is mapped to the projective representation in $N=2$ case.
Note that $N=2$ appears in practice when the target layer is batch normalization layer.
For the zero filter $0_{F}$, the embedding would be $[1:0:\cdots:0]$, which corresponds to axis line of extra parameter in $\mathbb{R}^{N+1}$.
Another benefit of this placement to $\mathbb{RP}^{N}$ is that it naturally enjoys the magnitude-indifferent characteristic readily noticed by the scale invariance: $embed(cF)=embed(F)$ for $c>0$.

### 3.2 PROscore: a Novel Pruning Importance Score

In projective space $\mathbb{RP}^{N}$, the distance between two points may be considered as an angle between two lines in $\mathbb{R}^{N+1}$ where each line represents each point in $\mathbb{RP}^{N}$.
If we denote $B_{r}(0)$ be the set of points in $\mathbb{RP}^{N}$ whose angular distance between the origin is equal to $r$, then we can see that $Im(embed)\subseteq B_{\pi/4}(0)$; i.e. the distance between $embed(F)$ and the origin is always equal to $\pi/4$.
This is visualized in [Fig. 3(a)](https://arxiv.org/html/2507.14171v2#S3.F3.sf1) using line representations in $\mathbb{R}^{N+1}$ that $B_{\pi/4}(0)$ becomes the cone centered at 0 and $embed(F)$ will become a line included in $B_{\pi/4}(0)$.

This phenomenon is exploited to ensure fair chance for the filters to be pruned, overcoming the sole dependency on the magnitude.
After placing the filters to $B_{\pi/4}(0)$, we gauge the fate of each filters by estimating whether it would move closer to origin (and be pruned) or not, according to the direction of gradient descent.
The direction for each filter $F_{i}$ is represented by the angular distance $\theta(p_{i}^{\prime})$ between the origin and one-step forwarded point $p_{i}^{\prime}$ in $\mathbb{RP}^{N}$.

We define the tangent $\tan(\theta(p_{i}^{\prime}))$ as a novel importance score for pruning decision of filter $F_{i}$, and name it PRojective Offset score (PROscore).
Precisely, given $F_{i}=(F_{i1},\cdots,F_{iN})$ let $D_{i}$ be extra variable initialized by $\|F_{i}\|$. Then the point $p_{i}=(\|F_{i}\|,F_{i})$ would be updated to [Eq. 3](https://arxiv.org/html/2507.14171v2#S3.E3) by gradient descent on loss function $\mathcal{L}$, and $\lambda$ which is a hyperparameter that serves a similar role to weight decay in the loss function.

$$ $p_{i}^{\prime}=\left(\|F_{i}\|-\lambda\frac{\partial\mathcal{L}}{\partial D_{i }},F_{i}-\lambda\nabla_{F_{i}}\mathcal{L}\right)$ (3) $$

And the PROscore is computed by

$$ $tan(\theta(p_{i}^{\prime}))=\frac{\|F_{i}-\lambda\nabla_{F_{i}}\mathcal{L}\|}{ |\|F_{i}\|-\lambda\frac{\partial\mathcal{L}}{\partial D_{i}}|},$ (4) $$

which would be the proposed importance score of $i$-th filter.
As depicted in [Fig. 3(b)](https://arxiv.org/html/2507.14171v2#S3.F3.sf2), the $i$-th channel would have small $\theta(p_{i})$ when the red-colored movement of gradient descent makes $p_{i}$ closer and otherwise (the blue-colored arrow) not.

## 4 Implementation of IPPRO

We present Importance-based Pruning with PRojective Offset (IPPRO), our implementation of structural pruning using PROscore presented in [Section 3](https://arxiv.org/html/2507.14171v2#S3).
First, we overview the parameter injection trick to realize the projective space embedding.
Then, we present the algorithm to compute PROscore, our novel importance score for IPPRO, and conclude the section with the remaining implementation details.

### 4.1 Projective Offsetting via Parameter Injection

We implement the idea of embedding into the projective space as outlined in [Section 3](https://arxiv.org/html/2507.14171v2#S3) by introducing additional parameter $D$ with desired initialization $D^{init}=diag(\|F_{1}\|,\cdots,\|F_{N}\|)$.
To bring filters to projective space, we inject $D$ parameter to the model using the extension method catalyst, which adds two identical parameters $D$ and $\overline{D}$, by modifying element-wise computation layer $\sigma$, which appear next to the target layer $W$ as follows:

$$ $\psi_{D,\overline{D}}(x)=Dx-\overline{D}x+\sigma(x).$ (5) $$

The modification by $\psi_{D,\overline{D}}$ does not harm the model’s functionality and performance, since $D$ and $\overline{D}$ are set to be same.
The introduced parameters are removed to retrieve the original model before actual pruning happens, as they are needed for PROscore computation.

### 4.2 Computing PROscore

In [Algorithm 1](https://arxiv.org/html/2507.14171v2#alg1) we present the algorithm that we implemented to compute PROscore.

Figure: Algorithm 1 PROscore Computation

Given pretrained model with filters $F_{i}=W_{i,:}$, we compute $D^{init}$ by line 2 of [Algorithm 1](https://arxiv.org/html/2507.14171v2#alg1), and extend the model by adding parameters to $\sigma$, which are initialized by $D^{init}$.
The choice of $\sigma$ would be followed by the choice of target layer. For example, if the target is scaling factor of batch normalization layer in ResNet then $\sigma$ would be the ReLU activation.
After extending model, we estimate the loss gradient using the training dataset via backpropagation.
Finally, we compute the tangent value of angular distance $\theta(p_{i}^{\prime})$ between the origin and $p_{i}^{\prime}$.

### 4.3 Implementation Overview

IPPRO computes the PROscore without updating model parameters. Gradients are accumulated across the full dataset, and pruning is applied to the original model using the precomputed scores. This design avoids iterative retraining during scoring and improves scalability. Further details of the implementation are provided in [Appendix A](https://arxiv.org/html/2507.14171v2#A1).

To ensure fair comparisons, we follow standardized implementation protocols, including Torchvision-style preprocessing torchvision2016 and adopt sui2021chip fine-tuning settings. We avoid augmentation tricks such as mixup to ensure reproducibility and controlled evaluation. Full details of the fine-tuning recipe and implementation specifications are provided in [Appendix B](https://arxiv.org/html/2507.14171v2#A2).

## 5 Empirical Validations

**Table 1: Comparison of pruning performance on ImageNet dataset**
| Model | Method | Top-1 Acc (%) | Top-5 Acc (%) | Params($\downarrow$%) | FLOPs($\downarrow$%) |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Base. | Prun. | $\Delta$ | Base. | Prun. | $\Delta$ |  |  |  |  |
| ResNet-50 | SFP he2018soft | 76.15 | 74.61 | -1.54 | 92.87 | 92.06 | -0.81 | N/A | 2.3G(41.8) |
| Autopruner luo2020autopruner | 76.15 | 74.76 | -1.39 | 92.87 | 92.15 | -0.72 | N/A | 2.1G(48.7) |  |
| FPGM he2019filter | 76.15 | 75.59 | -0.56 | 92.87 | 92.63 | -0.24 | 15G(37.5) | 2.3G(42.2) |  |
| Taylor molchanov2019importance | 76.18 | 74.50 | -1.68 | N/A | N/A | N/A | 14G(44.5) | 2.2G(44.9) |  |
| GAL lin2019towards | 76.15 | 71.95 | -4.20 | 92.87 | 90.94 | -1.93 | 21G(16.9) | 2.3G(43.0) |  |
| HRank lin2020hrank | 76.15 | 74.98 | -1.17 | 92.87 | 92.33 | -0.54 | 16G(36.6) | 2.3G(43.7) |  |
| SCOP tang2020scop | 76.15 | 75.95 | -0.20 | 92.87 | 92.79 | -0.08 | 14G(42.8) | 2.2G(45.3) |  |
| CHIP sui2021chip | 76.15 | 76.15 | 0.00 | 92.87 | 92.91 | +0.04 | 14G(44.2) | 2.1G(48.7) |  |
| RGPchen2023rgp | 76.22 | 75.30 | -0.92 | N/A | N/A | N/A | 14G(43.8) | 2.3G(43.8) |  |
| FPBICItang2024filter | 76.13 | 76.08 | -0.05 | 92.86 | 92.85 | -0.01 | 14G(45.9) | 2.1G(50.4) |  |
| IPPRO (ours) | 76.15 | 76.21 | +0.06 | 92.87 | 93.02 | +0.15 | 13G(46.4) | 2.1G(47.7) |  |
| SCOP tang2020scop | 76.15 | 75.26 | -0.89 | 92.87 | 92.53 | -0.34 | 12G(51.8) | 1.8G(54.6) |  |
| SIRFP wu2024structural | 76.15 | 75.14 | -1.01 | 92.87 | 93.12 | +0.25 | N/A | 1.7G(58.7) |  |
| CHIP sui2021chip | 76.15 | 75.26 | -0.89 | 92.87 | 92.53 | -0.34 | 11G(56.7) | 1.5G(62.8) |  |
| Torque 10484493 | 76.07 | 74.58 | -1.49 | N/A | N/A | N/A | 9G(64.5) | 1.7G(57.2) |  |
| FPBICItang2024filter | 76.13 | 75.01 | -1.12 | 92.86 | 92.30 | -0.56 | 11G(57.7) | 1.5G(63.8) |  |
| IPPRO (ours) | 76.15 | 75.51 | -0.64 | 92.87 | 92.67 | -0.20 | 10G(60.0) | 1.6G(60.0) |  |
| HRank lin2020hrank | 76.15 | 69.10 | -7.05 | 92.87 | 89.58 | -3.29 | 8G(67.5) | 0.9G(76.0) |  |
| CHIP sui2021chip | 76.15 | 73.30 | -2.85 | 92.87 | 91.48 | -1.39 | 8G(68.6) | 0.9G(76.7) |  |
| RGP chen2023rgp | 76.22 | 72.68 | -3.54 | N/A | N/A | N/A | 6.4G(75.0) | 1.0G (75.0) |  |
| IPPRO (ours) | 76.15 | 73.38 | -2.77 | 92.87 | 91.50 | -1.37 | 6.5G(74.2) | 1.0G(74.2) |  |
| MobileNetV2 | Meta liu2019metapruning | 74.70 | 68.20 | -6.50 | N/A | N/A | N/A | N/A | 0.14G (54.2) |
| GFP liu2021group | 75.74 | 69.16 | -6.58 | N/A | N/A | N/A | N/A | 0.15G (50.5) |  |
| IPPRO (ours) | 72.01 | 67.90 | -4.11 | 90.62 | 88.04 | -2.58 | 0.2G (42.3) | 0.14G (54.2) |  |

We validate the effectiveness of IPPRO across diverse vision tasks, including image classification and semantic segmentation. Specifically, experiments are conducted on ResNet-50 and MobileNetV2 using ImageNet-1k, ResNet-56 on CIFAR-10, VGG19 on CIFAR-100, and DeepLabv3-ResNet50 on the Cityscapes dataset.

IPPRO is compared with state-of-the-art pruning methods such as Taylor molchanov2019importance, FPGM he2019filter, Torque 10484493, SFP he2018soft, and SCOP tang2020scop, as well as approaches leveraging non-magnitude cues like HRank lin2020hrank, CHIP sui2021chip, and AutoPruner luo2020autopruner. Across all benchmarks, IPPRO consistently delivers better or comparable performance with reduced model size.

We also examine sensitivity to fine-tuning and data sampling, confirming that IPPRO yields stable and favorable trade-offs between accuracy and model efficiency, effectively challenging the “size-matters” heuristics prevailing in neural network pruning methods.

### 5.1 Pruning Performance Analysis

We compare IPPRO with existing magnitude-based pruning methods, showing that it maintains or even improves baseline performance.

#### 5.1.1 CIFAR-10 and CIFAR-100

We evaluated IPPRO on CIFAR-10 (ResNet-56) and CIFAR-100 (VGG19) using DepGraph pre-trained weights fang2023depgraph. As shown in [Table 2(a)](https://arxiv.org/html/2507.14171v2#S5.T2.st1), IPPRO reduces FLOPs and parameters by up to 49% on CIFAR-10 with a 0.55% accuracy gain. Even at higher compression rates (up to 77%), accuracy drops by only  1%, outperforming other methods. On CIFAR-100, IPPRO achieves 87% FLOPs and 87% parameter reduction with minimal accuracy loss, highlighting its effectiveness for compact, high-performing models.

Figure: Figure 4: Qualitative segmentation results of IPPRO on the Cityscapes dataset using DeepLabv3.
Refer to caption: x6.png

**Table 2: Pruning Performance results of various tasks and datasets.**
| Model<br>Dataset | Method | Top-1 Acc (%) | Params<br>($\downarrow$%) | FLOPs<br>($\downarrow$%) |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  |  | Base. | Prun. | $\Delta$ |  |  |
| ResNet-56<br>CIFAR-10 | Depgraphfang2023depgraph | 93.53 | 93.77 | +0.24 | - | - |
| HBFPbasha2024deep | 93.26 | 92.42 | -0.84 | 43.9 | 43.6 |  |
| GALlin2019towards | 93.26 | 93.38 | +0.12 | 12.3 | 37.6 |  |
| HRanklin2020hrank | 93.26 | 93.52 | +0.26 | 17.0 | 29.3 |  |
| CHIPsui2021chip | 93.26 | 94.16 | +0.90 | 43.9 | 47.4 |  |
| FPACyang2023filter | 93.26 | 93.71 | +0.45 | 42.8 | 47.4 |  |
| IPPRO (ours) | 93.53 | 94.06 | +0.53 | 49.7 | 49.3 |  |
| HRanklin2020hrank | 93.26 | 90.72 | -2.54 | 68.4 | 74.1 |  |
| HBFPbasha2024deep | 93.26 | 91.79 | -1.47 | 75.4 | 74.9 |  |
| GALlin2019towards | 93.26 | 91.58 | -1.68 | 66.1 | 60.2 |  |
| CHIPsui2021chip | 93.26 | 92.05 | -1.21 | 71.8 | 72.3 |  |
| SPSRCsun2024towards | 93.59 | 91.65 | -1.94 | 64.7 | 63.9 |  |
| IPPRO (ours) | 93.53 | 92.47 | -1.06 | 77.7 | 71.8 |  |
| VGG19<br>CIFAR-100 | Depgraphfang2023depgraph | 73.50 | 70.39 | -3.11 | - | 88.7 |
| Kron-OBSwang2019eigendamage | 73.34 | 60.66 | -12.6 | - | 83.5 |  |
| Greg-2wang2020greg | 74.02 | 67.75 | -6.27 | - | 88.6 |  |
| EigenDwang2019eigendamage | 73.34 | 65.18 | -8.16 | 90.9 | 88.6 |  |
| Torquegupta2024torque | 73.03 | 65.87 | -7.16 | 90.8 | 88.7 |  |
| GAT Transpruniglin2024gat | 73.26 | 66.68 | -6.58 | - | 89.0 |  |
| IPPRO (ours) | 73.50 | 70.47 | -3.03 | 87.9 | 87.5 |  |

#### 5.1.2 ImageNet-1k

On ImageNet-1k, we evaluate IPPRO with ResNet-50 under three compression levels (47%, 60%, and 74% FLOPs reduction). As shown in [Table 1](https://arxiv.org/html/2507.14171v2#S5.T1), IPPRO improves Top-1 accuracy by 0.06% at moderate pruning and remains competitive at higher compression, outperforming state-of-the-art methods. Furthermore, for other FLOP reductions on 60% and 74%, IPPRO exhibited significantly higher accuracy compared to other superior methods.

We further validate IPPRO on MobileNetV2, a compact model without residual connections, where it shows the smallest accuracy drop at comparable FLOPs, demonstrating architectural versatility.

#### 5.1.3 Segmentation

To further assess IPPRO, we apply it to a semantic segmentation task using DeepLabv3 with ResNet-50 on the Cityscapes dataset cordts2016cityscapes, following the experimental setup used by DCFP wang2023dcfp.

As seen in [Table 2(b)](https://arxiv.org/html/2507.14171v2#S5.T2.st2), IPPRO maintains full mIoU performance at around 60% reduction of model size and consistently outperforms or matches other methods across various pruning levels. The qualitative segmentation results [Fig. 4](https://arxiv.org/html/2507.14171v2#S5.F4), along with additional examples in [Fig. 6](https://arxiv.org/html/2507.14171v2#A6.F6), do not show noticeable degradation. From left to right in [Fig. 4](https://arxiv.org/html/2507.14171v2#S5.F4): input image, ground truth, unpruned model, and IPPRO-pruned models with 60%, 70%, and 80% sparsity, respectively. These results confirm that IPPRO preserves high-level task performance, even at high pruning ratios, by maintaining semantic structure and object boundary quality.

**Table 3: Global Pruning Performance results of ImageNet and CIFAR-10 datasets.**
| Method | Top-1 Acc (%) | FLOPs(G) |  |  |
| --- | --- | --- | --- | --- |
|  | Base. | Prun. | $\Delta$ |  |
| ResConv-Prunexu2020layer | 76.2 | 70.0 | -6.2 | 1.6 |
| DBP-0.5wang2019dbp | 76.2 | 72.4 | -3.8 | N/A |
| Metaliu2019metapruning | 76.2 | 73.4 | -2.8 | 1.0 |
| AutoSlimyu2019autoslim | 76.2 | 74.0 | -2.2 | 1.0 |
| GReg-2wang2020greg | 76.2 | 73.9 | -2.3 | 1.3 |
| HALPshen2022structural | 76.2 | 74.5 | -1.7 | 1.2 |
| IPPRO (ours) | 76.2 | 74.6 | -1.6 | 1.3 |
| HALPshen2022structural | 76.2 | 68.1 | -8.1 | 0.6 |
| MDP sun2024multi | 76.2 | 70.0 | -6.2 | 0.5 |
| IPPRO (ours) | 76.2 | 69.9 | -6.3 | 0.6 |

#### 5.1.4 Global Pruning

We define our importance score using angular deviation in the projective space, measuring how much each filter responds to data relative to a reference value of $\tan(\frac{\pi}{4})$. Although the absolute score can vary with the hyperparameter $\lambda$, PROscore enables global comparability without explicit normalization, unlike magnitude-based methods. This property is essential for global pruning, where importance scores must reside on a comparable scale to ensure fair competition among filters across layers. Without this, layers with intrinsically larger scores may be unjustly favored. To verify that score scaling induced by [Eq. 4](https://arxiv.org/html/2507.14171v2#S3.E4) $\lambda$ does not undermine pruning behavior, we analyze its effect and confirm the stability of our method in [Appendix G](https://arxiv.org/html/2507.14171v2#A7).

By leveraging this robust scoring scheme, our global pruning method consistently outperforms conventional magnitude-based global pruning. As shown in [Table 3](https://arxiv.org/html/2507.14171v2#S5.T3), it also achieves performance on par with other advanced approaches specifically designed to address the shortcomings of norm-based criteria.

### 5.2 Comparison of Importance Criteria

We compare IPPRO with other magnitude- and gradient-based pruning methods such as $L_{1}$-norm he2017channel, Taylor molchanov2019importance, and geometric he2019filter. For fairness, we compute gradients across the entire dataset, even for methods like Taylor that support randomized subsets. As shown in [Table 4](https://arxiv.org/html/2507.14171v2#A4.T4) and [Fig. 2](https://arxiv.org/html/2507.14171v2#S2.F2), traditional methods heavily depend on pre-trained weights and produce highly similar pruning patterns. In contrast, IPPRO demonstrates reduced reliance on initial weights, yielding distinct importance distributions. Histogram visualizations of ResNet-50 layers further confirm that IPPRO prunes differently, supporting its robustness against weight initialization bias.

Figure: (a) layer-wise pruning
Refer to caption: x9.png

### 5.3 Results without Finetuning

To examine whether the effectiveness of IPPRO depends heavily on fine-tuning, we evaluated its performance without any fine-tuning and compared it with $L_{1}$-norm he2017channel, Taylor molchanov2019importance, geometric he2019filter, and random pruning methods. Using a uniform pruning ratio across all layers, IPPRO consistently outperformed all baselines on the ImageNet-1k dataset with ResNet-50, as shown in [Fig. 5](https://arxiv.org/html/2507.14171v2#S5.F5). On CIFAR-10, while most methods performed similarly at high FLOPs, IPPRO exhibited significantly less performance degradation as the pruning ratio increased, demonstrating its applicability in low-resource environments such as embedded or on-device setting. ([Fig. 5](https://arxiv.org/html/2507.14171v2#S5.F5)).

### 5.4 Sampling Sensitivity Analysis

To evaluate the robustness of our importance-score estimation, we tested the effect of using only a small subset (5–50%) of the training data when computing gradients. As summarized in [Table 5](https://arxiv.org/html/2507.14171v2#A5.T5), pruning performance remains nearly unchanged even with 5% of the data, while achieving significant computational speedups. For detailed analysis and experimental results, please refer to [Appendix E](https://arxiv.org/html/2507.14171v2#A5).

## 6 Discussion

IPPRO outshines state-of-the-art methods in high-sparsity pruning regimes as demonstrated in [Section 5.3](https://arxiv.org/html/2507.14171v2#S5.SS3).
While conventional importance-based pruning methods are heavily influenced by filter magnitude, PROscore decouples pruning decision from filter magnitude and enables IPPRO to prune of both small and large-magnitude filters as necessary for effective pruning.
In some cases, layer-wise pruning shows marginally better results ([Table 1](https://arxiv.org/html/2507.14171v2#S5.T1), [Table 3(a)](https://arxiv.org/html/2507.14171v2#S5.T3.st1)), yet IPPRO controls the risk of over-pruning often seen in global pruning schemes and maintains strong overall performance.

PROscore balances computational feasibility and informative scoring for importance-based pruning: computing full-batch gradient only once to estimate filter importance with high correlation to the eventual prune-or-preserve decision.
Nevertheless, tracing the gradient descent trajectories beyond a single step may yield further refinement to stabilize and improve the importance score computation.
Moreover, there may be a deeper connection between IPPRO and Catalyst catalyst, an emerging regularizer for lossless structural pruning motivated by the geometry of pruning-invariant model parameter subspace.
This observation naturally leads to a research conjecture that PROscore-based pruning is related to Catalyst-based pruning and its empirical success is partially due to Catalyst’s theoretical guarantee of robust bifurcation between pruned and preserved filters.

Despite the demonstrated strengths, IPPRO has some limitations such as computational overhead facing large datasets as PROscore requires full-batch gradient.
Our pilot research shown in [Appendix E](https://arxiv.org/html/2507.14171v2#A5) suggests that PROscore is surprisingly reliable even when only a subset of full-batch data is used, and this leads to natural future research direction of a stochastic counterpart of IPPRO for online structural pruning.
Moreover, our empirical validations are confined to CNN-based models which represents most of current on-device vision models.
Since PROscore is model-agnostic and IPPRO generally applicable to convolutional components such as ASPP in the DeepLabV3-ResNet50 model chen2017deeplab, further validation studies such as on Vision Transformers and Swin Transformers remain as valuable future work.

## 7 Conclusion

We introduce PROscore, a novel magnitude-invariant importance score for structural pruning.
PROscore challenges conventional over-reliance on magnitude and takes a fundamentally different approach to gauge filter importance via angular displacement of filters under gradient descent in real projective space.
We construct a novel pruning algorithm, IPPRO, as a theoretically well-founded innovation beyond the longstanding “size-matters” heuristics in structured pruning.
Empirical validations on IPPRO shows superior performance in standard benchmark scenarios and robustness under extreme pruning conditions.
By introducing and validating the novel theoretical perspective on magnitude-decoupled importance-based pruning strategy, this work not only improves state-of-the-art performance but also offers a fresh lens of projective geometry for structural pruning research.

## References

- (1)
Lei Deng, Guoqi Li, Song Han, Luping Shi, and Yuan Xie.
Model compression and hardware acceleration for neural networks: A
comprehensive survey.
Proceedings of the IEEE, 108(4):485–532, 2020.
- (2)
Yang He and Lingao Xiao.
Structured pruning for deep convolutional neural networks: A survey.
IEEE Transactions on Pattern Analysis and Machine Intelligence,
pages 1–20, 2023.
- (3)
Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui
Zhang.
Learning efficient convolutional networks through network slimming.
In Proceedings of the IEEE international conference on computer
vision, pages 2736–2744, 2017.
- (4)
Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf.
Pruning filters for efficient convnets.
In International Conference on Learning Representations, 2017.
- (5)
Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li.
Learning structured sparsity in deep neural networks.
Advances in neural information processing systems, 29, 2016.
- (6)
Baoyuan Liu, Min Wang, Hassan Foroosh, Marshall Tappen, and Marianna Pensky.
Sparse convolutional neural networks.
In Proceedings of the IEEE conference on computer vision and
pattern recognition, pages 806–814, 2015.
- (7)
Jian-Hao Luo, Jianxin Wu, and Weiyao Lin.
Thinet: A filter level pruning method for deep neural network
compression.
In Proceedings of the IEEE international conference on computer
vision, pages 5058–5066, 2017.
- (8)
Yang He, Ping Liu, Ziwei Wang, Zhilan Hu, and Yi Yang.
Filter pruning via geometric median for deep convolutional neural
networks acceleration.
In Proceedings of the IEEE/CVF Conference on Computer Vision and
Pattern Recognition (CVPR), June 2019.
- (9)
Zhuang Liu, Mingjie Sun, Tinghui Zhou, Gao Huang, and Trevor Darrell.
Rethinking the value of network pruning.
In International Conference on Learning Representations, 2018.
- (10)
Pavlo Molchanov, Arun Mallya, Stephen Tyree, Iuri Frosio, and Jan Kautz.
Importance estimation for neural network pruning.
In Proceedings of the IEEE/CVF conference on computer vision and
pattern recognition, pages 11264–11272, 2019.
- (11)
Yang He, Ping Liu, Ziwei Wang, Zhilan Hu, and Yi Yang.
Filter pruning via geometric median for deep convolutional neural
networks acceleration.
In Proceedings of the IEEE/CVF conference on computer vision and
pattern recognition, pages 4340–4349, 2019.
- (12)
Song Han, Huizi Mao, and William J Dally.
Deep compression: Compressing deep neural networks with pruning,
trained quantization and huffman coding.
arXiv preprint arXiv:1510.00149, 2015.
- (13)
Davis Blalock, Jose Javier Gonzalez Ortiz, Jonathan Frankle, and John Guttag.
What is the state of neural network pruning?
Proceedings of machine learning and systems, 2:129–146, 2020.
- (14)
Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui
Zhang.
Learning efficient convolutional networks through network slimming.
In Proceedings of the IEEE international conference on computer
vision, pages 2736–2744, 2017.
- (15)
Hancheng Ye, Bo Zhang, Tao Chen, Jiayuan Fan, and Bin Wang.
Performance-aware approximation of global channel pruning for
multitask cnns.
IEEE Transactions on Pattern Analysis and Machine Intelligence,
45(8):10267–10284, 2023.
- (16)
Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf.
Pruning filters for efficient convnets.
In International Conference on Learning Representations, 2017.
- (17)
Yang He, Guoliang Kang, Xuanyi Dong, Yanwei Fu, and Yi Yang.
Soft filter pruning for accelerating deep convolutional neural
networks.
arXiv preprint arXiv:1808.06866, 2018.
- (18)
Edouard Yvinec, Arnaud Dapogny, Matthieu Cord, and Kevin Bailly.
Red: Looking for redundancies for data-freestructured compression of
deep neural networks.
Advances in Neural Information Processing Systems,
34:20863–20873, 2021.
- (19)
Edouard Yvinec, Arnaud Dapogny, Matthieu Cord, and Kevin Bailly.
Red++: Data-free pruning of deep neural networks via input splitting
and output merging.
IEEE Transactions on Pattern Analysis and Machine Intelligence,
45(3):3664–3676, 2022.
- (20)
Wenxiao Wang, Cong Fu, Jishun Guo, Deng Cai, and Xiaofei He.
Cop: Customized deep model compression via regularized
correlation-based filter-level pruning.
arXiv preprint arXiv:1906.10337, 2019.
- (21)
Zi Wang, Chengcheng Li, and Xiangyang Wang.
Convolutional neural network pruning with structural redundancy
reduction.
In Proceedings of the IEEE/CVF conference on computer vision and
pattern recognition, pages 14913–14922, 2021.
- (22)
Arshita Gupta, Tien Bau, Joonsoo Kim, Zhe Zhu, Sumit Jha, and Hrishikesh Garud.
Torque based structured pruning for deep neural network.
In 2024 IEEE/CVF Winter Conference on Applications of Computer
Vision (WACV), pages 2699–2708, 2024.
- (23)
Anonymous.
Catalyst pruning: Lossless structured pruning via algebraically
principled regularization.
Under Review, 2025.
- (24)
TorchVision maintainers and contributors.
Torchvision: Pytorch’s computer vision library.
[https://github.com/pytorch/vision](https://github.com/pytorch/vision), 2016.
- (25)
Yang Sui, Miao Yin, Yi Xie, Huy Phan, Saman Aliari Zonouz, and Bo Yuan.
Chip: Channel independence-based pruning for compact neural networks.
Advances in Neural Information Processing Systems,
34:24604–24616, 2021.
- (26)
Jian-Hao Luo and Jianxin Wu.
Autopruner: An end-to-end trainable filter pruning method for
efficient deep model inference.
Pattern Recognition, 107:107461, 2020.
- (27)
Shaohui Lin, Rongrong Ji, Chenqian Yan, Baochang Zhang, Liujuan Cao, Qixiang
Ye, Feiyue Huang, and David Doermann.
Towards optimal structured cnn pruning via generative adversarial
learning.
In Proceedings of the IEEE/CVF conference on computer vision and
pattern recognition, pages 2790–2799, 2019.
- (28)
Mingbao Lin, Rongrong Ji, Yan Wang, Yichen Zhang, Baochang Zhang, Yonghong
Tian, and Ling Shao.
Hrank: Filter pruning using high-rank feature map.
In Proceedings of the IEEE/CVF conference on computer vision and
pattern recognition, pages 1529–1538, 2020.
- (29)
Yehui Tang, Yunhe Wang, Yixing Xu, Dacheng Tao, Chunjing Xu, Chao Xu, and Chang
Xu.
Scop: Scientific control for reliable neural network pruning.
Advances in Neural Information Processing Systems,
33:10936–10947, 2020.
- (30)
Zhuangzhi Chen, Jingyang Xiang, Yao Lu, Qi Xuan, Zhen Wang, Guanrong Chen, and
Xiaoniu Yang.
Rgp: Neural network pruning through regular graph with edges
swapping.
IEEE Transactions on Neural Networks and Learning Systems,
2023.
- (31)
Xiaolong Tang, Shuo Ye, Yufeng Shi, Tianheng Hu, Qinmu Peng, and Xinge You.
Filter pruning based on information capacity and independence.
IEEE Transactions on Neural Networks and Learning Systems,
2024.
- (32)
Dongyue Wu, Zilin Guo, Li Yu, Nong Sang, and Changxin Gao.
Structural pruning via spatial-aware information redundancy for
semantic segmentation.
arXiv preprint arXiv:2412.12672, 2024.
- (33)
Zechun Liu, Haoyuan Mu, Xiangyu Zhang, Zichao Guo, Xin Yang, Kwang-Ting Cheng,
and Jian Sun.
Metapruning: Meta learning for automatic neural network channel
pruning.
In Proceedings of the IEEE/CVF international conference on
computer vision, pages 3296–3305, 2019.
- (34)
Liyang Liu, Shilong Zhang, Zhanghui Kuang, Aojun Zhou, Jing-Hao Xue, Xinjiang
Wang, Yimin Chen, Wenming Yang, Qingmin Liao, and Wayne Zhang.
Group fisher pruning for practical network compression.
In International Conference on Machine Learning, pages
7021–7032. PMLR, 2021.
- (35)
Gongfan Fang, Xinyin Ma, Mingli Song, Michael Bi Mi, and Xinchao Wang.
Depgraph: Towards any structural pruning.
In Proceedings of the IEEE/CVF Conference on Computer Vision and
Pattern Recognition, pages 16091–16101, 2023.
- (36)
SH Shabbeer Basha, Mohammad Farazuddin, Viswanath Pulabaigari, Shiv Ram Dubey,
and Snehasis Mukherjee.
Deep model compression based on the training history.
Neurocomputing, 573:127257, 2024.
- (37)
Huoxiang Yang, Yongsheng Liang, Wei Liu, and Fanyang Meng.
Filter pruning via attention consistency on feature maps.
Applied Sciences, 13(3):1964, 2023.
- (38)
Xinglong Sun and Humphrey Shi.
Towards better structured pruning saliency by reorganizing
convolution.
In Proceedings of the IEEE/CVF Winter Conference on Applications
of Computer Vision, pages 2204–2214, 2024.
- (39)
Chaoqi Wang, Roger Grosse, Sanja Fidler, and Guodong Zhang.
Eigendamage: Structured pruning in the kronecker-factored eigenbasis.
In International conference on machine learning, pages
6566–6575. PMLR, 2019.
- (40)
Huan Wang, Can Qin, Yulun Zhang, and Yun Fu.
Neural pruning via growing regularization.
arXiv preprint arXiv:2012.09243, 2020.
- (41)
Arshita Gupta, Tien Bau, Joonsoo Kim, Zhe Zhu, Sumit Jha, and Hrishikesh Garud.
Torque based structured pruning for deep neural network.
In Proceedings of the IEEE/CVF Winter Conference on Applications
of Computer Vision, pages 2711–2720, 2024.
- (42)
Yu-Chen Lin, Chia-Hung Wang, and Yu-Cheng Lin.
Gat transpruning: progressive channel pruning strategy combining
graph attention network and transformer.
PeerJ Computer Science, 10:e2012, 2024.
- (43)
Zixiao Wang, Hongtao Xie, Yuxin Wang, Hai Xu, and Guoqing Jin.
Dcfp: Distribution calibrated filter pruning for lightweight and
accurate long-tail semantic segmentation.
IEEE Transactions on Circuits and Systems for Video Technology,
34(7):6063–6076, 2023.
- (44)
Qingsong Lv, Jiasheng Sun, Sheng Zhou, Xu Zhang, Liangcheng Li, Yun Gao, Sun
Qiao, Jie Song, and Jiajun Bu.
Fgp: Feature-gradient-prune for efficient convolutional layer
pruning.
arXiv preprint arXiv:2411.12781, 2024.
- (45)
Dongyue Wu, Zilin Guo, Li Yu, Nong Sang, and Changxin Gao.
Structural pruning via spatial-aware information redundancy for
semantic segmentation.
In Proceedings of the AAAI Conference on Artificial
Intelligence, volume 39, pages 8368–8376, 2025.
- (46)
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler,
Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele.
The cityscapes dataset for semantic urban scene understanding.
In Proceedings of the IEEE conference on computer vision and
pattern recognition, pages 3213–3223, 2016.
- (47)
Pengtao Xu, Jian Cao, Fanhua Shang, Wenyu Sun, and Pu Li.
Layer pruning via fusible residual convolutional block for deep
neural networks.
arXiv preprint arXiv:2011.14356, 2020.
- (48)
Wenxiao Wang, Shuai Zhao, Minghao Chen, Jinming Hu, Deng Cai, and Haifeng Liu.
Dbp: Discrimination based block-level pruning for deep model
acceleration.
arXiv preprint arXiv:1912.10178, 2019.
- (49)
Jiahui Yu and Thomas Huang.
Autoslim: Towards one-shot architecture search for channel numbers.
arXiv preprint arXiv:1903.11728, 2019.
- (50)
Maying Shen, Hongxu Yin, Pavlo Molchanov, Lei Mao, Jianna Liu, and Jose M
Alvarez.
Structural pruning via latency-saliency knapsack.
Advances in Neural Information Processing Systems,
35:12894–12908, 2022.
- (51)
Xinglong Sun, Barath Lakshmanan, Maying Shen, Shiyi Lan, Jingde Chen, and Jose
Alvarez.
Multi-dimensional pruning: Joint channel, layer and block pruning
with latency constraint.
arXiv preprint arXiv:2406.12079, 2024.
- (52)
Yuanzhi Duan, Yue Zhou, Peng He, Qiang Liu, Shukai Duan, and Xiaofang Hu.
Network pruning via feature shift minimization.
In Proceedings of the Asian conference on computer vision,
pages 4044–4060, 2022.
- (53)
Yihao Chen and Zefang Wang.
An effective information theoretic framework for channel pruning.
arXiv preprint arXiv:2408.16772, 2024.
- (54)
Di Jiang, Yuan Cao, and Qiang Yang.
On the channel pruning using graph convolution network for
convolutional neural network acceleration.
In IJCAI, pages 3107–3113, 2022.
- (55)
Yajun Liu, Kefeng Fan, Dakui Wu, and Wenju Zhou.
Filter pruning by quantifying feature similarity and entropy of
feature maps.
Neurocomputing, 544:126297, 2023.
- (56)
Zidu Wang, Xuexin Liu, Long Huang, Yunqing Chen, Yufei Zhang, Zhikang Lin, and
Rui Wang.
Qsfm: Model pruning based on quantified similarity between feature
maps for ai on edge.
IEEE Internet of Things Journal, 9(23):24506–24515, 2022.
- (57)
Zhonghui You, Kun Yan, Jinmian Ye, Meng Ma, and Ping Wang.
Gate decorator: Global filter pruning method for accelerating deep
convolutional neural networks.
Advances in neural information processing systems, 32, 2019.
- (58)
Yihui He, Xiangyu Zhang, and Jian Sun.
Channel pruning for accelerating very deep neural networks.
In Proceedings of the IEEE international conference on computer
vision, pages 1389–1397, 2017.
- (59)
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L
Yuille.
Deeplab: Semantic image segmentation with deep convolutional nets,
atrous convolution, and fully connected crfs.
IEEE transactions on pattern analysis and machine intelligence,
40(4):834–848, 2017.

## Appendix A Implementation Detail

Note that the model parameters are not updated during the PROscore computation.
The extended model is reverted to the original model and then we prune the filters according to the obtained PROscore.

To reduce the randomness of the empirical results of our implementation, we fixed the pre-trained weights $W$ and the parameter $D$ obtained from the extended projective space computed using the pre-trained weights.
Also, we accumulate the gradients $\mathcal{\nabla}_{F_{i}}\mathcal{L}$ and $\frac{\partial\mathcal{L}}{\partial D_{i}}$ from each batch of the backpropagation process using the entire dataset $\mathcal{D}$, unless otherwise mentioned.

Using the accumulated gradient values, along with the original model weights and the expanded parameter $D$, the layer-wise PROscore is computed according to [Eq. 4](https://arxiv.org/html/2507.14171v2#S3.E4).
Using the PROscore, layer-wise pruning is performed on the original base model, which does not include the extended parameter $D$ using the previously computed importance scores.

## Appendix B Finetune Recipe

In this section, we detail the hyper-parameters used for fine-tuning. All or some of the models were fine-tuned using the PyTorch 1.13 framework on a Nvidia RTX 4090 GPU.
Additionally, we emphasize that we did not use external data augmentation skills such as Color-jitter or Mix-up, which often employed to improve the performance in other pruning methods, since it invokes ethical issue on fair-comparison, as pretrained model did not used them and results with additional augmentation may not show the advantage of pruning method only.

### B.1 ImageNet-1k dataset

For Importance score estimation on the ImageNet-1k dataset, we accumulated gradients over all data for one epoch for both ResNet-50 and MobileNetV2. For ResNet-50, fine-tuning was performed for a total of 180 epochs with a batch size of 128, using the SGD optimizer and a StepLR scheduler with a step size of 40.
The initial learning rate was set to 0.01, with weight decay of $10^{-4}$ and momentum of 0.9, while the norm weight decay was 5e-5 and the momentum was 0.
For MobileNetV2, fine-tuning was carried out for 300 epochs with a batch size of 256, using the SGD optimizer and a cosine scheduler. The initial learning rate was also set to 0.01, with weight decay of $10^{-4}$ and momentum of 0.9.

### B.2 CIFAR-10 dataset

On the CIFAR-10 dataset, for the ResNet-56 model with 71.8% FLOPs reduction, fine-tuning was conducted for 400 epochs with a batch size of 256, using the SGD optimizer and a cosine scheduler with a step size of 30. The initial learning rate was 0.025, with weight decay and momentum set to 0.0005 and 0.9, respectively. The same configuration was applied to the data size experiments for ResNet-56 in [Table 5](https://arxiv.org/html/2507.14171v2#A5.T5). For the ResNet-56 model with 49.3% FLOPs reduction, the same fine-tuning settings were used: 400 epochs, batch size of 256, SGD optimizer, cosine scheduler, initial learning rate of 0.03, and weight decay and momentum of 0.0005 and 0.9, respectively.

### B.3 CIFAR-100 dataset

On the CIFAR-100 dataset, for the VGG-19 model with 87.5% FLOPs reduction, fine-tuning was conducted for 400 epochs with a batch size of 256, using the SGD optimizer and a cosine scheduler. The initial learning rate was 0.017 with weight decay and momentum set to 0.005 and 0.9, respectively.

## Appendix C Pruning ratios

We provide the detailed layer-wise pruning ratios used for all models in our experiments. For each architecture, pruning rates are adjusted per layer group to account for differences in pruning sensitivity and to preserve overall performance.

- •
ResNet-50 (47.7%) : 0.25 for Downsample layer and 0.4 for the rest.
- •
ResNet-50 (60%) : 0.35 for Downsample layer and 0.45 for the rest.
- •
ResNet-50 (74.2%) : 0.45 for Downsample layer and 0.6 for the rest.
- •
MobileNetV2 (54.2%) : 0.2 for the first layer and 0.35 for the rest
- •
ResNet-56 (49.3%) : 0.15 for Downsample layer and 0.4 for the rest.
- •
ResNet-56 (71.8%) : 0.3 for Downsample layer, 0.6,0.65 and 0.7 for the 1st, 2nd and 3rd blocks respectively.
- •
VGG19 (87.5%) : 0.65 for first 8 blocks, 0.7 for the next 4 blocks, and 0.6 for the rest.

## Appendix D Importance score Correlation Table

[Table 4](https://arxiv.org/html/2507.14171v2#A4.T4) presents the pairwise correlation of importance scores among four pruning methods—Magnitude (L1-norm), Taylor, Geometric, and our proposed method—on ResNet-50 trained on ImageNet-1k.

The results reveal that traditional methods exhibit high similarity, indicating a shared reliance on pre-trained weight distributions. In contrast, our method shows lower correlation with them, suggesting it captures a distinct notion of importance.

**Table 4: Correlation of ResNet-50 on Imagenet-1k between different magnitude based pruning methods.**
| Method | Magnitude | Taylor | Geometric | IPPRO |
| --- | --- | --- | --- | --- |
| Magnitude | 1.000 | 0.682 | 0.809 | 0.033 |
| Taylor | 0.682 | 1.000 | 0.433 | 0.064 |
| Geometric | 0.809 | 0.433 | 1.000 | 0.003 |
| IPPRO (ours) | 0.033 | 0.064 | 0.003 | 1.000 |

## Appendix E Sensitivity Analysis

IPPRO computes importance scores by accumulating gradients, as defined in [Eq. 4](https://arxiv.org/html/2507.14171v2#S3.E4), resulting in linear time complexity with respect to the number of input samples. While using the full dataset ensures maximum accuracy, it incurs substantial computational cost. To evaluate the trade-off between efficiency and performance, we analyze the sensitivity of gradient-based importance estimation to the number of samples used.

We conduct experiments on ResNet-56 with CIFAR-10 and VGG-19 with CIFAR-100, varying the number of samples used to 100%, 50%, 25%, and 5% of the dataset. A balanced label sampler is employed to preserve label distribution across mini-batches. To account for randomness introduced by subset sampling, each configuration is repeated five times, and both the mean (Mu) and the maximum accuracy are reported.

As shown in [Table 5](https://arxiv.org/html/2507.14171v2#A5.T5), our method exhibits minimal accuracy degradation even when using as little as 5% of the dataset for importance score computation. This indicates that IPPRO is robust to subset-based sampling during pruning.
Moreover, this implies immense empirical speedup potential of our implementation without sacrificing pruning performance.

**Table 5: Experimental results on different dataset size**
| Dataset<br>size | Time<br>usage (s) | Top-1 Acc (%) |  |  |
| --- | --- | --- | --- | --- |
| Base. | Prun. Mu (Max) | $\Delta$ Mu (Max) |  |  |
| Full | 45.4 | 93.53 | 92.47 | -1.06 |
| 50% | 25.4 | 93.53 | 92.18 (92.44) | -1.35 (-1.09) |
| 25% | 16.6 | 93.53 | 92.27 (92.46) | -1.26 (-1.07) |
| 5% | 7.2 | 93.53 | 92.21 (92.43) | -1.32 (-1.10) |

## Appendix F Qualitative Results of DeeplabV3

In this section, we present qualitative results of the segmentation task.
We compare the original image, ground truth segmentation, segmentation result from the unpruned model, and the segmentation result after pruning using our PROscore.

Figure: Figure 6: Visualization CitySpace dataset Our Pruning results of using DeeplabV3-ResNet50 models
Refer to caption: x13.png

## Appendix G Sensitivity Analysis of λ 𝜆 \lambda italic_λ value

The absolute value of PROscore is affected by the hyperparameter $\lambda$, which controls the step size of the gradient movement in real projective space $\mathbb{RP}^{N}$, since the angle $\theta(p_{i}^{\prime})$ is related to the length $\overline{p_{i}p_{i}^{\prime}}$. In extreme case, if $\lambda\to 0$ then $\theta(p_{i}^{\prime})\to\frac{\pi}{4}$ and thus the PROscores would distribute close to one.

However, the pruning decision are invariant under selection of isotropic scaling on $\lambda>0$, since we are choosing filters with respect to their relative order which are invariant: if the $p_{i}^{\prime\prime}$ is new point under gradient movement with different step size $\lambda$, then following holds: if $\theta(p_{i}^{\prime})<\theta(p_{j}^{\prime})$ then $\theta(p_{i}^{\prime\prime})<\theta(p_{j}^{\prime\prime})$.

To validate this, we conduct experiments on Resnet56 model with $\lambda$ set to 1, 0.1, 0.01, and 0.001, examining both the preservation of pruning indices and the consistency of PROscore scales across layers.
As shown in [Fig. 7(a)](https://arxiv.org/html/2507.14171v2#A7.F7.sf1), the pruning indices remained stable regardless of the $\lambda$ value, and [Fig. 7(b)](https://arxiv.org/html/2507.14171v2#A7.F7.sf2) shows that the relative importance measured by PROscore is invariant under $\lambda$ selection.

Figure: (a) Preserve Pruning Index of various $\lambda$ value
Refer to caption: x22.png

## Peer Reviews

confidence: 5
contribution: 1
initial_rating: 0
presentation: 1
questions: I have no additional question. In weakness section, I have detailed the issues.
soundness: 1
strengths: The only strength of the article, in my opinion, is that it attempts to address a very timely problem is neural net.
summary: The work introduces a structured pruning approach for neural nets, that uses projective space. This attempts to address some of the shortcomings of magnitude based pruning. It projects the filters on latent space and then notices its movement during gradient descent. An importance score is then computed leveraging the movement information, which in turn forms the basis for pruning the filters.
weaknesses: The papers has several major weaknesses. 

1. First and foremost, the description of the proposed method is very poor. It is very difficult to understand what is going on. I do not think a reader would be able to implement the algorithm from the description given in the paper. Specifically, I do not find the following critical information. 
  A) When does it stop taking the filters out from the net? 
  B) What happens if all filters are removed from a layer and how does the approach handle layer collapse? 
  C) Does it need pre-trained model always? 

2. Projective geometry seems the key idea of the paper. Yet, there is hardly any clarity in the paper why and how does it help. The description seems too superficial and cursory. 

3. Algorithm 1 does not add any value, in my opinion. Rather the authors should use the space to better justify why and how of projective geometry. 

4. Results: Finally, the results are nowhere close to the state of the art. For example, on CIFAR 10, CURL (Neural network pruning with residual-connections and limited-data, 2020), SPvR (SPvR: Structured Pruning via Ranking , 2025), Hrank ( Hrank:Filter pruning using high-rank feature map, 2020), OTOv2 achieves the same performance as the proposed method with 40% (absolute) lesser parameters. Overall, I think the proposed approach uses much more parameters than the state of the art models to achieve the same performance.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 1
questions: If the proposed method does work better than other pruning strategies, it looks like it is due to the denominator in (4) which penalises certain behaviours. Which behaviours and how? What is the role of $\lambda$ (and how is it chosen)? What is the intuition behind $\frac{\partial\mathcal{L}}{\partial D_i}$? What rate of change does it measure?
soundness: 2
strengths: - Based on the empirical section of the paper, the method clearly works very well.
- The experiments and results are convincing and well put together.
- The scoring function (4) is interesting.
summary: The authors develop an architecture-agnostic pruning algorithm (although it is not entirely clear how it would work on a simple MLP) that is based on ideas from projective geometry. Concretely, the method uses a score called PROscore (defined in Eq (4)) involving a modified version of the model under consideration that includes some fresh parameters (parameter injection, the meaning of which isn’t entirely clear) and the gradient of the loss function of this modified model.

The authors then explain how to use this score function for pruning in the case of CNNs, vision transformers and LLMs and proceed to show that the method works well in a very wide range of experiments.
weaknesses: - The paper starts from the following axiom: when it comes to pruning a NN, the idea that the magnitude of the parameters of the NN is important is a myth (143-145), and “magnitude-invariant” methods must be developed. There is a mathematical and a motivational problem with this perspective:
	- Mathematical: What operation are involved in a forward pass through a NN? Lots of additions and multiplications, ReLU and softmax to name the most important. Addition, multiplication by positive numbers, ReLU and softmax are isotone in their arguments, multiplication by negative numbers is antitone; in particular they are all monotone. Therefore, inputs with bigger magnitudes (i.e. absolute values) produce outputs with bigger magnitudes. Thus magnitude matters at a very fundamental level in any NN. The onus is therefore on the authors to substantiate their claim that the importance of magnitude is a myth on the face of this very basic mathematical fact. Which brings me to the second point,
	- Motivational: one cannot do science by simply stating that a certain way of doing things is a myth and taking another approach. If magnitude is not, or less, important than one might think, this must be documented and explained. This paper doesn't do either.
- Despite the claim that the proposed method is “magnitude-invariant” (a term that is never defined precisely), the proposed solution is very much magnitude dependent. The expression (4) -- which has a typo, it should $\lVert F_i\rVert$ in the denominator, not $D_i$ -- is large when the numerator $\lVert F_i-\lambda\nabla_{F_i}\mathcal{L} \rVert$ is large, i.e. the updated $F_i$ with learning rate $\lambda$ has a large magnitude, and the denominator $\lvert \lVert F_i\rVert-\lambda\frac{\partial\mathcal{L}}{\partial D_i} \rvert$ is small (the interpretation of this term is more complicated, see below). In which sense is this magnitude-invariant?
- The expression (4) could have been written without any reference to projective space and projective geometry. It really plays no role in this story. Eq (4) provides a gradient-based method with an unusual denominator, and this denominator is the heart of the story. 
- I wish more time and care had been spent on this denominator $\lvert \lVert F_i\rVert-\lambda\frac{\partial\mathcal{L}}{\partial D_i} \rvert$. To start to understand it one must jump to the next section to find the definition of $\frac{\partial\mathcal{L}}{\partial D_i}$ (this is not a great way to present things…). But this is not enough to understand the meaning of this term. What exactly is $\psi$ in (5)? What does “modifying element-wise computation layer $\sigma$” mean? If $\sigma(x)$ returns a tuple of dimension N and $x$ is a tuple of dimension $M\neq N$ how are we supposed to understand (5)? Since $\frac{\partial\psi}{\partial D_i}=x$ (and the second derivative will therefore vanish), what can we say about the general shape of $\frac{\partial\mathcal{L}}{\partial D_i}$ from the chain rule? And what does this mean for the proposed method and the meaning of the denominator of (4)?
- The paper is littered with grammatical errors, making it quite hard to read in some places.

---

confidence: 4
contribution: 2
initial_rating: 6
presentation: 4
questions: See the weaknesses.
soundness: 4
strengths: **1. Clear and principled formulation of scale invariance through projective geometry**

The paper formalizes magnitude-independent pruning not as heuristic normalization but as a property of the underlying space. By embedding filters into a projective space, the method guarantees scale invariance at the definition level, providing a clean and mathematically grounded justification for direction-based importance.

**2. Unified pruning framework applicable across architectures**

The same projective formulation applies consistently to CNNs, Vision Transformers, and large language models. This unification makes the approach architecture-agnostic and highlights that the proposed importance measure is not tied to a specific model design or normalization scheme.

**3. Comprehensive empirical validation across diverse tasks**

Experiments cover image classification, semantic segmentation, and language modeling, using models such as ResNet-50, DeepLabV3, DeiT, and LLaMA-2-7B. The breadth of evaluation supports the claim that IPPRO is a general framework rather than a task-specific trick.

**4. Robustness and fine-tuning-free performance**

IPPRO maintains accuracy even when computed with limited data sampling and performs competitively without any fine-tuning after pruning. This property makes the method practical for large-scale models where retraining is expensive or infeasible.

**5. High clarity and reproducibility**

The paper is well-organized, with precise notation, clear pseudo-code, and detailed ablations that enhance transparency. The paper’s presentation quality and completeness make reproduction straightforward and strengthen the empirical credibility of the results.
summary: This paper proposes IPPRO (Importance-based Pruning with PRojective Offset), a structured pruning framework that overcomes the limitations of magnitude-based pruning by focusing on directional importance. The method embeds each filter into a projective space, where importance depends on direction rather than scale, ensuring scale-invariant comparison across layers. A new PROscore measures how much each filter’s direction changes under a gradient step, and filters that move closer to the origin are pruned. A simple parameter injection trick allows this computation without affecting model outputs. Experiments on CNNs, Vision Transformers, and LLMs show that IPPRO achieves slightly better accuracy and greater stability than previous direction-based pruning methods even without fine-tuning.
weaknesses: **1. Direction-based pruning has been extensively explored in prior work**

Several recent methods such as Torque, Catalyst, and geometric pruning already focus on gradient direction rather than magnitude. IPPRO provides a cleaner mathematical reformulation but does not introduce a fundamentally new optimization insight.

**2. Limited theoretical gain from adopting projective geometry**

Although the paper frames pruning in the language of projective geometry, the practical effect largely reduces to normalizing vectors and measuring angular displacement. The framework adds elegant terminology but yields little new theoretical understanding beyond existing direction-based normalization schemes.

**3. Increased computational overhead with limited performance improvement**

Computing PROscores requires additional gradient accumulation and parameter injection, which significantly increases pruning cost. However, the resulting accuracy gain over previous direction-based methods is minimal, raising concerns about the efficiency–benefit trade-off.

**4. Unclear behavior in hybrid architectures without manual layer-wise control**

While the method is claimed to be unified across CNNs and Transformers, it is unclear how well the approach generalizes to mixed architectures such as ConViT or hybrid CNN–ViT models when global pruning is applied without manually setting layer-wise ratios. This raises questions about the true level of architectural unification achieved by the framework.

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 3
questions: see weaknesses
soundness: 2
strengths: 1. The projective geometry framework provides a principled alternative to magnitude-based heuristics. 

2. Testing on 7+ architectures (ResNet, MobileNet, DeiT, EfficientFormer, DeepLabV3, LLaMA).

3. Robust performance without fine-tuning.
summary: This paper proposes IPPRO (Importance-based Pruning with PRojective Offset), a magnitude-indifferent structured pruning method for neural networks. The key innovation lies in using projective geometry to evaluate filter importance through angular displacement under gradient descent, rather than relying on filter magnitudes. 

The authors introduce PROscore, computed as the tangent of angular distance in projective space after one gradient descent step, as the importance criterion. Extensive experiments demonstrate IPPRO's effectiveness across CNNs, Vision Transformers, and LLMs, showing consistent performance improvements especially in high compression scenarios and without fine-tuning.
weaknesses: 1. Why should angular distance in projective space correspond to filter importance? Is there a theoretical justification that angular distance is better than geometric distance or norm? A formal theorem would strengthen the contribution. It would also be important to theoretically justify why angular distance is superior, rather than merely demonstrating that it performs better.

2. No wall-clock time comparisons with baselines. This is important for pruning papers.

3. The performance improvement is marginal. For example, in Table 2 (Cityscapes – DeepLabV3-ResNet50), when comparing IPPRO (ours) with SIRFP (Wu et al., 2025), the improvement in mIoU is very small. Under the base pruning setting, SIRFP achieves 81.3 mIoU, while IPPRO reaches 81.5 mIoU, giving only a +0.2 gain. In terms of FLOPs reduction, IPPRO obtains 61.8 %, just 0.5 % higher than SIRFP’s 61.3 %. This slight edge shows that the performance improvement is marginal, i.e., the proposed method performs almost on par with the previous state-of-the-art under the same compression ratio.

4. No analysis on modern architectures (e.g., Swin Transformer). 

5. Experiments on Imagenet should be put in the main text, not supplementary.


Decision: Reject