RCPU: Rotation-Constrained Error Compensation for Structured Pruning of Large Language Models
Authors: Shuichiro Haruta
Kazunori Matsumoto
Zhi Li
Yanan Wang
Mori Kurokawa
Venue: ICLR 2026 Poster (2026)
Keywords: LLM
Compression
Pruning
Primary area: foundation or frontier models, including LLMs

Abstract: In this paper, we propose a rotation-constrained compensation method to address the errors introduced by structured pruning of large language models (LLMs). 
LLMs are trained on massive datasets and accumulate rich semantic knowledge in their representation space. 
In contrast, pruning is typically carried out with only a small amount of calibration data, which makes output mismatches unavoidable. 
Although direct least-squares fitting can reduce such errors, it tends to overfit to the limited calibration set, destructively modifying pretrained weights. 
To overcome this difficulty, we update the pruned parameters under a rotation constraint. 
This constrained update preserves the geometry of output representations (i.e., norms and inner products) and simultaneously re-aligns the pruned subspace with the original outputs.
Furthermore, in rotation-constrained compensation, removing components that strongly contribute to the principal directions of the output makes error recovery difficult. 
Since input dimensions with large variance strongly affect these principal directions, we design a variance-aware importance score that ensures such dimensions are preferentially kept in the pruned model. 
By combining this scoring rule with rotation-constrained updates, the proposed method effectively compensates errors while retaining the components likely to be more important in a geometry-preserving manner.
In the experiments, we apply the proposed method to Llama-7B and Llama-2-13B, and evaluate it on WikiText2 and multiple language understanding benchmarks. 
The results demonstrate consistently better perplexity and task accuracy compared with existing baselines.

Published as a conference paper at ICLR 2026
RCPU: ROTATION-CONSTRAINED ERROR COMPEN- SATION FOR STRUCTURED PRUNING OF LARGE LAN- GUAGE MODELS
Shuichiro Haruta, Kazunori Matsumoto, Zhi Li, Yanan Wang, & Mori Kurokawa AI Division, KDDI Research, Inc. 2-1-15 Ohara, Fujimino-shi, Saitama 356-8502, Japan aarraar mz mz-mat zh-li, wa-yanan, mo-kurokawa @kkdii com
ABSTRACT
In this paper, we propose a rotation-constrained compensation method to address the errors introduced by structured pruning of large language models (LLMs). LLMs are trained on massive datasets and accumulate rich semantic knowledge in their representation space. In contrast, pruning is typically carried out with only a small amount of calibration data, which makes output mismatches unavoid- able. Although direct least-squares fitting can reduce such errors, it tends fit to the limited calibration set, destructively modifying pretrained weights. To overcome this difficulty, we update the pruned parameters under rotation con- straint. This constrained update preserves the geometry of output representations (i.e., norms and inner products) and simultaneously re-aligns the pruned subspace compensation, re-
with the original outputs. Furthermore, in otaaton-constraingg moving components that strongly contribute to the principal directions of the out- put makes error recovery difficult. Since input dimensions with large variance strongly affect these principal directions, we design a variance-awarr importance score that ensures such dimensions preferentially kept in the pruned model. By combining this scoring rule with rotation-constrained updates, the proposed method effectively compensates errors while retaining the components likely be more important in a geometry--reserving manner. In the experiments, we ap- ply the proposed method to Llama-7B and Llama-2-13B, and evaluate it on Wiki- Text2 and multiple language understanding benchmarks. The results demonstrate consistently better perplexity and task accuracy compared with existing baselines.
INTRODUCTION
Large language models (LLMs) are driving a rapid wave of transformation and are now being de- ployed across wide range of applications, including code assistance, conversational agents, search and summarization, agentic execution, and content generation Jiang et al.) (2025); Fan et al.] (2024); Zhang al. (2025): Wang al, (2024). At the same time, their inference costs computation and memory remain substantial, creating significant bottlenecks for deployment. In mobile and embed- ded settings, there is a strong need for model compression techniques that reduce computational cost while preserving task performance Saha & Xu (2025); Girija et al.) (2025). Against this backdrop, variety of efficient methods have been explored, such quantization, knowledge distillation, and pruning Miao et al. (2025). Among them, structured pruning typically removes parameters at the granularity of weight-matie rows columns, and in some cases even entire transformer blocks Kim et al. 2 Such structured removal directly reduces the parameter count and thereby lowers memory usage and inference cost He & Xiao (2024).
Early work targeting LLMs, such as LLMM-runer, demonstrated the feasibility of compression but tends to rely on downstream fine-tuning to recover high accuracy Ma et al. (2023). Therefore, methods that preserve accuracy without re-training are desirable. WANDA is widely used for un- structured sparsification, and prunes using simple activation-ane aware heuristics with only a small cal- ibration without downstream fine-tuning Sun et al. (2024). Its structured variant, Wanda-sp, provides a simple column-pruminn importance score and serves as a familiar baseline in structured pruning An et al. (204). Yet, removing columns inevitably introduces output discrepancies. In

---

Published as a conference paper at ICLR 2026
W
WK
aum 100
X Calibration data
Column Pruning Score
Whil X y. (ar(X (.l)
Original Model Output
Q-Constrained Fit
WX
min Q Q-1
QZ
Pruned Model Output Wx.x
Wx Q'Wx
Figure 1: Overview of RCPU framework. Input activations are scored by variance-aware impor- tance score, less important columns are pruned, and the retained subspace is updated through the rotation-constrained fitting. The pruned output vectors (arrows) are rotated to align with the original output vectors, showing how RCPU compensates for pruning errors.
real deployments, the data available for calibration is often limited. Thus, pruning decisions must be made under sparse observations, and how errors are handled becomes decisive for overall per- formance. Recent work, FL.AP, has shown that compensating the mean component of post-pruning errors with a bias term can be practical and effective An et al. (2024). Nonetheless, input-dependent directional mismatches are not easily addressed by a constant bias. As another possible approach, one might consider least-squares style fitting that directly minimizes output error. However, such a broad parameter update, under limited calibration data, risks overfitting that damages knowledge acquired during pretraining even if using regularization method like Ridge Hoerl & Kennard (970)..
Motivated by these limitations, in this paper, we propose RCPU, a Rotation Rotation-Constraince Constrained Parameter Update, to reduce pruning error while preserving the norm and inner-product structure of output representations. Figure [ shows the overview RCPU. Compared to general linear least-squares updates, restricting the update rotations preserves angles and lengths, which helps avoid geometric distortion under small calibration sets. We formulate the alignment between the retained outputs and the original outputs as an Orthogonal Procrustes problem and, for each layer, estimate the optimal rotation and use it to update the parameters. The constraint reduces the update's 's degrees of freedom, which improves statistical stability and makes less prone to overfitting. Moreover, since the choice of retained components strongly affects the effectiveness of rotation-constrained compensation, we adopt simple pruning score that augments weight magnitude and input scale with input variance. By doing this, components contributing to principal output directions are preferentially kept. We combine this scoring rule with ootation-constraing errors while retaining the components likely to be more important in geometry preserving manner. In the experiments, we apply RCPU to existing LLMs and evaluate on a variety of language understanding benchmarks. As result, we demonstrate improvements over existing baselines in both perplexity and task accuracy. The main contributions of this paper are as follows:
updates, and RCPU effectively compensates
seomeetr--peeervvngg
We formulate the compensation method via orthogonal rotation applied immediately af- ter column pruning, and combine with a simple pruning score that incorporates input variability. We show improvements over existing baselines in post-pruning evaluation.
The compensation can be inserted directly after Wanda-sp style column pruming, requiring no additional model modifications. requires no extra architectural changes and adds only modest computation.
2

---

Published as a conference paper at ICLR 2026
2 PROBLEM FORMULATION
2.1 NOTATION AND SETUP
We consider linear sub-layer in transformer block (e.g., attention output projection multi layer perceptron (MLP) down-projection) with weight matrix W E Rdoo From small calibration set of N token positions, we record input activations X RdaxN and the corresponding original outputs Y WX ε X
We focus structured pruning methods that drop entire columns of the weight matrix, i.e., column pruning. Structured column pruning selects binary mask m E {0, 1 and keeps the index set K = [j my 1} with |K] = d', while D = (1..... dnn \ K denotes the dropped indices.
Using and D, we define WK K == -WEE E Rdd WD == WI.DD R (dh.-") XK ==
X(K.) IRd' and XD XJD.:] [dd,--d)xN, where W [[.K]] denotes selecting the columns of W indexed by K, and XJK.] denotes selecting the rows of X indexed by K. Similarly, D selects the dropped indices. Then the original output decomposes
Y
WX
WKXK
WDXD
kept
dropped
After pruning, the post-pruning output is
Z W WRXK
RdmxXN
To compensate the discrepancy without using dropped columns, one possible approach to update the kept parameters. Concretely, the following regularized optimization problem can be considered:
L(W*) = Y W + A||W* Wkll?
(3)
where is regularization hyyer-paramees This corresponds to Ridge regression, in which the updated weights are penalized for deviating from the original ones.
2.2 LEAST-SQUARES COMPENSATION
A straightforward way to minimize the eтror defined in equation 3 is to apply a least-squares fitting. The closed-forn solution is
W = (YX) + (.WR)
(4)
(i) Geometric distortion: an unconstrained linear fit may introduce scaling and shear that reduce in-sample error while altering angles and norms the output space, which can harm generalization beyond the calibration set. Desideratum: preserve the geom- etry of the outputs as much as possible. (ii) Limited effectiveness of regularization: The À values that minimize calibration perplexity do not stabilize the estimator and, in our experiments, often degrade downstream performance. Desideratum: ensure stability in way that does not depend on regularization tuned purely for calibration set.
Limitations under limited calibration.
These issues motivate restricting the compensation update to geometry preeerrvineeerriggggg transformations with limited flexibility, while ensuring that the update operates only within the kept subspace.
RCPU (ROTATION CONSTRAINED PARAMETER UPDATE)
We target the eттоr introduced by structured column pruning in linear sub-layers of transformer block. After pruning, the kept subspace still carries most of the signal, yet its orientation relative to the original outputs can be misaligned. Our idea is to re-align the orientation by rotation- constrained parameter update computed from small calibration set. Restricting the update to rota- tions preserves norm and inner- inner-product relationships of output representations, helping reduce error while maintaining the pretrained geometry.
This compensation is more effective when the dropped columns do not dominate principal out- put directions. We therefore combine the rotation with a variance-aware importance score that avoids dropping columns likely contribute those directions. Concretely, extend com- mon magnitude-ane preferentially keeps columns which seem to be relevant for orientation recovery.
heuristic with an input-variance factor, yielding a simple score that
3

---

Published as a conference paper at ICLR 2026
3.1 ROTATION-BASH BASED COMPENSATION VIA ORTHOGONAL PROCRUSTES
Given (X, Y) and pruning mask K, we form Z = W K Xxx x as in equation 2
Optimization problem. We seek a rotation matrix Q Rd d that aligns the kept output the original output on calibration data:
Q' arg min
|| Q QQQ=I
OZI
(5)
Equation is known as the classical Orthogonal Procrustes problem Golub & Van Loan (2013). 3). It corresponds to the least-squares formulation in equation 3 and equation but with the compensation update restricted to orthogonal matrix.
Closed-form solution. Let M = YZT and take its singular value decomposition M = UEVT where U and V are orthogonal matrices whose columns give the left and right singular vectors and E is a diagonal matrix containing the singular values of M. Then the minimizer of equation 5 is
given by
Q UV.
We update only the kept parameters by rotation
Wx Q'W K.
other words, applying the update makes the new output W = Q*Z. Thus, the kept compo- nent Z is explicitly rotated by Q" so that its orientation matches the original output Y. Finally, we replace the sub-layer weight with the compact matrix W == WW K E Rd=x meaning that columns in D are physically removed. Note that we follow common practice and simply refer to Q as "rotation" throughout the paper, even though it may occasionally include reflection.
Scaled variant. As a natural extension of the rotation-only solution, we can introduce single isotropic scaling factor. Although the benefit expected to be limited, this variant is intuitively reasonable: it preserves the angular structure and norm ratios of the outputs, while also allowing the overall magnitude to be better matched to the original model.
Formally, the optimization problem is defined
(Q", arg
min
Y - QZ [2
(8)
Q
Q=1,
With M YZ UEVT
tr Σ
Q' =V
(9)
Z
and we set WK = s* SQ' K- This variant rescales all vectors by common factor s* > 0. The ordering of vector lengths (within the same set) is invariant.
Geometric intuition. Pruming removes the W WDXD term in equation I but the kept subspace often still captures much the useful signal. By restricting the update to rotation (with an optional isotropic scaling), the retained subspace be re-aligned with the original output geometry while preserving angles and relative norms. This avoids the arbitrary scaling and shear distortions that the least-squares fit may introduce under limited calibration.
3.2 VARIANCE-AWARE COLUMN SCORING
To fully exploit rotation-constrained compensation, it is important to retain columns that preserve strong directional information. We therefore assign each input column ý with
(10) The variance term emphasizes columns whose activations fluctuate across calibration tokens, which more likely align with dominant output directions. The weight and input norms further bias the
W Ball p.al Var(X b.a)-
4

---

Published as a conference paper at ICLR 2026
Algorithm Layerwise post-pruning orientation compensation with variance-aware selection
Require:
Calibration tokens; target sub-layers S; pruning ratio p
1: for each transformer layer and each sub-layer s E S
2: Collect: record calibration activations X and original outputs Y
3: Score and select columns:
Compute scores 7/ = [. bلaا (a1)
Keep top-k indices K where k = din [dmpp
Define dropped indices D (1, ( din) K
4: Form reduced matrices:
X [K WK Wi.K]. and Z W xXK
5:
A lignn kept subspace:
Solve equation 5 (or equation () for Q" (and s*)
Update kept weights: W K Q' W к (or s Q' R)
6:
Finalize: replace the weight by W WK and remove columns in D
7:
end for
score toward columns with inherently larger contributions. This formulation is a natural extension of the WANDA-sp score, which uses only the product of weight and input norms and omits the variance factor. By incorporating variance, our scoring favors columns that not only have large magnitude but also actively contribute under diverse inputs.
Let p E [0, 1) be the pruning ratio for a sub-layer with input width dig we prune [dmp]] columns and keep k dm [dmp] indices with the largest Yy.
3.3 ALGORITHM AND COMPLEXITY
We apply the procedure layerwise to a designated set of linear sub-layers. Algorithm | summarizes the steps. This procedure greedy and layerwise: each layer Procrustes subproblem admits closed-form minimizer ( equation 6 and equation 9 but the overall routine is not joint global optimization across the network.
Complexity. Per treated sub-layer, computing scores takes O(din + N)), forming Z costs (No and constructing M YZ" requires (V The dominant cost is the SVD of M E geduxxda which is typically (d Thus the overall complexity cubic in dout. on par with unstructured pruning methods such as SparseGPT Frantar & Alistarh (023)
4
EXPERIMENTAL RESULTS
4.1 SETTINGS
We evaluate RCPU on Llama-7B and Llama-2-13B as the base models Touvron et al. 2022a b As baselines, we use WANDA-sp. -p, a pruning method based on weight magnitude and activation scale; and FLAP, bias-based eтror compensation method. In addition, we compare RCPU with SliceGPT. Since SliceGPT does not support Llama-1-7B, we evaluate it on Llama-2-1333 2-3BB with benchmarks.
modeessTouvron
Following prior work, we use WikiText-2 the calibration dataset Merity et al. (206)). We perform pruning based on the input channels of o. o_proj and down_proy _poj_.pooo.o......o and simultaneously remove the corre- sponding positions the other projection matrices. For the attention modules, we prune at the head level. Parameter updates, however, are applied only to o_proj and down_proj. We evaluate pruning ratios of 10%, 20%, and 30%.
Evaluation metrics follow prior studies. We report perplexity (PPL) WikiText-2, as well as curacy on a suite language understanding benchmarks: BoolQ. PIQA, HellaSwag, WinoGrande, ARC-easy, ARC-challenge, and OpenBookQA Clark al. 2019); Bisk et al. (2020): Zellers et al.
Coodes available at httc:a////gghibb..cc///ttttaaa/a

---

Published as a conference paper at ICLR 2026
Pruning
Pruning Ratio
Pruning Ralio 0.3
PPL (log scale)
32
128
256
Noxta (log scale)
Figure 2: Perplexity versus calibration-set size for RCPU on Llama-7B.
Table 1: Representative PPL (4) of WANDA-sp. FLAP, and RCPU under Nealib 128, 512. The
best score in each setting in bold, while the second-best score is underlined. See also Figure
Llama-7B
Llama-2-13B
Method
PR
128
512
128
512
Original
0%
12.4
12.4
10.98
10.98
WANDA-sp
10%
14.66
14.53
12.29
12.29
FLAP
10%
14.14
14.11
12.22
12.08
RCPU (Rot.)
10%
13.55
13.48
11.65
11.57
RCPU (Rot++ccale)
10%
13.52
13.45
11.66
11.57
WANDA-sp
20%
16.70
16.96
14.62
14.62
FLAP
20%
15.36
15.07
14.49
14.14
RCPU (Rot.)
20%
14.40
14.83
13.12
12.75
RCPU(RoL.+Sclee)
20%
14.55
14.81
13.07
12.72
WANDA-sp
30%
24.13
26.20
61.66
63.35
FLAP
30%
18.59
18.31
17.15
16.71
RCPU (Rot.)
30%
18.35
16.99
16.99
16.01
RCPU (Rot+Sccale)
30%
18.21
16.91
16.88
15.96
Sakaguchi et al. (200)); Clark et al. (2018): Mihaylov et al. (0118). These benchmarks cover diverse domains and reasoning types, enabling us to evaluate the model's performance comprehensive manner. For evaluation, we adopt the Language Model Evaluation Harness Gao et al. (2024). All experiments were conducted on a single NVIDIA A100 GPU with 80GB memory.
2019
4.2 PERPLEXITY
First of all, we conducted experiments regarding PPL, which changes Nealib (The number of cal- ibration samples). Figure reports how the PPL of RCPU varies with the number of calibration samples. We observe that PPL drops rapidly as Nealib increases and becomes roughly stable once Nealb reaches around 64. Based on this trend, we adopt Nealib 128 and 512 the calibration set sizes for other experiments. While 128 is a common choice prior work, we additionally include 512, which lies well within the empirically stable region, providing more reliable evaluations.
Table summarizes PPL on WikiText-2 across pruning ratios. As we can see from Table I RCPU consistently improves upon WANDA-sp and performs better than FLAP regardless of the number of calibration samples and models. These results show that rotation-constraine competitive with bias-based compensation. While the advantage is not uniform at all pruning levels, the gcometry-preserring preserving nature of rotational transformations helps prevent the distortions that often arise from unconstrained updates. We also compare the scaled variant (Rot.+Scale) plain rotation. The results show only marginal differences, indicating that introducing a global rescaling does not significantly alter PPI.. We believe this is because the rotation already aligns the retained subspace
updates can be

---

Published as a conference paper at ICLR 2026
WANDA-sp compensation
29
Proposed compensation
WANDA-p LS compensation
27-
Proposed score LS compensation
Perplex erplexity (PPL)
25
WA RCPU compensation
Perpley perplexity (PPL)
25
Proposed score RCAU compensation
23
23-
21
21
15
13
11
11
10%
10%
20% Pruning ratio
Pruning ratio
(a) WANDA-sp score+ comp. (Nealib =128)
(b) Proposed score + compensation (Neatib = -128
sp compensation
29
Proposed scare campensation
WA LS compensation
27
Proposed LS compensation
Perplexity (PPL)
25
WANDA sp ACPU compensation
Verplexity (PaL)
25
Proposed score RCPU compensation
23
23
17
15
13
13
11
11
10%
20%
10%
20% Pruning ratio
Pruning ratio
(c) WANDA-sp score+ comp. (Nealib =512)
(d) Proposed score compensation (Neatib =512)
Figure 3: PPL P.R. ratio for different calibration sizes and compensation methods Llama-7B.
with the dominant output directions, effectively discarding less informative components. As a result, the overall output norm is well preserved, and an additional scaling factor brings limited benefit.
Next, in Figure we compare the compensation effectiveness of RCPU and LS (Least Siuge regularization (equation 4. We performed grid search over À (10-6, 1:0-6,10-600000000 108} and reported the result on best À (See Section 7.1). Figure 3 focuses on the effect of different error-compensation methods (LS or Rot.) in different calibration sizes. From Fig- ure 3aladdd and Figure 3b) we observe that with Nealih = 128, the proposed rotation- based compensation achieves the best PPL. In Figure Bal the regularized LS even worsens PPL, reflecting its tendency to overfit under limited calibration. In contrast, Figure 3c and Figure show that when Neallb 512, both rotation and Ridge-based compensation effectively reduce PPL. but the regularized LS update contributes more strongly to the improvement. Similar trend is observed in Llama-2- 13BB as shown Appendix 7.2) Although is intuitively expected that the effect of the least square fitting becomes larger as the number of calibration samples increases, emphasize that this does not necessarily translate into better downstream benchmark performance. Indeed, for example Table B in Ap- pendix, LS L.S-based compensations accuracy. According to Hastie et al.] (2001), the degree of freedom in ridge regression is computed by dout E: o7+x where denotes the singular value of the input matrix. Using this equation and the best X, obtain the degree of freedom values in the range 1.395 10 to 1.578 10 across each pruning ratio. contrast, the degree of freedom in RCPU is given by donr since Q constrained to the orthogonal matrix. Its degree of freedom 5.36 10 which is smaller than that of LSdRidge. This implies that, from the standpoint of preserving the pretrained knowledge of LLMs, the rotation- based compensation tends to be more robust. We also highlight that, in the context of LLMs, selecting appropriate regularization syper--arameter À can be computationally expensive, as it requires repeatedly computing large matrix inverses for multiple candidate values of X. In contrast, method has parametrrs, avoiding this overhead.
not the top performer, whereas RCPU often achieves the best

---

Published as a conference paper at ICLR 2026
Table Zero-shot accuracy (1) Benchmark datasets when Nealib 128 on Llama-7B, The best score in each setting is highlighted in bold, while the second-best score is underlined.
Method
P.R.
BoolQ
PIQA
Hella
Wino
ARC-e
ARC-c
OBQA
Mean
Llama-7B (Orig.)
0%
75.10
78.67
76.18
70.01
72.85
44.79
44.40
66.00
FLAP
10%
73.33
77.37
72.81
68.59
70.54
41.13
42.80
63.80
WANDA-Sp
10%
75.17
76.82
74.43
67.01
69.53
43.43
39.60
63.71
moo.Score+LSS (Abest)
10%
72.75
75.19
71.03
68.19
64.65
39.93
39.60
61.62
RCPU (Rot.)
10%
74.89
76.71
74.20
69.53
69.99
42.32
40.40
64.01
RCPU (Rot.+Scale)
10%
74.22
76.44
74.33
69.69
69.99
42.15
40.20
63.86
FLAP
20%
70.45
74.91
67.42
67.64
66.79
39.00
43.00
61.32
WANDA-Sp
20%
69.57
75.08
69.88
65.74
65.95
40.44
38.80
60.78
nop.Score+LLS (Abest)
20%
67.83
71.71
64.40
64.56
58.71
34.81
35.80
56.83
RCPU (Rot.)
20%
71.50
74.81
70.32
68.43
66.58
39.93
39.40
61.57
RCPU (Rot++ccale)
20%
71.77
74.97
70.25
68.03
66.71
39.93
38.80
61.49
FLAP
30%
66.67
71.49
59.53
61.56
59.76
34.81
39.60
56.20
WANDA-sp
30%
65.29
67.74
58.09
59.12
58.63
34.56
34.60
54.00
Prop.Score+LS (Abest)
30%
62.54
67.08
55.55
60.77
50.13
31.06
35.00
51.73
RCPU (Rot.)
30%
61.25
70.46
62.76
62.67
58.59
34.98
37.80
55.50
RCPU SRote+
30%
65.14
70.46
62.95
64.25
58.21
34.98
38.40
56.34
Table Zero-shot accuracy (f) on Benchmark datasets when Nealib 512 on Llama-2-11
Method
P.R.
BoolQ
PIQA
Hella
Wino
ARC-e
ARC-e
OBQA
Mean
Llama2-13B (Orig.)
0%
80.55
79.05
79.37
72.14
77.44
49.06
45.20
68.97
FLAP
10%
74.22
78.56
76.12
71.11
74.07
44.54
45.20
66.26
SliceGPT
10%
62.84
77.09
71.80
71.59
76.35
49.40
45.20
64.90
WANDA-Sp
10%
79.14
78.02
77.99
70.64
75.76
48.12
44.80
67.78
Soo.Secoe++LS (Ahest)
10%
78.78
77.69
77.56
72.30
73.99
47.53
43.40
67.32
RCPU (Rot.)
10%
79.82
78.13
78.09
72.45
75.00
47.78
44.40
67.95
RCPU(Rot.+Scale)
10%
79.79
78.29
78.14
72.69
75.04
47.78
44.60
68.05
FLAAP
20%
67.00
74.97
70.41
68.19
67.09
40.78
43.20
61.66
SliceGPT
20%
52.20
71.76
63.17
67.32
70.45
43.77
41.80
58.64
WANDA-Sp
20%
72.78
76.61
73.32
69.46
72.39
44.80
41.80
64.45
Prop.Score+LS (Abest)
20%
73.12
76.28
73.02
69.93
70.50
43.86
41.20
63.99
RCPU (Rot.)
20%
73.76
76.44
73.91
70.09
71.25
44.20
42.20
64.55
RCPU Scot.+
20%
73.30
76.33
73.95
69.46
71.21
43.34
41.80
64.20
FLAP
30%
65.78
72.14
64.57
64.25
62.71
38.91
40.20
58.37
SliceGPT
30%
38.35
66.10
52.64
66.38
56.78
35.15
40.00
50.77
WANDA-sp
30%
61.99
62.68
36.09
51.07
41.54
25.60
28.80
43.97
Soo.Score+LLS (Abest)
30%
66.61
72.96
64.62
65.04
66.62
36.77
40.40
59.00
RCPU (Rot.)
30%
64.37
73.72
66.22
64.88
67.05
38.31
42.80
59.62
RCPU (Rot++ccale)
30%
65.17
73.67
66.69
64.88
67.30
38.23
42.60
59.79
4.3 BENCHMARK
Table and Table reports accuracy seven language understanding benchmarks on Llama-7B and 1lama-2-13B. Overall, performance degrades as the pruning ratio increases. RCPU achieves higher mean accuracy than FLAP across all pruning levels, indicating that geometry-preserving compensation can be more effective than bias-only correction. Comparing Table with Table the scaled variant performed better and ranked the best baseline more often in the 5 512-sample setting than in the 28-sample setting. Intuitively, the additional samples stabilize the norm statistics the unpruned versus pruned outputs, allowing the global scale s* more effectively restore the original
8

---

Published as a conference paper at ICLR 2026
Table 4: PPL under different pruning ratios and compensation targets (Llama-7B, Neatib 128).
Pruning ratio
No compensation
o_proj only
down.proj only
Both updated
10%
13.96
13.61
13.62
13.55
20%
16.85
15.47
15.76
14.40
30%
21.94
18.91
20.22
18.35
Table 5: Size of the pruned model and time required for pruning (Llama-7B).
Pruning ratio
# of parameters
Memory size
Time required for pruning layer
0% (FP32)
6.73B
25,705MiB
10%
6. 6.10B
23,295MiB
8.36s
20%
5,47B
20,875MiB
8.90s
30%
4.84B
18,456MiB
9.20s
magnitude. Regarding Table SliceGPT transforms the entire model into an equivalent structure using orthogonal matrices, and then performs or column deletion in single global step. Due to this property of applying a global transformation followed by global pruning, we expect that, at high pruning ratios, a mismatch arises between the input distributions assumed by each layer and the actual input distributions after pruning. This mismatch is also expected to accumulate across layers and thus we believe accuracy tends degrade high pruning ratios. In contrast, RCPU optimizes the pruning induced error layer by layer. Thus, even at high pruning ratios, each layer more likely to maintain representations close the inputs it assumes, which we believe leads to better benchmark performance. Looking at individual tasks, HellaSwag and WinoGrande show relatively stronger performance with RCPU. These tasks require contextual consistency and pronoun resolution, both of which are sensitive to orientation shifts in the representation space. The benefit observed here aligns with the perplexity improvements reported earlier, suggesting that rotation- constrained updates help preserve the structural properties of output representations that underlie these tasks.
4.4 ANALYSIS
Where to apply rotation? In order clarify which parts effective apply RCPU, ducted ablation study that changes the module to apply the proposed compensation. Table/Arepone PPL when rotation-based compensation is applied to different projection sub-layers. First, applying compensation to either o-proj or down-proj alone improves PPL compared to the uncompensated baseline. This indicates that the kept subspace indeed contains recoverable signal, and aligning it to the original outputs partially restores the lost information. Second, updating o o.proj is consistently more effective than updating nlow.ppro.. A plausible explanation lies the forward order of com- putation in transformer blocks: attention is followed by the MLP. Misalignment at o. o_proj propagates directly into the subsequent MLP input, thereby amplifying its negative effect. Correcting the orien- tation earlier at o_proj provides the MLP with already aligned features, reducing the burden of later layers. In contrast, compensating only down_proj cannot undo the upstream misalignment originat- ing from o proj. and thus achieves smaller gain. Finally, applying compensation to both proj and down proj yields the largest improvement, suggesting that errors at the two sites are complemen- tary. Moreover, the benefit becomes larger at higher pruning ratios, where the retained subspace is smaller and orientation recovery plays a more critical role.
Efficiency Table 5 summarizes the number of parameters, memory usage, and pruning time per layer at different pruning ratios. As expected, both the parameter count and memory decrease mono- tonically as the pruning ratio increases, confirming the resource savings of structured pruning. In contrast, pruning time exhibits a counter-intuitive trend. While the dominant computation is the SVD of M YZT x which incurs constant O ) cost, the computation of Z W XK depends on the number of kept columns (Section (.3).. In principle, a larger pruning ratio (smaller k)) should make this step cheaper. However, in practice, observed that pruning time becomes

---

Published as a conference paper at ICLR 2026
slightly longer higher ratios. This is because when & is small, W results in tall and nar- row matrix multiplication, which GPU libraries (e.g., cuBLAS) handle less efficiently than more square-shaped matrices. Similar behavior has been reported in prior work Rivera et al. (202I). No- tably, pruning completes within 10 seconds per layer, suggesting that the proposed method remains practically applicable even to larger-scale models.
5 RELATED WORK
Model compression in LLMs Large language models (LLMs) incur substantial computational and memory costs, motivating the development of compression techniques. Among the most widely studied approaches are quantization, which lowers precision to improve efficiency Lang al. (2024). and distillation, which transfers knowledge from a large teacher to a smaller student model Yang et al. (2024). In this work, we focus on pruming, which removes unnecessary parameters.
Unstructured pruning Unstructured pruning accelerates inference by sparsifying weight ma- trices. SparseGPT Frantar & Alistarh (2023) enables one-shot pruning of LLMs via efficient second-order update. WANDA (Snn et al. (2024) introduces an activation-aware importance score that works with only a small calibration set. There also exist approaches that pursue higher accuracy, such as AiphaPu Lu et al. (2024). which varies the pruning ratio across layers. While effective, unstructured methods do not actually reduce parameter count. Their memory and speed benefits depend on specialized hardware supports, and thus they are generally unsuitable for small devices.
Structured Pruning Structured pruning removes parameters the level of rows, columns, or blocks, directly shrinking model size and memory footprint. LLMM-Puner Ma et al. (2023) demon- strates structured pruning for L.LMs but typically requires downstream fine-tuning, and Shortened- LLaMA Kim et al. (2024) prunes depth (layers) with retraining, In contrast, methods such as Wanda-sp and FLAP are applicable without additional retraining An et al. (2024): Wanda-sp ex- tends WANDA's activation-aware rule column pruning. while FLAP compensates post-pruning errors via a bias term. SliceGPT AAhhboos et al. (2024) leverages a computational invariance of ReMSnorn-comeected transformers: by applying orthogonal reparameterization (derived via prin- cipal component analysis), the model is reformulated into a rotated basis where entire rows and columns can be deleted while preserving functional equivalence. Rotation has also been used to improve prunability in other forms: RotPruner Chen & Wang (2025) learns layer-wise orthogonal transforms obtain pruning-friendly parameterizations, and DenoiseRotator Gu et al. (2025) trains rotations that concentrate importance scores before pruning. These methods use rotation as an ad- ditional parameterization facilitate pruning before or during parameter removal. Unlike these rotation-learning or slicing-based approaches, RCPU focuses on reducing the pruning error after structured column removal through an analytically derived orthogonal compensation. Our work closer to FLAP, trying to minimize pruning-induced output error and achieve better performance.
6 CONCLUSION
In this paper, we proposed RCPU, a rotation-constraine error compensation method, for struc- tured pruning of large language models. By formulating post-pruning recovery as an Orthogonal Procrustes problem, our approach aimed to preserve the geometry of output representations while re-aligning the retained subspace to the original outputs. To complement this update, we adopted a variance-aware importance score that preferentially retains columns contributing to principal out- compensation. Through
put directions, thereby enhancing the effectiveness of cotation-constrain experiments on Llama-7B and lllam-2--133 we demonstrated that RCPU improves perplexity and task accuracy across pruning ratios, outperforming existing baselines. The improvements were par- ticularly pronounced at higher pruning levels, indicating that preseeiy updates become increasingly critical as the retained subspace shrinks. Moreover, we showed the method requires no additional architectural changes and only modest computational overhead, making it practically applicable to large-scale deployments.
Overall, our findings suggest that incorporating geometric constraints into error compensation for pruning can be a promising direction. We hope that this framework stimulates further investigation into pruning-aware model updates that balance statistical stability and computational efficiency.
10

---

Published as a conference paper at ICLR 2026
ETHICS AND REPRODUCIBILITY STATEMENT
Disclosure of Al assistance. We used large language model to edit and polish the manuscript text. All research ideas, methods, and experiments were conducted solely by the authors.
Data usage and privacy. All calibration and evaluation datasets used in this work are publicly available and contain no sensitive personal information. We used the data under their respective licenses, made no attempts at re-identification, and did not store or share model inputs and outputs beyond the scope of calibration and evaluation.
Environmental impact. This study adds only a small incremental compute footprint: calibration consists of forward passes plus one small SVD per targeted sub-layer. We did not perform gradient- based fine-tume fine-tuning in our experiments. At deployment time, structured pruning reduces parameter count and effective FLOPs, which can lower inference cost under comparable hardware and batching conditions.
Fairness and safety. Structured pruning can alter performance unevenly across tasks, domains, or languages. We therefore evaluate on diverse benchmarks and report per-task metrics to surface potential regressions. No safety-critical deployment is claimed.
Reproducibility. All experimental settings and tools are described in the main text, including the models and datasets and the calibration setup.
REFERENCES
Yongqi An, Xu Zhao, Tao Yu, Ming Tang, and Jinqiao Wang. Fluctuation-based adaptive struc- tured pruning for large language models. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pp. 10865-10873, 2024.
Saleh Ashkboos, Maximilian I.. Croci, Marcelo Gennari do Nascimento, Torsten Hoefler, and James Hensman. SliceGPT: Compress large language models by deleting rows and columns. In The Twelfth International Conference on Learning Representations, 2024.
Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 7432-7439, 2020.
Haoxian Chen and Limin Wang. Rotpruner: Large language model pruning in rotated space, 2025.
URL httpss///penreeveew..t http:/pwwww.eee..eeeeeoeerrroeaa forum Peni-id-ii 1d-w999-iy00cc
Christopher Clark, Kenton Lee, Ming-Wer Wei Chang, Tom Kwiatkowski. Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. In Proceedings of the Conference of the North American Chupter of the Association for Computational Linguis- fies: Human Language Technologies, pp. 2924-292229 2946, 2019.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try are, the ai2 reasoning challenge. arXiv preprint 1=XXiv:803.004555 2018
Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. A survey on rag meeting llms: Towards eetrieval-augmenee large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 6491-6501, 2024.
Elias Frantar and Dan Alistarh. Sparsegpt: Massive language models can be accurately pruned in one-shot. In International conference on machine learning, pp. 10323-10337. PMLR, 2023.
Leo Gao, Jonathan Tow, Baber Abbasi. Stella Biderman, Sid Black, Anthony DiPofi, Charles Fos- ter. Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonell, Niklas Muen- nighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang
11

---

Published as a conference paper at ICLR 2026
Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. The language model evaluation harness, 2024.
Sanjay Surendranath Girija, Shashank Kapoor, Lakshit Arora, Dipen Pradhan, Aman Raj, and Ankit Shetgaonkar. Optimizing IIms for resource-constrained environments: survey of model com- pression techniques. arXiv preprint arXiv:2505.0233 2025. To appear in Proceedings of the 49th IEEE Annual Computers, Software & Applications Conference.
Gene H Golub and Charles F Van Loan. Matrix computations. JHU press, 2013.
Tianteng Gu, Bei Liu, Bo Xiao, Ke Zeng, Jiacheng Liu, and Yanmin Qian. Denoiserotator: Enhance pruning robustness for Ilms via importance concentration. arXiv preprint arXiv:2505.23049, 2025.
Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning. 2001. Accessed: 2025-11- 11-26.
Yang He and Lingao Xiao. Structured pruning for deep convolutional neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5):2900-25 29199, 2024.
Arthur E Hoerl and Robert W Kennard. Ridge regression: Biased estimation for nonorthogonal
problems. Technometrics, 12(1):55 67, 1970.
Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. A survey on large language models for code generation. ACM Trans. Softw. Eng. Methodol,, 2025. Just Accepted.
Bo-Kyeong Kim, Geonmin Kim, Tae-Ho Kim, Thibault Castells, Shinkook Choi, Junho Shin, and Hyoung-Kyu Song. Shortened LLaMA: A simple depth pruning for large language models. In ICLR Workshop Mathematical and Empirical Understanding Foundation Models, 2024.
Jiedong Lang, Zhehao Guo, and Shuyu Huang. A comprehensive study on quantization techniques for large language models. In Proceedings of 4th International Conference on Artificial Intelli- gence, Robotics, and Communication, pp. 224 231. IEFE, 2024.
Haiquan Lu, Yefan Zhou, Shiwei Liu, Zhangyang Wang, Michael W. Mahoney, and Yaoqing Yang. Alphapruning: Using heavy-tailed self regularization theory for improved layer-wise pruning of large language models. In The Thirty-eighth Annual Conference on Neural Information Process- ing Systems, 2024.
Xinyin Ma, Gongfan Fang, and Xinchao Wang. LIm-pruner: On the structural pruning of large language models. Advances in neural information processing systems, 36:21702-21720, 2023.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. arXiv preprint 1609.0:60 2016.
Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Xinhao Cheng, Hongyi Jin, Tianqi Chen, and Zhihao Jia. Towards efficient generative large language model serving: A survey from algorithms to systems. ACM Comput. Surv., 58(1), 2025.
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? dataset for open book question answering. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 2381-2391 Association for Compu- tational Linguisties, 2018
Cody Rivera, Jieyang Chen, Nan Xiong, Jing Zhang, Shuaiwen Leon Song, and Dingwen Tao. Tsm2x: High- Biggh-peffrmance tall-and-skinny matrix-matriy multiplication on gpus. Journal of Par- allel and Distributed Computing, 151:70-88. 85, 2021.
Shaibal Saha and Lanyu Xu. Vision transformers on the edge: comprehensive survey of model compression and acceleration strategies. Neurocomputing, pp. 130417, 2025.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adver- sarial winograd schema challenge at scale. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 8732-8740, 2020.
12

---

Published as a conference paper at ICLR 2026
Mingjie Sun, Zhuang Liu, Anna Bair, and J. Zico Kolter. A simple and effective pruming approach for large language models. In Proceedings of the 12th Interaational Conference on Learning Representations, 2024.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arX1v:230..1337,, 2023a
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.099 0.98888 2023b.
Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, al. A survey large language model based autonomous agents. Frontiers of Computer Science, 18(6):186345, 2024.
Chuanpeng Yang, Yao Zhu, Wang Lu, Yidong Wang, Qian Chen, Chenlong Gao, Bingjic Yan, and Yiqiang Chen. Survey on knowledge distillation for large language models: Methods, evaluation, and application. ACM Trans. Intell. Syst. Technol., 2024. Just Accepted.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can ma- chine really finish your sentence? In Proceedings of the 57th Anmual Meeting of the Association for Computational Linguisties, pp.
Association for Computational Linguistics, 2019.
Haopeng Zhang, Philip S. Yu, and Jiawei Zhang. A systematic survey of text summarization: From statistical methods to large language models. ACM Comput. Surv. 57(11), 2025.
13

---

Published as a conference paper at ICLR 2026
7 APPENDIX
7.1 HOW TO DETERMINE THE BEST
We select the regularization strength Abest for the Ridge+LS baseline the value that minimizes the perplexity on the same calibration data used for computing the compensation. This choice is moti- vated by the following reasons. First, the calibration set very small, consisting of only 128-512 samples. Partitioning this already limited set into separate train and validation subsets would render estimation statistically unstable. Indeed, our experiments on Llama-2-13B, even with 512 calibration samples, the perplexity varies substantially across different À values (see Section 7.4 Introducing an additional split would further reduce the effective sample size and increase estima- tion variance. Second, RCPU itself also relics solely on the calibration samples and does not use separate validation set. Using validation only for the Ridge baseline would therefore introduce an inconsistency in the comparison protocol.
7.2
PPL COMPARISON LLAMA-2-13B
WANDA-sp + No compensation
29
Proposed score + No com pompaattiion
WANDA-Sp + LS compensation
Proposed score + LS compensation
Dexitty (PPL)
WANDA-SP + RCPU compensation
Proposed score RCPU compensation
23
23
21
21
19-
17
15
13
13
11
10%
20% Pruning ratio
30%
10%
20% Pruning ratio
30%
(a) WANDA-sp score + compensation (128 samples)
(b) Proposed score + compensation (128 samples)
29
WANDA-Sp + No compensation
29
Proposed score + No com pensattioon
27
sA + compensation
27
Proposed score + compensation
(PPL)
RCPU compensation
Proposed score RCPU compensation
23
21
21
19
19-
17
13
11
11
20% Pruning ratio
Pruning ratio
(c) WANDA-sp score + compensation (512 samples)
(d) Proposed score + compensation (512 samples)
Figure PPL P.R. for different calibration sizes and compensation methods on Llama-2- 13B.
7.3
OTHER BENCHMARKS AND MODELS
14

---

Published as a conference paper at ICLR 2026
Table 6: Llama-1-7B calib 512
Method
P.R.
BoolQ
PIQA
Hella
Wino
ARC-e
ARC-c
OBQA
Mean
Llama-7B (Original)
0%
75.10
78.67
76.18
70.01
72.85
44.79
44.40
66.00
FLAP
10%
74.46
77.75
73.05
68.19
70.66
41.98
43.20
64.18
WANDA-Sp
10%
75.35
76.82
74.14
68.51
71.17
44.20
38.80
64.14
mop.Score++L. (Abest)
10%
75.02
76.77
73.33
68.59
70.75
42.49
39.00
63.71
RCPU (Rot.)
10%
76.06
76.88
74.45
68.27
70.96
42.92
41.40
64.42
RCPU(Rot.+Scable
10%
76.18
76.77
74.50
68.43
70.83
43.00
41.80
64.50
FLAP
20%
71.07
75.24
68.46
66.93
68.06
40.27
41.00
61.58
WANDA-Sp
20%
66.94
75.14
69.85
66.06
64.98
40.53
38.40
60.27
Nop.Score+LS (Abest)
20%
71.71
74.76
69.70
67.48
66.29
37.97
39.20
61.02
RCPU (Rot.)
20%
72.87
76.12
70.93
67.96
68.01
39.93
39.40
62.17
RCPU SRot++
20%
73.43
75.52
70.80
67.80
68.06
40.61
39.60
62.26
FLAP
30%
67.00
71.38
60.85
63.22
58.88
34.39
40.20
56.56
WANDA-sp
30%
58.44
68.28
56.69
56.35
56.14
33.70
36.00
52.23
Prop.Score+L.S (Abest)
30%
64.53
71.11
62.32
61.96
59.76
33.79
37.20
55.81
RCPU (Rot.)
30%
65.47
71.65
64.47
62.19
61.95
36.18
36.60
56.93
RCPU (Rot.+Scale)
30%
65.29
71.71
64.41
63.30
63.13
36.35
37.60
57.40
Table Llama-2-13B calib 128
Method
P.R.
BoolQ
PIQA
Hella
Wino
ARC-e ARC-c
OBQA
Mean
Llama2-13B (Original)
0%
80.55
79.05
79.37
72.14
77.44
49.06
45.20
68.97
FLAP
10%
70.98
78.13
75.72
69.53
72.85
45.22
44.80
65.32
SliceGPT
10%
68.10
76.00
71.17
71.59
75,25
48.29
43.20
64.80
WANDA-sp
10%
78.01
77.86
77.98
71.03
75.88
48.81
44.60
67.74
SS (Abest)
10%
78.47
77.26
77.34
71.27
73.02
46.42
43.00
66.68
RCPU(Rot.)
10%
78.96
77.75
77.78
71.82
74.11
46.24
43.60
67.18
RCPU(Rot
10%
78.89
78.12
77.75
71.90
73.73
46.16
44.00
67.22
FLAP
20%
69.48
74.65
69.59
67.17
66.67
40.87
42.60
61.58
SliceGPT
20%
44.89
71.55
62.78
68.35
67.42
42.06
41.40
56.92
WANDA-Sp
20%
73.21
76.99
73.06
69.14
71.17
44.54
42.80
64.42
Prop.Score+LS. (Abett)
20%
71.77
74.70
72.30
69.53
69.28
42.41
41.40
63.06
RCPU(Rott)
20%
72.93
76.06
73.03
70.32
70.03
43.08
43.20
64.09
RCPU(Rot. Scale)
20%
72.50
75.95
72.93
70.24
69.52
43.08
42.00
63.74
FLAP
30%
64.07
71.00
63.33
63.54
62.92
39.68
40.80
57.91
SliceGPT
30%
39.08
65.13
52.29
65.43
53.45
36.69
39.20
50.18
WANDA-sp (Abest)
30%
62.01
63.49
35.69
49.64
42.21
25.76
28.40
43.89
30%
61.19
69.21
60.20
61.17
57.87
33.19
37.00
54.26
RCPU(Rot.)
30%
66.48
72.85
65.37
65.66
61.53
36.68
40.40
58.42
RCPU(Rot. Scale)
30%
66.45
72.79
64.93
64.95
61.57
36.26
39.80
58.11
15

---

Published as a conference paper at ICLR 2026
Table PPI. comparison in Vicuna-7B under Nealib 128, 512.
Vicuna-7B
Method
PR
128
512
Original
0%
16.24
16.24
Prop.Score+LSS (Ahest)
10%
19.10
17.58
RCPU (Rot.)
10%
17.54
17.26
RCPU (Rot.+Scale)
10%
17.52
17.22
Prop.Score+LS (Abest)
20%
20.53
18.54
RCPU (Rot.)
20%
19.62
18.99
RCPU (Rot++Scale)
20%
19.59
18.93
Prop.Score+LS (Abest)
30%
26.54
20.52
RCPU (Rot.)
30%
23.11
21.55
RCPU (Rot.+ccale)
30%
22.91
21.52
Table 9: Vicuna-7B calib 128
Method
P.R.
BoolQ
PIQA
Hella
Wino
ARC-e
ARC-e
OBQA
Mean
Vicuna-78 7B (Original)
0%
80.92
77.31
73.76
69.38
71,25
45.90
45.00
66.22
Prop.Score+L.S (Abest
10%
71.22
72.63
68.55
66.54
65.49
39.42
38.60
60.35
RCPU (Rot.)
10%
78.26
76.33
72.39
68.67
71.25
44.62
42.80
64.90
SCaal))t.+SSale)
10%
78.04
76.22
72.29
68.51
71.38
44.80
43.00
64.89
Prop.Score+L.. (Abest)
20%
66.67
73.18
65.56
63.93
64.90
39.25
36.60
58.58
RCPU (Rot.)
20%
70.95
73.56
68.37
66.14
66.20
41.21
40.80
61.03
RCPU (Rot.+Scale)
20%
70.21
73.88
68.35
66.77
66.33
40.87
41.40
61.12
Score++S (Ahest)
30%
52.23
65.07
53.62
59.12
54.97
31.74
36.60
50.48
RCPU (Rot.)
30%
62.66
69.91
60.34
62.59
61.28
35.84
40.00
56.09
RCPU (Rot.+Scale)
30%
63.70
69.70
60.79
62.67
61.87
37.20
40.60
56.65
Table 10: Vicuna-7B calib 512
Method
P.R.
BoolQ
PIQA
Hella
Wino ARC-e ARC-e
OBQA
Mean
Vicuna-7B (Original)
0%
80.92
77.31
73.76
69.38
71.25
45,90
45,00
66.22
Prop.Score+L.S (Abest)
10%
78.10
76.66
71.62
67.88
70.92
43.94
41.60
64.39
RCPU (Rot.)
10%
78.35
76.44
72.60
68.43
71.93
44.80
43.00
65.08
RCPa(RRt.+Scalle)
10%
78.44
76.61
72.54
67.88
72.01
45.48
42.80
65.11
Prop.Score+LS (Abest)
20%
71.90
74.32
68.19
64.01
67.76
41.89
40.40
61.21
RCPU (Rot.)
20%
70.55
74.86
69.01
65.90
67.93
41.89
41.40
61.65
RCPU (Rot.+Scale)
20%
71.22
74.21
69.35
65.98
67.42
41.55
40.80
61.50
ScoreSco (Abest)
30%
65.75
70.89
61.43
63.06
62.33
37.80
40.00
57.32
RCPU (Rot.)
30%
64.89
70.67
62.55
63.61
63.01
38.40
39.80
57.56
RCPU (Rot.+Scale)
30%
65.29
70.67
62.38
63.69
63.34
37.63
39.20
57.46
16

---

Published as a conference paper at ICLR 2026
7.4
PPL VS IN LEAST SQUARE WITH RIDGE
no-128, pruning-0.1
nS=128, prining-0.2
ns= pruning-0.3
16.5
14.50
RCPU best
21
1425
16:0
14.00
15.5
15
20
RCPU
13.75
15.0
13.50
18-
L5
RCPU
1325
14.0
10a
10
10-h
10"
ns=512, pruning=0.1
ns pruning=0.2
ns=512, pruning=0.3
13.7
15.5
LS
13.6
15.0
16-
14.5
13.3
17
14.0
132
10
10th
10
10
107
10
13.5
10th
10-3
16a1
10
107
10:
10th
10
10
10:
10'
10'
Figure 5: Llama-7B: PPL vs X in LS (Pruned by proposed score)
nS=128, pruning-8.1
pruning-0.2
n5=128, pruning-0.3
30
RCPU best
10a
10*
best
best X
10a.
10a
301.
DPL
10a
10a
101.
10
10
10
10
1023
10
10
10'
10a3
16
nS=512, pruning--.1 0.1
ns=512, pruning-O.Z 0.2
ns=512, pruning-0.3
L5
LS
20a
10'
RCPU
14.52
10a
10
10
10
10
10
10
30
10
10
10
10
10
10
10
10
10'
10
Figure 6: Llama-2-13B: PPL in LS+Ridge (Pruned by proposed score)
Figure and B/swww the PPL versus in Llama-7B and Llama-2-13B models. Overall, the best À varies widely depending on the model type, pruning ratio, and the number of calibration reliable manner is inherently difficult. For
samples. This suggests that finding an optimal Llama-7B, the best À chosen on the calibration set does not yield the best test-set performance, indicating insufficient generalization. For Llama-2-13B, the PPL becomes unstable for certain values (especially near zero). This instability is particularly pronounced when the calibration size is 128, which likely reflects the severe mismatch between the number of parameters and the amount of available calibration data. The best À achieves performance comparable to RCPU when using 128 samples, and slightly better than RCPU when using 512 samples.
However, across models and calibration sizes, RCPU consistently outperforms LS+Ridge down- stream tasks (Table 2 Table 3, Table 6 Table 7). This indicates that LS+Ridge compensation achieves some degree of in-domain generalization but fails to generalize out-of-domalle From the perspective of preserving the pretrained knowledge of the LLM, RCPU provides a more robust form of compensation.
17


Reviews:
[{"review_id": "pS8vUCbe37", "reviewer": "Reviewer_SQAA", "summary": "The paper proposes RCPU, a post‑pruning compensation method for LLMs that restricts the update to an orthogonal rotation (optionally with a single global scale) computed via an Orthogonal Procrustes fit between the pruned layer’s outputs and the original outputs on a small calibration set. This preserves output norms and inner products, reducing overfitting and geometric distortion that plague unconstrained least‑squares fixes. It’s paired with a simple variance‑aware column score, (\\gamma_j=|W_{:,j}||X_{j,:}|\\operatorname{Var}(X_{j,:})), to keep columns likely to support principal output directions. Applied layerwise to LLaMA‑7B with 128 WikiText‑2 calibration samples, RCPU consistently lowers perplexity and improves or matches zero‑shot accuracy versus WANDA‑sp and the bias‑only FLAP baseline across 10–30% structured pruning. Figure 1 (p.2) sketches the pipeline; Algorithm 1 (p.5) gives the exact steps; Tables 1–2 (pp.6–7) report the gains; Tables 3–4 (p.8) show where to apply it and the runtime/memory trade‑offs. \n\nKey contributions\n\n* Rotation‑constrained compensation: Formulates post‑pruning recovery as a closed‑form Orthogonal Procrustes problem and updates only the retained columns via (W_K \\leftarrow Q^\\star W_K) (optionally (s^\\star Q^\\star W_K)), preserving geometry while aligning to the original outputs (Sec. 3.1, Fig. 1). \n* Variance‑aware selection rule: Extends WANDA‑sp’s magnitude/activation score with an input‑variance term to prefer columns supporting dominant directions, improving compatibility with the rotation update (Sec. 3.2). \n* Drop‑in, no finetuning, modest cost: Layerwise, closed‑form SVD per treated sublayer; no architectural changes; measured pruning time under ~10 s per layer on LLaMA‑7B while reducing parameters and memory with higher pruning ratios (Algorithm 1; Table 4). \n* Empirical gains on LLaMA‑7B: Lower perplexity than WANDA‑sp and typically better than FLAP at 10–30% pruning (e.g., at 20%: PPL 14.40 vs 16.70/15.36) and competitive zero‑shot accuracy across BoolQ, PIQA, HellaSwag, WinoGrande, ARC‑e/c, OBQA; rotation outperforms unconstrained least‑squares fixes, especially as pruning increases (Tables 1–2; Fig. 2). \n* Ablations on where to rotate: Best results when compensating both attention (o_proj) and MLP (down_proj); (o_proj) alone helps more than (down_proj) alone (Table 3).", "questions": "1. “Geometry preservation” beyond intuition.\n   The paper asserts that rotations preserve norms and inner products and reduce distortion relative to LS (§2.2, §3.1), but this is not measured directly.\n   Request: Report cosine similarity and norm‑ratio histograms between original outputs Y and compensated outputs QZ on held‑out data. Correlate those with perplexity deltas per sublayer. This would substantiate the geometric motivation. (Fig. 1; §2.2–§3.1) \n\n2. Calibration size and distribution shift.\n   All results use 128 WikiText‑2 samples (§4.1). The stated advantage is reduced overfitting under small calibration, but sample‑efficiency is not quantified.\n   Request: Provide calibration‑size curves (N ∈ {32, 64, 128, 512}) and a distribution‑shift test: calibrate on WikiText‑2, evaluate on the seven tasks, and vice versa. Present curves similar to Figure 2 to show robustness. (Fig. 2; §4.1) \n\n3. When not to rotate.\n   Practitioners need a diagnostic to decide whether a sublayer will benefit.\n   Request: Report a simple predictor such as the principal angle between span(Z) and span(Y) or the condition number of M = YZ⊤, and plot expected PPL change vs that quantity. This could become a decision rule for where to apply RCPU. (Eq. 6; Table 3)", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Strengths: originality\n\n* Recasts post‑pruning error correction as an orthogonal Procrustes alignment on the layer outputs, restricting the compensation to a rotation (optionally with a single global scale). This is a crisp, geometry‑preserving reframing that removes a well‑known limitation of unconstrained least‑squares fixes under tiny calibration sets: shear/scale distortions and overfitting. The formulation and closed‑form solution are given in Sec. 3.1, with the pipeline visualized in Figure 1 (p. 2). \n* Pairs that rotation with a variance‑aware column score that augments magnitude and activation scale with input variance, which is a simple but thoughtful twist that prefers columns supporting principal output directions (Sec. 3.2). The idea is original in the structured‑pruning context and complementary to existing heuristics such as Wanda‑sp. \n* Offers a drop‑in procedure that can be inserted immediately after column pruning without architectural changes or fine‑tuning, expressed succinctly in Algorithm 1 (p. 5). The combination of a classic SVD‑based alignment with modern LLM pruning is a creative, low‑friction synthesis. \n\nStrengths: quality\n\n* Methodological soundness. The paper motivates the failure modes of unconstrained least squares, formalizes the rotation‑constrained problem, and provides a closed‑form optimizer via SVD along with a scaled variant. The constraints directly preserve norms and inner products, aligning with the desiderata stated in Sec. 2.2 and solved in Sec. 3.1. Complexity is analyzed (O(d_out³) per treated sublayer), and the computational steps are transparent (Sec. 3.3). \n* Strong empirical evidence. On LLaMA‑7B with only 128 WikiText‑2 calibration samples, rotation consistently reduces perplexity vs. Wanda‑sp and typically matches or beats FLAP across 10–30% pruning. Table 1 (p. 6) shows, for example, at 20% pruning: PPL 14.40 (RCPU Rot.) vs. 16.70 (Wanda‑sp) and 15.36 (FLAP). Figure 2 (p. 7) further demonstrates that least‑squares compensation often harms PPL while rotation reliably helps, under both scoring rules. \n* Breadth of evaluation. Zero‑shot accuracy is reported on seven diverse language understanding tasks; RCPU variants are competitive and often superior to baselines, including at higher pruning where accuracy is most fragile (Table 2, p. 7). The study also includes targeted ablations that identify where compensation matters most: updating both attention o_proj and MLP down_proj yields the largest gains (Table 3, p. 8). \n* Practicality demonstrated. The efficiency table reports per‑layer pruning times under 10 s and monotonic memory savings with higher pruning ratios (Table 4, p. 8), strengthening the claim that the method is deployable without exotic infrastructure. \n\nStrengths: clarity\n\n* The paper is exceedingly readable: notation is introduced cleanly (Sec. 2.1), the failure modes of LS are enumerated before the proposed fix (Sec. 2.2), and the solution path is linear. Figure 1 on page 2 gives an at‑a‑glance workflow that matches Algorithm 1 on page 5, which makes the procedure easy to reimplement. \n* Tables and plots are well‑chosen and interpretable: Table 1 isolates PPL vs. pruning ratio; Figure 2 explicitly contrasts “no compensation,” “LS,” and “RCPU” under both scoring rules; Table 2 summarizes per‑task and mean accuracies; Tables 3–4 provide ablations and resource figures. The accompanying text consistently ties these visuals back to the stated hypotheses. \n* Reproducibility details are reasonable: calibration dataset, number of samples, layers targeted, evaluation harness, and hardware are all specified (Sec. 4.1), and an anonymized code link is provided. \n\nStrengths: significance\n\n* Practical impact for structured pruning. Many deployments can only spare tiny calibration sets and cannot fine‑tune; a one‑shot, geometry‑preserving compensation that is drop‑in and SVD‑simple is immediately useful. Gains are most pronounced exactly where practitioners need them: at higher pruning ratios (e.g., 30% PPL improvements over Wanda‑sp in Table 1), where retaining performance is hardest. \n* Conceptual influence. Framing error recovery as orientation correction rather than generic regression is a clean lens that could inform future compression methods, e.g., combining with other structure‑preserving transforms or smarter selection rules. The ablation showing o_proj benefits more than down_proj (Table 3) offers actionable insights about where geometry matters most in transformer blocks. \n* Broad applicability. Because the update is layer‑local, closed‑form, and architecture‑agnostic, it should transport to other LLM families and even beyond LMs to any setting where structured pruning removes subspaces and small calibration sets are the norm. The efficiency numbers and memory reductions in Table 4 make this a credible path for edge or resource‑constrained deployments. \n\nOverall: a neat, principled, and practical piece of work. The rotation‑constrained view is conceptually tidy, the experiments are careful and targeted, and the clarity of exposition makes the method easy to adopt.", "weaknesses": "* The central idea is to restrict post‑pruning compensation to an orthogonal Procrustes fit. That’s a neat reuse of a classic tool, but the paper underplays overlap with closely related “structure‑preserving reparameterizations” such as SliceGPT, which also leverages orthogonal transforms to manage structured deletions. The Related Work section asserts a difference but offers no head‑to‑head comparison or synthetic study clarifying when layer‑local Procrustes is preferable to a single global reparameterization. Add a direct comparison against SliceGPT, both as a baseline and as a combined pipeline (SliceGPT first, then RCPU), and make the distinctions operational rather than rhetorical. Cite Figure 1 for the RCPU pipeline and §5.3 for the SliceGPT discussion, then show empirical deltas. \n\n* “Geometry preserving” is asserted, not quantified. Orthogonal maps preserve norms and pairwise angles within the calibration subspace, but the paper does not measure whether downstream layers see closer activations after compensation. Add layerwise geometric diagnostics: cosine similarity and norm‑ratio histograms between original outputs Y and compensated outputs QZ at calibration and test time; report mean and tail statistics per sublayer. Tie these to PPL changes to show that geometry preservation correlates with generalization. Anchor this to the optimization in §3.1 and the workflow in Figure 1. \n\n* The choice to rotate across the full dout mixes attention heads in o_proj. That may be unnecessary coupling. Test block‑diagonal Q that respects head boundaries vs full Q, and report both quality and compute trade‑offs. The ablation in Table 3 shows where rotation helps (o_proj vs down_proj), but not whether cross‑head mixing is essential. This is low‑effort to implement and could improve robustness. \n\n* The scaled variant uses a single global s; Table 1 shows it is largely neutral. Before discarding scaling entirely, probe slightly richer but still well‑conditioned classes: per‑channel diagonal scaling or a small number of Householder reflections. These preserve most geometry while giving capacity to fix systematic magnitude shifts that a single s cannot capture. Evaluate them beside “Rot.” and “Rot.+Scale.” Refer to §3.1–3.3 and Table 1. \n\n* Dependence on calibration size and distribution is not analyzed. All results use 128 WikiText‑2 samples. Provide calibration‑size curves (e.g., N ∈ {32, 64, 128, 512}) and a distribution‑shift test: calibrate on WikiText‑2 but evaluate on the seven downstream tasks, and vice versa. Figure 2 already contrasts LS vs RCPU; extend that figure style to show RCPU’s sample‑efficiency and brittleness under shift. This directly tests the “less prone to overfitting” claim from §2.2–§3.1. \n\n* The variance‑aware score is only partially ablated. Figure 2 contrasts WANDA‑sp vs the proposed score, but the score itself bundles three factors. Add controlled ablations: remove just the variance term, replace sample variance with a shrinkage estimator, and test robust alternatives (median absolute deviation). Report how each affects M = YZ⊤ conditioning and the singular value spectrum that drives Q. Ground this in §3.2 and Figure 2. \n\n* Numerical details of Procrustes are missing. In practice, reflections (det(Q) < 0) can arise; some implementations flip the last column of U to enforce det(Q) = 1. State which variant you use and whether it matters for quality. Also note precision (fp16 vs fp32) and any stabilization for tall‑skinny Z multiplication. This belongs in §3.1–3.3.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fxWy62eWaU", "reviewer": "Reviewer_hBF1", "summary": "The paper proposes **RCPU (Rotation-Constrained Parameter Update)**, a structured pruning method designed to minimize post-pruning output distortion in large language models. After column-wise pruning, the authors introduce an **orthogonal Procrustes-based reparameterization** to align the preserved subspace with the original outputs, optionally scaled by a global scalar factor. This design preserves vector norms and inner products, aiming to maintain geometric consistency with minimal calibration data. Additionally, the paper presents a variance-aware column importance score\n to prioritize dimensions contributing more significantly to dominant output directions.\n Experiments on LLaMA-7B (using 128 WikiText-2 calibration samples) show that RCPU outperforms or matches WANDA-sp and FLAP in perplexity and several zero-shot downstream benchmarks.", "questions": "1. The anonymized code repository appears incomplete; most files are missing except for *main.py*. Could the authors verify that the full implementation has been properly uploaded?\n2. Did the authors visualize how the scores $\\gamma_j$ are distributed across columns? \n3. What's the choice of scaled variant in your experiments, Please clarify how this choice affects the reported results ?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. **Clean and Intuitive Formulation**\n   The method constrains post-pruning compensation to an orthogonal transformation, which is both mathematically sound and conceptually clear. The use of Procrustes alignment offers a closed-form SVD solution that preserves geometric structure and mitigates overfitting.\n\n2. **Innovative Use of Rotation Constraints**\n    Introducing rotation constraints to reduce pruning error is a novel and effective idea. It maintains representational geometry while minimizing reconstruction error without retraining.\n\n3. **Layer-Agnostic and Easily Transferable**\n    The approach is layer-wise, gradient-free, and architecture-agnostic, allowing easy integration into existing pruning pipelines and adaptation to other large models.", "weaknesses": "1. **Limited Experiments and Comparisons**\n   The experiments were conducted only on the 7B model, with no evaluation on larger models. Analysis over varying calibration sizes (8, 32, 64, 128, 512) is needed for robustness. Additionally, comparisons are limited to FLAP and WANDA-sp, ignoring other state-of-the-art methods. \n2. **Unclear Strengths and Limitations of Rotation Constraint**\n   While the paper claims orthogonal alignment preserves inner products and mitigates overfitting, it lacks formal analysis. Additionally, the reasons for not considering other constraints, like regularized least squares, are not discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Lk0ooFmKeG", "reviewer": "Reviewer_hZjV", "summary": "The paper proposes a new method to compensate errors encountered during structured pruning of Large Language models. Generally their algorithm identifies columns (input channels) of the mlp_down projection (or attn_out) to prune and removes the corresponding structures (outputs channels) of the up projection. \nOnce the columns are identified they propose to update the remaining parameters to optimize the loss over a calibration dataset. Instead of updating all non-pruned parameters they only train a rotation matrix, their main motivation being to mitigate overfitting.\n\nTheir algorithm can be applied efficiently for each matrix in isolation, without requiring gradients or similar through the entire model.\n\nThey evaluate their method on Llama 7B and show tiny improvements over previous work \"FLAPS\" but their method also still has a 2% downstream accuracy drop even when only pruning 10% of the model weights.", "questions": "- Can you provide empirical insights that the overfitting could not simply be mitigated by a simple ridge regression or similar?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "The method is conceptually very simple and can be efficiently implemented and also in principle scaled to other methods. It only requires simple linear algebra. Furthermore, since it does structured pruning, the resulting model can readily be served and the speedups easily realized (as opposed to unstructured pruning for example). \nThe idea of constraining the updates to rotations seems interesting and new. Overall it is easy to follow the paper, but that's partially because it is not very deep.", "weaknesses": "- The paper motivates their constraint to rotations by \"overfitting\" however, there is no data presented that this actually happening in any way. Furthermore there are much more straightforward standard regularization approaches (for example Tikhonov) beyond constraining to rotations. \n- The loss 3 decouples over output dimensions. it is unclear why it should not be best to just optimize each output channel independently.\n- The evaluation is pretty constraint and only done on a single, quite outdated and rather small model (Llama-1 7B).\n- The improvements over existing works are marginal (<0.5%), but the large accuracy gap to the dense model remains(2%-9%). Thus the practical relevance  of this work is very minor.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RjL4Jr5c4j", "reviewer": "Reviewer_f6Ag", "summary": "In this paper, the authors aim to mitigate pruning errors in structured pruning for LLMs. They propose a rotation-constrained compensation scheme to preserve geometric information and minimize mismatches during parameter updates. Furthermore, a variance-aware importance score is introduced to maintain the principal output directions. Experiments conducted on LLaMA-7B demonstrate the effectiveness of the proposed method.\n\nStructured pruning is valuable for the practical deployment of LLMs. I lean toward a marginal accept for this paper. If the authors address my concerns during the rebuttal phase, I would like to raise my score.", "questions": "1. The proposed rotation-based pruning method appears conceptually similar to RotPruner and DenoiseRotator. Could the authors clarify the key differences and novel aspects of their approach compared to these prior works? In particular, how does the proposed rotation constraint contribute uniquely to structured pruning beyond what has been achieved in unstructured or semi-structured settings?  \n\n2. The experiments are conducted only on LLaMA-7B. Do the authors expect the proposed method to scale effectively to larger models such as LLaMA-13B or LLaMA-70B?  \n\n3. Can the proposed rotation scheme be generalized to unstructured or semi-structured pruning?  \n\n4. Please correct the minor grammatical issue in Lines 166–167 (“an rotation-constrained” → “a rotation-constrained”).", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Originality:  \nThe paper introduces a rotation-based structured pruning method for LLMs. While the concept of rotation-based pruning is not new, this work offers an alternative approach to structured pruning with clear practical value for real-world deployment.\n\nQuality:  \nThe paper is well-written and logically organized. The motivation is clearly articulated, and the problem formulation is well-grounded. Both the proposed algorithm and its accompanying analysis are reasonable. The experimental results support the authors’ claims.\n\nClarity:  \nThe paper is clearly structured and easy to follow. The presentation of content and formulas are well-explained.\n\nSignificance:  \nThis work focuses on recovering pruning errors in structured pruning for LLMs, which is a meaningful contribution that enhances the practical deployability of large-scale models.", "weaknesses": "Main Comments:  \n1. The proposed rotation idea is very similar to prior works such as RotPruner[1] and DenoiseRotator[2]. Although this paper applies the rotation concept to structured pruning, the aforementioned methods were designed for unstructured and semi-structured pruning. The authors should discuss these related studies in the Introduction and Related Work sections to clarify their distinctions.  \n\n2. The experiments are limited to LLaMA-7B. It would strengthen the paper to include results on larger models (e.g., 13B–70B) to verify scalability and generality.  \n\n3. It would be helpful to discuss whether the proposed rotation scheme can also be extended to unstructured or semi-structured pruning scenarios.  \n\nMinor Comments:  \n1. Lines 166–167: “an rotation-constrained parameter update” should be corrected to “a rotation-constrained parameter update.”\n\n\n[1] RotPruner: Large Language Model Pruning in Rotated Space.  \n[2] DenoiseRotator: Enhance Pruning Robustness for LLMs via Importance Concentration. NeurIPS 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)