Enhancing In-Context Learning Performance with just SVD-Based Weight Pruning: A Theoretical Perspective

Authors: xinhao  yao
xiaolin  hu
shenzhi  yang
yong  liu
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-12 17:22:36

Abstract: Pre-trained large language models (LLMs) based on Transformer have demonstrated striking in-context learning (ICL) abilities. With a few demonstration input-label pairs, they can predict the label for an unseen input without any parameter updates. In this paper, we show an exciting  phenomenon that SVD-based weight pruning can enhance ICL performance, and more surprising, pruning weights in deep layers often results in more stable performance improvements than in shallow layers. However, the underlying mechanism of those findings still remains an open question. To reveal those findings, we conduct an in-depth theoretical analysis by presenting the implicit gradient descent (GD) trajectories of ICL and giving the mutual information based generalization bounds of ICL via full implicit GD trajectories. This helps us reasonably explain the surprising experimental findings.  Besides, based on all our experimental and theoretical insights, we intuitively propose a simple, model-compression and derivative-free algorithm for downstream tasks in enhancing ICL inference. Experiments on benchmark datasets and open source LLMs display the method effectiveness.

Main paper: 
 introduction: 
 Recently, large language models (LLMs) based on the Transformer architecture [43] have emerged striking in-context learning (ICL) capabilities: Given a prompt containing demonstration sample and a test data, the model can make a prediction for the test data and achieve excellent performance without any parameter updates [7, 26, 34, 6] . This leads considerable works that aim to shed light on it [54, 21, 59, 3, 10, 16, 2, 46, 35] .In this paper, we show our surprising findings in ICL inference by experimentally analyzing the effect of singular value decomposition (SVD)-based pruning on the model performance at different depth layers. As demonstrated in Figure 1 : (i) SVD-based weight pruning can enhance ICL performance. Across all cases, it is evident that SVD-based weight pruning can generally enhance model performance at various depth layers, compared to the baseline performance without weight pruning (indicated by the dashed lines); (ii) Pruning weights in deep layers often results in more stable performance improvements than in shallow layers. Specially, deep layers weight matrices can be drastically reduced without much degradation in model performance, and may even get large improvements on it, while the model performance collapses after a sharp reduction at the shallow layers (see Section 2 for details). A similar case in Sharma et al. [38] notes that a large portion of singular values can be removed from linear layers in large language models without affecting or even improving reasoning performance. However, the underlying mechanism of this phenomenon still remains a mystery, and this paper seeks to explore the issue from the following two aspects. layer1 layer26 layer27 no SVD 0 0.1 0.5 0.75 0.9 0.95 0.99 0.995Clipping Rate(0-1) Notice that this figure mainly focuses on exhibiting the impact of weight pruning to the first two and the last two layers of the model and different colors are used to distinguish between these layers. The dashed line represents the pretrained model performance without SVD. We operate on the whole of MLP or ATTN and specifically marked the points of highest performance. The amount of weight pruning is severe, for instance, the highest model performance sometimes occurs at a clipping rate of 0.995. This is about 99.5% of the matrix's original rank. For the definitions of "deep" and "shallow", please refer to Appendix B.2.We conduct an in-depth theoretical analysis to explain the findings. To be more specific, we first analyse the ICL form with Transformer from the perspective of ICL as implicit gradient descent fine-tuning, and we present the implicit gradient descent trajectories of ICL in Theorem 1 in Section 3.2. Afterwards, we use the information-theoretic approaches [55, 32, 51] to give the generalization bounds of ICL via full implicit GD trajectories in Theorem 2 in Section 3.3, explaining (Q1) why SVD-based weight pruning can enhance ICL performance? and (Q2) why do deep and shallow layers exhibit different behaviors when their weight matrices are drastically reduced?How to better use this phenomenon (Q3)? Based on all our experimental and theoretical insights, we intuitively propose a simple, derivative-free and model-compression algorithm in Section 3.4 for downstream tasks in enhancing ICL inference, providing developers and researchers with an effective approach for handling downstream tasks. Experimental results for our algorithm on benchmark datasets [39, 58, 8, 12, 5, 11, 18] and open source LLMs [48, 42] verify that the method can visibly influence performance across different language understanding tasks in ICL inference.Our primary objective is to establish a general theoretical framework that uncovers the underlying mechanism behind the phenomenon that SVD-based weight pruning can enhance ICL performance. Based on our theoretical insights, one can design new ICL algorithms. Accordingly, we did not directly compare our approach with other pruning methods. The algorithm in Section 3.4 is presented solely to illustrate how theoretical analysis can guide experimental procedures effectively.Model compression. In recent years, there has been growing theoretical and experimental evidence that models can be significantly pruned with very little drop in accuracy, thereby significantly reducing their storage requirements. To name a few, Frankle and Carbin [15] indicate that neural networks can typically have over 90% of their weights eliminated with little to no loss in performance. Analogous results have been demonstrated in both feed-forward and convolutional networks used for image classification [24, 49, 22, 37] . More specifically, Sharma et al. [38] present that careful pruning done at specific layers of Transformer models can produce significant boosts in performance on some tasks. The discovery of this phenomenon heightens interest in the connection between generalization and over-parameterization [56, 57] , it also spurs research into developing pruning strategies that facilitate efficient model inference [30] . Additionally, the works [19, 40] find that performance remains nearly unchanged until a significant portion (up to half) of the layers are removed.In-context learning and gradient descent. In order to better understand the ICL capabilities, considerable works try to understand ICL capabilities from the perspective of gradient descent. Irie et al. [21] and Dai et al. [10] explain ICL as implicit fine-tuning by figuring out a dual form of gradient descent for linear Transformer attention. However, the linear attention setting is less commonly used than Softmax attention in the LLMs and the details of gradient descent such as the choice of loss function and training data have not been clearly defined. Therefore, by using weight construction, von Oswald et al. [46] show the equivalence of linear self-attention mechanism and gradient descent on a linear regression task and Akyiurek et al. [3] prove that based on gradient descent and closed-form ridge regression, Transformers can learn linear models as learning algorithms. Further without using weight construction, Ren and Liu [35] connect Softmax attention with kernels and then give a novel interpretation that ICL with Transformer is really equivalent to a contrastive learning pattern.Information-theoretic generalization bounds. The information-theoretic generalization bounds have been developed to analyze the expected generalization error of a learning algorithm. Given that they are dependent of distribution and algorithm, they are ideal tools to study the generalization behaviour of models performed with a specific algorithm. Russo and Zou [36] and Xu and Raginsky [55] first propose the Mutual information (MI) based bounds, which are then strengthened by additional techniques [4, 31, 20, 51] . Specifically, Negrea et al. [31] derive MI-based bounds by developing a PAC-Bayes-like bounding technique and Wang and Mao [51] develop generalization bounds for SGD via constructing an auxiliary iterative noisy process. 
svd-based weight pruning can enhance icl: 
 This section shows our surprising findings in ICL inference by performing a motivating analysis of three benchmark datasets in conjunction with two open source LLMs, the details are as follows.We choose GPT-J (6B,28 layers) [48] and LLAMA2 (7B,32 layers) [42] as our primary models for investigation, owing to their robust ICL performance and moderate model size, which align with our hardware resource. The attention (ATTN) layers are made up of key, query, value, out matrices both in GPT-J-6B and LLAMA2-7B. The mlp (MLP) layers in GPT-J-6B consist of input, output matrices, while the MLP layers in LLAMA2-7B are made up of up, gate, down matrices. For datasets, we use SST-2 [39] for sentiment analysis, RTE [5] for textual entailment and COPA [18] for causal inference. Details regarding datasets and how they were used are shown in Appendix C.1 and C.3.Specially, we use the optimal rank-r approximation mentioned later as SVD-based weight pruning method to show the effect of weight pruning across different layer and module types in Section 2.1, then we further analyze the effect of the ICL shot numbers on it in Section 2.2.The optimal rank-r approximation and SVD. Given a matrix W ∈ R m×n and a constant r ≤ min(m, n), r ∈ N. Eckart-Young-Mirsky theorem [14] provides an optimal solution W * (r) = U :r Σ :r V T :r , U :r , V :r are matrices containing the singular vectors corresponding to the largest r singular values [σ] r 1 . Let ξ = 1 -r min(m,n) ∈ (0, 1) be the clipping rate.We plot the results of applying various amounts of clipping rate ξ to each module matrices in the Transformer architecture on the corresponding evaluation index for each dataset, as depicted in Figure 1 . These plots are grouped, such that each sub-figure corresponds only to the indicated type of dataset, model and module. Notice that this investigation mainly focuses on assessing the impact of weight pruning to the first two and the last two layers of the model to further clarify the impact of layer depth on the model's performance and different colors are used to distinguish between these layers.All sub-figures clearly show an interesting phenomenon about these models in ICL inference: SVDbased weight pruning can enhance ICL performance in both shallow and deep layers across different module types. More surprising, deep layers weight matrices can be drastically reduced without much degradation in model performance, and may even get large improvements on it. This suggests that pruning weights in deep layers can effectively reduce model complexity while maintaining or enhancing model performance. And the model performance collapses to 0.5 (the expectation of a random guess in the binary task) after a sharp reduction at the shallow layers by contrast, indicating a higher sensitivity of the model to changes in shallow layers.Based on the surprising findings in ICL inference mentioned above, three questions are certain to arise: (Q1) Why SVD-based weight pruning can enhance ICL performance? (Q2) Why do deep and shallow layers exhibit different behaviors when their weight matrices are drastically reduced? (Q3) How can we better use the phenomena about ICL in downstream tasks? We will address the first two questions theoretically in Section 3.3 and give a heuristic algorithm in Section 3.4 to answer the last. We also specifically marked the points of highest performance.Given that ICL can achieve higher performance with more demonstrations [23, 1] , we further analyze the non-ignorable effect of different ICL shot numbers. To control for other influencing factors, we focus on SST-2 dataset [39] and retain the same test set for a single random seed as Section 2.1.As shown in Figure 2 , we compare the settings of three different ICL shot numbers: 0, 4 and 10. Following this, we analyze how the phenomenon changes across different shot numbers.Firstly, we note that without weight pruning, the performance of the model improves with an increase in the number of ICL shots. Which is consistent with prior works. Besides, for every shot number, a phenomenon consistent with what is described in Section 2.1 is observed: SVD-based weight pruning can enhance ICL performance, pruning deep layer weight is more stable than pruning shallow weight. Last but not the least, Figure 2 also demonstrates roughly that with a decrease in the number of shots, the rate of performance collapse in the model slows down after a sharp reduction at the shallow layer. Intuitively, this is because LLMs exhibit a shift in focus in the ICL setting, which results in a reduced scope of the output space. The more shots there are, the more pronounced the shift becomes, and this also leads to a faster collapse. We also theoretically discuss it in Section 3.2 and 3.3 (Remark 6).Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes]Justification: In Section 4, we discuss the limitations of our work and identify them as valuable future work.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
theoretical analysis results: 
 In this section, we first describe the core components of our study by reviewing some basic notations in Section 3.1, and present the implicit gradient descent trajectories of ICL in Section 3.2. Afterwards, we give a mutual information based generalization bounds of ICL via full implicit GD trajectories in Section 3.3. Based on all our experimental and theoretical insights, we intuitively propose a derivative-free and effective method for downstream tasks in enhancing ICL inference in Section 3.4. Complete proofs can be found in the Appendix. For ease of qualitative analysis, our theoretical analysis is mainly focuses on linear attention setting. We discuss the standard Softmax attention setting in Appendix A.2 and feed-forward (MLP) layers in Appendix A.3.We let H be the instance space and µ be an unknown distribution on H, specifying random variable h. In ICL setting, the model accepts a sequence of input H = [H s , h N +1 ] drawn i.i.d. from µ , where H s = [h 1 , h 2 , ..., h N ] represents the demonstration sample and h N +1 is the test data. In the information-theoretic analysis framework, we let W ∈ R d be the space of hypotheses related to the model, and Transformer performs an implicit stochastic learning algorithm A which takes the demonstration sample H s as its input and outputs a hypothesis W ∈ W according to some conditional distribution Q W |Hs . Similar to previous works [55, 51] , we give the definition of expected generalization error.Expected generalization error. Given a loss function ℓ : W ×H → R + , where ℓ(w, h) measures the "unfitness" or "error" of any h ∈ H with respect to a hypothesis w ∈ W. We take ℓ as a continuous function and assume that ℓ is differentiable almost everywhere with respect to w. The goal of learning is to find a hypothesis w that minimizes the population risk, and for any w ∈ W, the population risk is defined as L µ (w) ≜ E h∼µ [ℓ(w, h)]. However, since µ via the sample H s can only be partially observed, we instead turn to use the empirical risk, defined as L Hs (w) ≜ 1 N N i=1 ℓ(w, h i ). Then the expected generalization error of A is defined aserror ≜ E W,Hs [L µ (W ) -L Hs (W )],where the expectation is taken over(H s , W ) ∼ µ N ⊗ Q W |Hs .In-context learning with Transformers3 . By prompt design, most context semantic understanding tasks can be unified into classification tasks. Simplify the form of each token in H toh i = [x i , y i ],where[x i ] N 1 ∈ R din and [y i ] N 1 ∈ R doutare encoded input text and corresponding labels respectively. The test token has the form h N +1 = [x N +1 , mask], where mask is the label needed to predict, and usually set 0 as its initialization. Therefore, the form of attention with residual connection4 is as follows:EQUATIONwhere W V ,W K ,W Q ∈ R (dout+din)×(dout+din) are projection matrix, and the mask matrix M = I N ×N 0 0 0 is included in the attention. Specifically, ĥN+1 can be formulated asEQUATIONand for ease of qualitative analysis, Eq.( 2) can be approximated as a relaxed linear attention mechanism by removing the Softmax operation and scale factor:EQUATIONTo begin with, we present the implicit gradient descent of ICL, inspired by [21, 10, 35] . These works describe how ICL with attention can be connected to a meta-optimizer which produces implicit gradient. The following lemma demonstrates the result. Lemma 1 (The Implicit Gradient Descent of ICL in a Single Linear Attention Layer). Consider a Transformer consists of a single linear layer attention with residual connection, parameterized by W V ,W K ,W Q as in Eq.( 3). Same to Section 3.1, let H = [H s , h N +1 ] be the input, whereH s = [h 1 , h 2 , ..., h N ]represents the demonstration sample and h N +1 is the test data. And let ĥN+1 be the single layer output. Then, it holds that∆W icl = W V H s (W K H s ) T W Q = N i=1 W V h i ⊗ W K h i W Q , ĥN+1 = h N +1 + ∆W icl h N +1 ,where W V H s is regarded as the meta-gradient of ICL, which is used to generate the implicit gradient matrix ∆W icl to act on the final feature output. See Appendix A.1 for a proof. Remark 1. It is worth noting that rank(∆W icl ) ≤ rank(H s ) ≤ N by rank relations in matrix multiplication, indicating that this is a low-rank operation. This explains why the effect of different ICL shot numbers is not uniform in Section 2.2. Details of the discussion in standard Softmax attention setting can be found in Appendix A.2. Plus, a significant body of work has discovered that the ICL capabilities of models are not robust to the order of ICL sample. For instance, Lu et al. [28] observed that large language models (LLMs) are sensitive to the sequence of ICL examples, and Liu et al. [27] reported that the context-based ICL performance exhibits a U-shaped curve-models tend to perform better with information that appears at the beginning or at the end of the input context. This aligns with observations from actual model training where the sequence of training samples affects outcomes especially when training with a batch size of 1.According to the above, we further analyze the implicit gradient descent trajectories of ICL and provide the following Theorem. We will denote ∆W t icl by ∆W t when there is no ambiguity, where t represents t-th layer. Theorem 1 (The Implicit Gradient Descent Trajectories of ICL). Consider a Transformer as a stack of L linear attention blocks with residual connection, parameterized by[W t V ] L 1 ,[W t K ] L 1 ,[W t Q ] L 1 . Denote [h t i ] N +1 1 as the output of the t-th layer, [h 0 i ] N +1 1 as the initial input. Then for t ∈ [L], it holds that G t = ∆W t (1 + W t-1 ) = ∆W t (1 + W 0 + t-1 j=1 G j ), h t N +1 = h 0 N +1 + t j=1 G t h 0 N +1 = h 0 N +1 + W t h 0 N +1 ,where∆W t ≜ N i=1 W t V h t-1 i ⊗ W t K h t-1 i W t Q , W 0 = 0, W t = W t-1 + G t and G 1 = ∆W 1 . See Appendix A.4 for a proof. Remark 2. Note that the exclusion of Transformer weight ([W t K ] L 1 , [W t Q ] L 1 , [W t V ] L 1 )implies that G t is only dependent on W t-1 and H s , this is consistent with gradient descent in terms of relevance.In this part, for simplicity of representation, we flatten the weight matrix into a vector form (vec(W t ) = W t ∈ R d , vec(G t ) = G t ∈ R d) and conduct our analysis within the weight and implicit gradient space of hypotheses W as detailed in Section 3.1. Notably, Ahn et al. [2] observe that, with the optimal parameters, a single layer linear Transformer implements a single step of preconditioned gradient descent and the preconditioning matrix not only adapts to the distribution of input data but also to the variance caused by data inadequacy. Garg et al. [16] show empirically that the trained Transformer is able to in-context learn the class of linear functions with respect to the prompt distribution, performing comparably to the optimal least squares estimator for any number of in-context examples considered, as exhibited in Figure 5a .In addition, we also make a discussion on the noise of the implicit gradient in Section C.4, our analytical discussion indicates that the implicit gradients produced by Transformers in practical applications are noisy due to factors such as the extent of model pre-training and data characteristics (e.g., ICL shot number). We first present the assumption used in this subsection.Let G t ≜ 1 N N i=1 ∇ℓ i be the best implicit gradient of ICL that the model can produce, Gt ≜ 1 b b i=1∇ℓ i be the implicit gradient of ICL generated by the model in practical applications. N is the threshold and b is the the shot number in the actual input defined in Section C.4. And V t ≜ G t -Gt is the gradient noise caused by shot number,C t ≜ N -b b(N -1) ( 1 N N i=1 ∇ℓ i ∇ℓ T i -G t G T t )is the implicit gradient noise covariance. Similar to Wang and Mao [51] 's assumption in SGD, we approximate V t up to its second moment. Assumption 1. Assume the implicit gradient noise V t follows a Gaussian distribution, i.e., V t ∼ N (0, C t ), then in ICL implicit gradient descent trajectories,EQUATIONwhereN t ∼ N (0, I d ) is the standard Gaussian 5 . Remark 3.Regarding the validation of this assumption, empirical evidence from works [53, 25] , suggests that SGD and Eq.( 4) can achieve the similar testing performance. Additionally, we refer readers to some recent works [21, 16, 3, 10, 2, 46, 35] , where the authors empirically verify that in-context learning of Transformer can achieve the similar testing performance to SGD. Together suggesting that studying Eq.( 4) is arguably sufficient to understand generalization properties of ICL.Theorem 1 indicates that the initial parameter W 0 = 0, which is independent of all other random variables. And an L-layer Transformer does implicit GD of ICL stops after L updates, outputting W L as the implicit learned parameter. Our main results are mutual information based expected generalization error bounds of ICL, as presented in Theorem 2. Theorem 2 (The Generalization Bounds of ICL via Full Implicit Gradient Descent Trajectories).Under the conditions of Theorem 1 and Assumption 1, assume the implicit gradient noise covariance C t is a positive-define matrix, the loss ℓ(w, h) is R-subGaussian for anyw ∈ W ∈ R d , then error ≤ R 2 N L t=1 E Hs Wt-1   d log    ∥∆W t ∥ 2 F • 1 + t-1 j=1 G j 2 F + tr{C t } d    -tr{log C t }   , where vec(G t ) ∈ R d , ∆W t ≜ N i=1 W t V h t-1 i ⊗ W t K h t-1 i W t Q and G t = ∆W t (1 + W 0 + t-1 j=1 G j ) = ∆W t (1 + W t-1). tr{•} denotes the trace of a matrix, ∥•∥ F denotes the Frobenius norm of a matrix and E X Y is the conditional expectation. Proof details in Appendix A.5. Remark 4 (Deal with Q1). Theorem 2 indicates that one can control the generalization performance of ICL via controlling the implicit gradient norm along the entire ICL implicit GD trajectories. Specifically, modulating the norm of [G t ] L 1 or [∆W t ] L 1 may enhance performance when utilizing ICL. Note that controlling implicit gradient norm can also control the magnitude of the trace of implicit gradient noise covariance. This elucidates why weight pruning through SVD, even if it only alters a single weight matrix, can confer advantages on the performance of Transformers in ICL inference. We will present an example below demonstrating how weight pruning can affect the norm of G t or ∆W t , thereby influencing the expected generalization error. Additional examples provided in Appendix A.6. This is also why, as illustrated in Figure 1 , the highest model performance sometimes occurs at a clipping rate of 0.995. Furthermore, this could elucidate the utility of normalization in Transformers. However, weight pruning also impacts the expressive power, implying that increased pruning does not invariably lead to better outcomes. This clarifies why the highest model performance may also occur at clipping rates lower than 0.995.Example 1 (Prune W Q , W K , W V ). Consider ∆W k = N i=1 W k V h k-1 i ⊗ W k K h k-1 i W k Q when t = k.And we primarily consider the changes in a upper bound written asU B(∥∆W k ∥ 2 F ) ≜ N i=1 W k V h k-1 i ⊗ W k K h k-1 i 2 F W k Q 2 F .Let r represents the remained rank and δ represents the potential noise consisting of parts with small singular values, that is,W k V = W k Vr + δ V = U V :r Σ V :r (V V :r ) T + δ V , the same operation is applied to W k Q and W k K as well. Then we have W k V h k-1 i 2 2 = (W k Vr + δ V )h k-1 i 2 2 = W k Vr h k-1 i 2 2 + 2(W k Vr h k-1 i ) T (δ V h k-1 i ) + δ V h k-1 i 2 2 = W k Vr h k-1 i 2 2 + δ V h k-1 i 2 2 ≥ W k Vr h k-1 i 2 2 , where W k Vr = U V :r Σ V :r (V V :r ) T and δ V = U V r: Σ V r: (V V r:) T , and U :r , U r: are orthometric (properties of SVD), and further haveU B(∥∆W k ∥ 2 F ) ≥ N i=1 W k Vr h k-1 i ⊗ W k K h k-1 i 2 F W k Q 2 F (*) ≥ N i=1 W k Vr h k-1 i ⊗ W k Kr h k-1 i 2 F W k Q 2 F ≥ N i=1 W k Vr h k-1 i ⊗ W k Kr h k-1 i 2 F W k Qr 2 F = U B(∥∆W k (r)∥ 2 F ), (**)where Eq. (*) is by6 ∥vec(A) ⊗ vec(B)∥ F = i j |vec(A) i vec(B) j | 2 = max i |σ i (vec(A) ⊗ vec(B))|and Eq. (**) is by ∥P∥ F = i σ 2 i (P) for any matrix A, B and∥P∥ F ≥ ∥P(r)∥ F . U B(∥∆W k (r)∥ 2 F ) is the upper bound on ∥∆W k ∥ 2F after using SVD. Remark 5 (Deal with Q2). It is notable that G t is highly correlated with the sequence (∆W 1 , ..., ∆W t ). More precisely, adjusting the weight matrix of the k-th layer, will invariably impact the norm of[∆W t ] L t>k , further influence [G t ] L t>k .When we calibrate the weight matrix of the k-th layer, the span of affected weight updates ∆W t encompasses L -k + 1 matrices, this indicates that the deeper the layer of the adjusted parameters is, the fewer the number of G t affected. It also suggests that tweaks to the deep layers yield a more steadfast influence on the global norm of[G t ] L1 , thereby exerting a steadier influence over generalization performance. The enhanced stability observed in adjusting deeper layers, as demonstrated in our experimental findings in Section 2, can be theoretically explained based on the analysis presented above. Remark 6 (How should Theorem 2 be interpreted?). Expected generalization error (Theorem 2) = population risk (L µ ) -empirical risk (L Hs ). More specifically, on the one hand, Theorem 2 shows that clipping weights controls the F-norm of the implicit gradient([∆W t ] L 1 /[G t ] L 1 ), which helps reduce the expected generalization error. On the other hand, we can evaluate the empirical risk (L Hs ) by assessing the model's performance on the validation set. If the generalization error is known, it is possible to estimate the population risk (L µ ). Therefore, the most challenging aspect is addressing the generalization error. Furthermore, we will illustrate the interpretation of Theorem 2 from the perspectives of pruning methods and the ICL shot numbers. (i) Adjusting the F-norm of([∆W t ] L 1 /[G t ] L 1) could enhance performance when utilizing ICL, suggesting that other weight-based pruning methods may also be effective. For example, magnitude-based pruning [52] directly controls the matrix F-norm. Certainly, there are also some layer-based pruning methods (e.g., drop-layer method [19] ), we discuss in detail in Appendix B.6. (ii) Reducing the ICL shot numbers from N to N ′ can indeed lead to more robust outcomes specifically in the "SVD weight pruning" operation, rather than in the model's overall performance. Experimentally, in Section 2.2, we analyze the effect of different ICL shot numbers. As shown in Figure 2 , with a decrease in the number of shots, the rate of performance collapse in the model slows down after a sharp reduction at the shallow layer. Theoretically,∆W(N ) -∆W(N ′ ) = N i=N ′ +1 W V h i ⊗ W K h i W Q ,suggesting that the implicit gradient for N is expected to be more sensitive than that for N ′ .In this portion, we deal with Q3. We further explore the relationship between model performance improvement and SVD, as illustrated by a simple case study. Case 1. Assuming that W has two unit eigenvectors x 1 , x 2 , based on the properties of eigenvectors:Wx 1 = λ 1 x 1 , Wx 2 = λ 2 x 2 ,and for a vector b on the hyperplane, it can be decomposed into a linear combination of two eigenvectors:b = l 1 x 1 + l 2 x 2 = W l 1 λ 1 x 1 + l 2 λ 2 x 2 = Wx,where l 1 , l 2 are coefficients and λ 1 , λ 2 are eigenvalues. Notice that if λ 1 ≫ λ 2 , when point b moves in the direction of x 1 , the value of l 1 changes but the solution set x changes insignificantly. Conversely, if it moves in the direction of x 2 , the value of l 2 changes and the solution set x changes dramatically, this is an ill-conditioned7 problem whose solution is highly biased under small perturbations.Given that SVD can be considered to play a role in noise reduction in ICL inference, and that this noise can cause significant disturbances to model output. We thus introduce the matrix ill-conditionedness, which can be measured by the Matrix condition number defined as follows.Matrix condition number 8 . Cond(W) = ||W|| p ||W -1 || p . Specifically, the condition number Cond(W) = σmax σmin when p = 2, where σ max , σ min denote the maximum and minimum singular values respectively. Generally, for matrices of the same order, the higher the condition number of a matrix is, the greater its ill-conditionedness will be. In deeper layers, condition numbers tend to be higher. These matrices are ill-conditioned for they satisfy σ max ≫ σ min .Then we analyze the matrix condition number of the models as shown in Figure 3 . We observe that the condition number of deeper layers (especially the last layers) in the model is generally higher, indicating that the condition number may serve as a reference for adjusting the model.Based on all our exciting insights, we find it intuitive to design ICL improvements based on them, especially in downstream tasks. We propose a method where we first select layers with the top-k largest condition numbers and then identify the layer with the largest number among these. We perform a greedy search for the optimal clipping rate ξ * on the validation set and subsequently evaluate it on the test set. This procedure is reported in Algorithm 1 in Appendix C.2.Experimental setup. We conduct experiments for Algorithm 1 on widely adopted benchmark datasets, including SST-2 [39] , AGNEWS [58] , EMOC [8] , MRPC [12] , RTE [5] , CB [11] , COPA [18] . Please refer to Appendix C.1 and C.3 for more detailed dataset and prompt settings. And for models, we use the same models as in Section 2.Experimental results. As Figure 4 shows, the results indicate that Algorithm 1 can visibly influence performance across different language understanding tasks in ICL inference. Specially, the effectiveness of Algorithm 1 in enhancing performance varies not only by the model but also significantly by the task, indicating a potential task-specific and model-specific threshold for the benefits derived from algorithmic enhancements. More notably, Algorithm 1 is a derivative-free optimization method and compresses the model to some extent, providing developers and researchers with an effective approach for handling downstream tasks. We also invite readers to refer to Appendices B. Accuracy/F1 (0-1) ICL(GPT-J-6B) ICL+Algorithm1(GPT-J-6B) ICL(LLAMA2-7B) ICL+Algorithm1(LLAMA2-7B)some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. • The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
conclusion and limitation: 
 In this paper, we show our surprising findings in ICL inference: SVD-based weight pruning can enhance ICL performance both in shallow and deep layers across different module types, and pruning weights in deep layers often results in more stable performance improvements than in shallow layers.We conduct an in-depth theoretical analysis and explanation of these findings. Specifically, we first present the implicit gradient descent trajectories of ICL, afterwards, we give a mutual information based generalization bounds of ICL via full implicit GD trajectories. Based on all our experimental and theoretical insights, we intuitively propose a derivative-free and effective method for downstream tasks in enhancing ICL inference. However, further studies are required on (i) how to extend our generalization theory to a more standard Transformer architecture, (ii) do the results about ICL hold true for tasks beyond natural language processing and (iii) how to minimize the cost of searching the optimal clipping rate. Those will deepen our understanding of the ICL capabilities. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? Answer: [No] Justification: The primary focus of our paper is on the theoretical analysis of the underlying mechanism of a phenomenon we discovered. While we have conducted experiments on various publicly available datasets and different models to support our theoretical findings, we did not include discussions on error bars. Our experiments across different datasets and models reflect to some extent the correspondence between theory and practice. And due to limitations in computational resources, we did not report error bars, confidence intervals, or conduct statistical significance tests. Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? Answer: [Yes] Justification: In Section 3.4, we provide sufficient information on the computer resources. Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? Answer: [Yes] Justification: The research conducted in the paper conform with the NeurIPS Code of Ethics in every respect, and we make sure to preserve anonymity. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 10. Broader Impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: Our work mainly focuses theoretically analysing the underlying mechanism of a phenomenon that we found. For positive societal impacts, understanding the underlying mechanism can contribute to use ICL. For negative societal impacts, the understanding of the mechanism can not be used in harmful ways. 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: Our paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]  
 
 Appendix: A.1 Proof of Lemma 1Proof. Given that numerous studies [21, 10, 35] describe ICL with attention can be connected to a meta-optimizer which produces implicit gradient, we show the dual form between linear attention and gradient descent. First, consider a very simple linear layer,F (x) = Wx,where W ∈ R dout×din is the projection matrix. Given training inputs [x i ] N i=1 ∈ R din with their labels [y i ] N i=1 ∈ REQUATIONwhere ⊗ denotes the outer product according to the chain rule of differentiation. On the other hand, this process can be associated with linear attention,EQUATIONwhere[k i ] N 1 ,[v i ] N 1 ∈ R din representthe key and value vectors respectively, forming the key and value matrix K, V ∈ R din×N in the attention mechanism. Based on Eq.( 6), rewrite Eq.( 5) as:EQUATIONThen let's go back to the ICL process and approximate standard attention Eq.( 2) as a relaxed linear attention mechanism by removing the Softmax operation and scale factor:EQUATIONWhere W 0 = 0, and W V H s is regarded as the meta-gradient of ICL, which is used to generate the implicit gradient matrix ∆W icl to act on the final feature output.Inspired by Ren and Liu [35] , by connecting Softmax Attention with Kernels, we can interpret ICL as a gradient descent process in contrast learning pattern. For simplicity, as in Section 3.1, scale factors are removed as follows:EQUATIONwhere exp(•) is element-wise. Following Choromanski et al. [9] and Ren and Liu [35] , a Softmax kernel function K Softmax : R dout × R dout → R + is defined as:EQUATIONwhereK gauss (x, y) = exp -∥x-y∥ 2 2is the Gaussian kernel with a variance σ 2 = 1. According to Mercer's theorem [29] , there exists some mapping function ϕ satisfying K Softmax (x, y) = ϕ(x) T ϕ(y). Thus, rewrite Eq.( 9) as:EQUATIONthen, connect with Eq.( 2) we have,EQUATIONwhereD ′ = 1 N ϕ(W K H s ) T ϕ(W Q h N +1 ) + ϕ(W K h N +1 ) T ϕ(W Q h N +1) represents a constant for the normalized attention scores. It can be considered that the mapping function ϕ, or rather the effect of the Softmax function, is to project the original features into a higher-dimensional space to capture more profound features. Subsequently, learning and meta-optimization are conducted under this new feature space. It is also noteworthy that rank(∆W ′ icl ) ≤ rank(H s ) ≤ N .The LLMs based on the Transformer architecture not only contain an Attention mechanism, but also include feed-forward (MLP) layers. It can be estimated that the feed-forward layers account for approximately two-thirds of the parameters in a Transformer model [45, 44] . Thus, MLP layers are essential for LLMs and there has been considerable work hypothesizing and analyzing the model's feed-forward layers. For example, Geva et al. [17] , Qiu et al. [33] show that feed-forward layers in Transformer based language models operate as key-value memories, Tian et al. [41] propose Joint MLP/Attention (JoMA) dynamics by integrating out the self-attention layers in Transformers, analyzes joint training of MLP and self-attention layers, and qualitatively explains dynamics of multi-layer Transformers. Based on the inspiration above, we take MLP into consideration and rewrite Eq. (3) as:ĥN+1 = h N +1 + MLP(W V HM(W K H) T W Q h N +1 ) = h N +1 + W out relu(W in W V HM(W K H) T W Q h N +1 ), where W V ,W K ,W Q ∈ R (dim1)×(dout+din) , W in ∈ R (dim2)×(dim1) , W out ∈ R (dout+din)×(dim2)are projection matrices and relu(•) is the activation function. And we can further relax the activation function for ease of qualitative analysis:ĥN+1 = h N +1 + W out W in W V HM(W K H) T W Q h N +1 = h N +1 + W M LP W V HM(W K H) T W Q h N +1 ,where W M LP ∈ R (dout+din)×(dim1) , which can be seen as a dimensional adaptation. And then do the similar derivation that we do in Appendix A.1, we can get∆W ′′ icl = W M LP N i=1 W V h i ⊗ W K h i W Q .Proof. Similar to A.1, Pay attention to the test token when i = N + 1,h 1 N +1 = h 0 N +1 + ∆W 1 h 0 N +1 , h 2 N +1 = h 1 N +1 + ∆W 2 h 1 N +1 = h 0 N +1 + ∆W 1 h 0 N +1 + ∆W 2 (1 + ∆W 1 )h 0 N +1 , h 3 N +1 = h 2 N +1 + ∆W 3 h 2 N +1 = h 0 N +1 + ∆W 1 h 0 N +1 + ∆W 2 (1 + ∆W 1 )h 0 N +1 + ∆W 3 (1 + ∆W 1 + ∆W 2 (1 + ∆W 1 )) h 0 N +1 , ......EQUATIONOn the one hand, Eq.( 13) showsG t = ∆W t (1 + W t-1 ), on the other hand, ∆W t ≜ N i=1 W t V h t-1 i ⊗ W t K h t-1 i W t Q , which is only depend on ([h 0 i ] N 1 , [W t K ] L 1 , [W t Q ] L 1 , [W t V ] L 1 ).t K ] L 1 , [W t Q ] L 1 , [W t V ] L 1 )implies that G t is only dependent on W t-1 and H s .And the prediction can be read from the corresponding position in L-th layer output h L N +1 as follows,h L N +1 = * y pred = h 0 N +1 + W L h 0 N +1 = (1 + W L )x N +1 mask .The origin form of the mutual information based bound is predicated on a sample-specific MI, which quantifies the shared information between the output variable W and the input sample set H s . The following lemma shows the result: I(W L ; H S ) ≤ L t=1 I(-G t + C 1/2 t N t ; H S |W t-1 ). Let -G t + C 1/2 t N t ≜ G t .Proof. Recall Assumption 1 Eq.( 4), Wang and Mao [51] get,EQUATION= I(W L-1 ; H S ) + I(-G L + C 1/2 L N L ; H S |W L-1 ) ≤ L t=1 I(-G t + C 1/2 t N t ; H S |W t-1 ) + I(W 0 ; H S ) = L t=1 I(-G t + C 1/2 t N t ; H S |W t-1 ).where Eq. ( 14) is by the data processing inequality (e.g., Z -(X, Y ) -(X + Y ) form a Markov chain then I(X + Y, Z) ≤ I(X, Y ; Z)), Eq. ( 15) is by the chain rule of the mutual information, and learning rate η is dropped since mutual information is scale-invariant. I(W 0 ; H S ) = 0 for W 0 is independent of all other random variables in Theorem 1 and Assumption 1.Besides, we present the variational representation of mutual information: Lemma 4. (Polyanskiy and Wu [32, Corollary 3.1.] ). For two random variables X and Y , we have I(X; Y ) = inf P E X [D KL (Q Y |X ||P )],(P Gt|Wt-1 ∥P Ñt|Wt-1 ) < ∞, we have I(G t ; H s |W t-1 ) = E Wt-1 inf P Ñt |W t-1 E Wt-1 H S D KL (Q Gt|Hs,Wt-1 ∥P Ñt|Wt-1 ), where the infimum is achieved whenP Ñt|Wt-1 = Q Gt|Wt-1 .The KL divergence may be viewed as an estimate of the sensitivity of the full batch implicit gradient to a specific demonstration sample H s = H s .From Lemma 5, every choice of P Ñt|Wt-1 gives rise to an upper bound of the MI of interest viaI(G t ; H s |W t-1 ) ≤ E Wt-1 E Wt-1 Hs D KL (Q Gt|Hs,Wt-1 ∥P Ñt|Wt-1) . Same to Wang and Mao [51] , choose an isotropic Gaussian prior P Ñt|Wt-1 = N ( gt , σ 2 t I d ), where both gt and σ t are only allowed to depend on W t-1 under Theorem 1, and optimize the KL divergence in Lemma 5 over σ t for a fixed gt . Additionally, Under Theorem 1 where we define the implicit GD trajectories of ICL, assume C t is a positive-definite matrix, for any t ∈ [L], we have,EQUATION)where Eq. ( 16) is by the KL divergence between two Gaussian distributions:D KL (p∥q) = 1 2 log det(Σq) det(Σp) -k + (µ p -µ q ) T Σ -1q (µ p -µ q ) + tr(Σ -1 q Σ p ) , and Eq. ( 17) is due to the fact thatG T t G t = tr(G t G T t ) and log det(C t ) = tr(log C t ) when C t is a positive-definite matrix. Let A 1 (t) = E Hs [∥G t -gt ∥ 2 2 + tr {C t }] and A 2 (t) = E Hs [tr {log C t }]and fix gt can rewrite Eq. ( 17) as,1 2 inf σt≥0 1 σ 2 t A 1 (t) + d log σ 2 t -d -A 2 (t) = 1 2 d log A 1 (t) d - 1 2 A 2 (t),where the optimal σ * = A1(t) d when we take the derivative of σ t . Then combine Lemma 2 and Lemma 3 , we can finish the proof of the following lemma. Lemma 6. Under the conditions of Theorem 1, assume the implicit gradient noise covariance C t is a positive-define matrix, the loss ℓ(w, h) is R-subGaussian for any w ∈ W ∈ R d . For any t ∈ [L], let gt be any constant vector for a given w t-1 , thenerror ≤ R 2 N L t=1 E Wt-1 [d log(A 1 (t)/d) -A 2 (t)] = R 2 N L t=1 E Hs Wt-1   d log    vec(∆W t (1 + t-1 j=1 G j )) -gt 2 2 + tr{C t } d    -tr{log C t }   ,whereA 1 (t) = E Hs Wt-1 ∥G t -gt ∥ 2 2 + tr{C t } , A 2 (t) = E Hs Wt-1 [tr{log C t }], vec(G t ) = G t , tr{•}denotes the trace of a matrix and E X Y is the conditional expectation. Further, if we let gt = 0, reverse the process of flattening the weight matrix into a vector form, and by Eq. ( 19) we obtainvec(∆W t (1 + t-1 j=1 G j ))) -gt 2 2 + tr{C t } = ∆W t (1 + t-1 j=1 G j ) 2 F + tr{C t } ≤ ∥∆W t ∥ 2 F • 1 + t-1 j=1 G j 2 F + tr{C t }.Then plug everything into Lemma 6, we conclude the proof.First, we will present several expressions that will be utilized later. For any matrix A and B, we have, ∥vec(A) ⊗ vec(B)∥ F = i j |vec(A) i vec(B) j | 2 = max i |σ i (vec(A) ⊗ vec(B))| (18) ∥AB∥ F ≤ ∥A∥ F ∥B∥ F (19) ∥A + B∥ F ≤ ∥A∥ F + ∥B∥ F (20) Same in Lemma 1, ∆W icl = N i=1 W V h i ⊗ W K h i W Q , let). Suppose we decompose W Q by SVD, W Q = W Qr + δ Q = U Q :r Σ Q :r (V Q :r ) T + δ Q , let z ≜ N i=1 W V h i ⊗ W K h i for simplicity. ∥∆W icl ∥ F = zU Q :r Σ Q :r (V Q :r ) T + zδ Q F ≤ ∥zW Qr ∥ F + ∥zδ Q ∥ F, where the inequality part takes advantage of the Eq. ( 20) , so the upper bound on ∥∆W icl ∥ F is decreasing when using SVD. Example 3 (Prune W V or W K ). Suppose we decompose W V by SVD, W V = W Vr + δ V = U V :r Σ V :r (V V :r ) T + δ V ,EQUATIONwhere Eq. ( 21) is by Eq. ( 18), so the upper bound on ∥∆W icl ∥ F is also decreasing. The same is true for decomposing W K . Example 4 (Prune W M LP ). Suppose we decompose W M LP by SVD,W M LP = W M LPr + δ M LP = U :r Σ :r V T :r + δ M LP , let z ≜ N i=1 W V h i ⊗ W K h i W Q for simplicity. ∆W ′′ icl F = U :r Σ :r V T :r z + δ M LP z F ≤ ∥W M LPr z∥ F + ∥δ M LP z∥ F , where recall Appendix A.3, ∆W ′′ icl = W M LP N i=1 W V h i ⊗ W K h i W Q .And the inequality part takes advantage of the Eq. ( 20) , so the upper bound on ∆W ′′ icl F is decreasing when using SVD.Our notation here follows the related work [2] , which explains: Note that the prompt is asymmetric since the label for x N +1 is excluded from the input. To reflect this asymmetric structure, the mask matrix M is included in the attention. More specifically, if you pay attention to the (N + 1)-th item, M is supposed to represent a causal mask (For H ∈ R (dout+din)×(N +1) , HM = (h 1 , ..., h N , h N +1 )M = (h 1 , ..., h N , 0) = H s ). Besides, this mask method is used in GLM training [13] : Part A tokens can attend to each other, but cannot attend to any tokens in B. Part B tokens can attend to Part A and antecedents in B, but cannot attend to any subsequent tokens in B. So it is reasonable for Eq.( 1) and Eq.( 2) use the same mask.Indeed, there is no universally accepted definition for the terms "deep" and "shallow" as their interpretation can be subjective and dependent on the reference frame (e.g., model size). Intuitively, in this paper, our definition is similar to that of work [50] : "shallow" layers refer to those closer to the input, while "deep" layers are closer to the output. In Figure 1 and Figure 2 (Section 2), "shallow" typically denotes the first few layers, and "deep" denotes the last few layers of the network.In our theory, the example (pruning only W K or W V ) is provided in Appendix A.6, demonstrating how weight pruning can affect the norm of[∆W t ] L 1 /[G t ] L t .Thus, it may confer advantages on the performance of Transformers in ICL inference. However, in Theorem 2 (Remark 5), it also suggests that modifications to the shallow layers have a less steady impact. Additionally, please review the supplementary experimental results provided below. (We choose key projection matrix W K and select the 3-layer (large matrix condition number in Figure 3 ) of GPT-J-6B, others are the same to Section 2.1). * = 0 for each ξ in C do M ′ = Clip(M, l, ξ, O) score = Evaluate(M ′ , V, H s ) if score > score * then ξ * = ξ score * = score end if end for M * = Clip(M, l, ξ * , O) test score = Evaluate(M * , T , H s ) Output test score on TThe details. Firstly, the details of the algorithm can be reviewed in the code provided. Specifically, the set of clipping rate candidates is predefined. In our study, the clipping rate candidates are set as shown in Figures 1 and 2 : [0, 0.1, 0.5, 0.75, 0.9, 0.95, 0.99, 0.995]. Besides, we analyze the impact of different hyperparameters through comparative experiments (details in Section 2 and Section 3.4), as detailed below. (1) Clipping rate ξ:We search for the optimal ξ in the predefined clipping rate candidates. (2) Predefined module O: The module containing the target pruning weights, which can be chosen from [k_proj, q_proj, v_proj, out_proj, f c_in, f c_out, all, mlp, attn]. (3) Selected layer l : The layer containing the target pruning weights. For examlpe: In Section 2, we mainly focus on comparing the impact of weight pruning to the first two and the last two layers of the model. ( 4) ICL shot number : The demonstration number in ICL, we analyze the effect of different ICL shot numbers in Section 2.2.For experiments, we consider numerous classification datasets, including: SST-2 [39] , AGNEWS [58] , EMOC [8] , MRPC [12] , RTE [5] , CB [11] . We also include a multiple-choice dataset COPA [18] . SST-2. SST-2 focuses on sentiment analysis, specifically the task of determining the sentiment of movie reviews. The dataset consists of sentences labeled as having a positive or negative sentiment. These sentences are divided into a training set, a validation set, and a test set. The training set includes about 67,349 sentences, whereas the validation set contains around 872 sentences. The test set comprises approximately 1,821 sentences without labels. We random sample N (10) data from the train set as demonstration sample, 8,000 data from the train set as validation set to search the optimal clipping rate, the rest 59,000+ data as test set.AGNews. The AGNews dataset is a collection designed for text classification tasks, specifically for news categorization. It consists of news articles gathered from the AG's corpus of news on the web, which is categorized into four main topics: World, Sports, Business, and Science/Technology. In terms of structure, the AGNews dataset comprises approximately 120,000 training samples and 7,600 test samples. We random sample N (8) data from the train set as demonstration sample, 8000 data from the train set as validation set to search the optimal clipping rate, the 7,600 test samples as test set.EmoC. The EmoC dataset focuses on contextual emotion detection in text. It consists of short text conversations extracted from three-turn English Tweets. The dataset is annotated with four emotion labels: happiness, sadness, anger, and others, we relabel the dataset with happiness and others. The training set includes 30,160 samples, while the test set comprises 5,509 samples. We random sample N (10) data from the train set as demonstration sample, 5,000 data from the train set as validation set to search the optimal clipping rate, the 5,509 test samples as test set.MRPC. The MRPC dataset focuses on paraphrase detection in text. It consists of sentence pairs automatically extracted from online news sources. The dataset is annotated with binary labels indicating whether the sentences are paraphrases of each other. The training set includes 3,668 pairs, the validation set includes 408 pairs, while the test set comprises about 1,725 pairs. We random sample N (10) data from the train set as demonstration sample, the 408 pairs from validation set to search the optimal clipping rate, the 1,725 test samples as test set.Recognizing Textual Entailment (RTE) task, which involves assessing the relationship between a pair of sentences-a premise and a hypothesis. The objective of the task is to determine whether the hypothesis can be logically inferred from the premise. Specifically, the model must evaluate whether the relationship between the two sentences is one of "entailment," where the content of the hypothesis is directly or indirectly supported by the premise, or "non-entailment," which includes both contradiction and neutrality. In RTE from SuperGLUE [47] , the training set includes about 2,490 items, whereas the validation set contains around 277 items. We random sample N (10) data from the train set as demonstration sample, the remain items from train set to search the optimal clipping rate, the 277 validation samples as test set.CB. The CommitmentBank (CB) dataset is part of the SuperGLUE benchmark and focuses on textual entailment with an emphasis on pragmatic inference. The task involves determining whether a hypothesis can be logically inferred from a given text, which in this dataset, typically comprises a premise followed by a hypothesis. The training set includes about 250 items, whereas the validation set contains around 56 items. We random sample N (15) data from the train set as demonstration sample, the remain 235 items from train set to search the optimal clipping rate, the 56 validation samples as test set.COPA. The Choice of Plausible Alternatives (COPA) dataset is specifically designed to evaluate causal reasoning abilities in natural language processing models. This dataset presents a task where models must determine causal relationships within simple scenarios. Each question in COPA consists of a premise and two possible choices, one of which is the correct cause or effect of the premise. COPA's dataset is relatively straightforward and consists of 500 questions split evenly into training (400) and validation (100) sets. We random sample N (10) data from the train set as demonstration sample, the 200 items from train set to search the optimal clipping rate, the 100 validation samples as test set. As mentioned in Garg et al. [16] 's work, the trained model is able to learn unseen linear functions from in-context examples with performance comparable to the optimal least squares estimator, this can be seen in Figure 5a . More importantly, the number of in-context examples (shot number) plays a significant role. In this case, the error approaches zero only when the shot number is greater than or equal to d. This indicates that the implicit gradients of ICL are influenced by the shot number, and there exists a threshold N , when the actual shot number b is below this threshold, the implicit gradients are noisy. On the other hand, following the same setting of [16] in figure 5a , we compare the performance of the Trained Transformer at different train steps as shown in figure 5b . This indicates that the performance of the model varies with the number of train steps, which also means that the implicit gradients of ICL generated by the model are influenced by the extent of its pre-training.In conclusion, actual implicit gradient descent involves noise, which primarily stems from the shot number and the degree of model pre-training.C.5 The Detailed Results of Algorithm 1 Justification: The paper discusses enhancements in in-context learning (ICL) performance for pre-trained large language models (LLMs), particularly focusing on the effects of SVDbased weight pruning. Key contributions made include: What is the phenomenon? Why this phenomenon? How to better use this phenomenon?Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).Please refer to Appendix B.1 for an explanation on how Eq.(1) and Eq.(2) can utilize the same mask.In real-world scenarios, the residual connection module in Transformers is indispensable.Consider the continuous SDE: dW = -∇L Hs (W )dt + [ηC(W )] 1/2 dθt, where C(W ) is the gradient noise covariance at W and θt is a Wiener process. We can view Eq. (4) as discretization of the SDE.For any vector a and b, rank(a ⊗ b) = 1, so the matrix (a ⊗ b) only has one non-zero singular value. Combined with ||P||F = i σ 2 i (P) and singular value is nonnegative, we can get ||a ⊗ b||F = i σ 2 i (a ⊗ b) = maxi[σi(a ⊗ b)], therefore, the unique non-zero singular value will decrease after performing SVD on a and/or b.Now consider a linear system Wx = b, where the solution set x is highly sensitive to the coefficients of W and b. In such cases, the system of equations is termed ill-conditioned.Matrix condition number is an option, any indicator that can guide the control of norms is a potential option. 


Decision: The reviewers generally acknowledge the importance of research on in-context learning (ICL) and largely agree that the proposed method is intuitive and simple and that the manuscript is insightful. It was initially brought up, that the theoretical analysis was limited in that it only explains why stable performance is achievable, but not why better performance can be achieved with SVD-based weight pruning. After clarification in the discussion phase, reviewers appeared to be convinced that the presented analysis was sufficiently detailed and provides an explanation of the empirical results. The authors’ response to the question by reviewer CqwV, on whether dropping a layer entirely may sometimes be the best option compared to pruning a layer’s weights, seems reasonable.
Overall, all reviewers agree that the work meets the standards of a NeurIPS publication and I recommend acceptance.

Reviews: review: paper_summary: paper_summary: This paper discusses the phenomenon: SVD-based weight pruning can increase the in-context learning abilities of transformer based LLMs. In this paper, the authors conduct theorectical analysis by presenting the implicit gradient descent trajectories of ICL and providing the generation bounds visa full implicit gradient descent trajectories. This paper also provide a simple yet effective algorithm to clip the LLM by SVD to enhance ICL inference.
strength_weakness: strength_weakness: 1. More details of algorithms is not shared. e.g. the range / number of clipping rate candidates set. 
2. In experiments result of C.5, the optimal $\xi$ varies a lot across different tasks and different modules. However, this phenomenon is not touched in the theoretical part.
questions: questions: 1. Matrix condition number is an option for the indicator. But could there be more options, such as compute the decreasing rate of eigenvalues? Because when p=2, conditional number only leverages two values among all the eigenvalues.
2. Could authors provide further more clarification why optimal $\xi$ varies, and is there a way to explain this phenomenon under current theoretical framework provided in this paper?
limitations: limitations: No
paper_summary: paper_summary: This paper demonstrates that (1) SVD-based weight pruning can sometimes achieve better in-context learning performance, and (2) pruning weights in deeper layers often results in more stable outcomes compared to shallow layers. The authors explain their findings through theoretical analysis and propose an intuitive matrix condition number-based weight pruning algorithm to achieve both stable and improved performance.
strength_weakness: strength_weakness: Despite adopting various simplifications (such as using a linear attention transformer without MLP and layer normalization, treating each in-context example as a single vector, implementing attention masks for query tokens, and using meta-gradients for in-context learning) in their theoretical analysis, the results are still limited. They only explain why SVD-based weight pruning can achieve "stable" performance, leaving the more intriguing question of why transformers can achieve "better" performance with pruning unclear. Additionally, even with detailed hyperparameter tuning, the effectiveness of Algorithm 1 remains uncertain. Further details are provided in the questions section.
questions: questions: [1] How should Theorem 2 be interpreted? It seems only provide a weak upper bound for in-context learning stableness. Can this also be applied to empirical risk, such as $L_{H_S}$ and $L_\mu$?

 [2] Theorem 2 gives the upper bound for expected generalization error. If we fix N in the constructed transformer and reduce the number of in-context examples to $N'$ in the input sequence, then we can find that while the factor $R^2/N$ remains unchanged in theorem 2 , $\Delta W_t$ will change from $(\sum_{i=1}^N)…$ to $(\sum_{i=1}^{N'})…, where $ ($N' < N$). Based on the analysis across different layers, could this mean that fewer context examples are more robust for SVD weight pruning?

[3] Note that in fig-3, large matrix condition numbers can exist in some modules of shallow layers, such as the attention key (K) in GPT-J-6B . What would be the effect of pruning only a single module in a shallow layer (e.g., the key projection matrix) rather than pruning the entire attention module (including Q, K, and V)?

 [4] In C.5, it's noted that the optimal clipping rate is sometimes very small and varies across datasets. What would happen if we apply the same clipping rate (e.g., 0.95) as used in SST-2 to other datasets?
limitations: limitations: See Weaknesses and Questions. Besides, there seem many mistakes on pages 7 and 8, for example, the equation between line 228 and 229, and the equation for about F-norm in lines 230,231,232. These inaccuracies cast doubt on the overall reliability of the paper's findings. If there are any misunderstandings on my part, please point them out, and I will reconsider my evaluation of this work.
paper_summary: paper_summary: This paper investigates the effect of singular value decomposition (SVD)-based weight pruning on the in-context learning (ICL) performance of large language models. 

The Authors show that SVD-based pruning can enhance ICL performance, with deeper layers showing more stable improvements. 
They provide theoretical analysis to explain these findings, presenting implicit gradient descent trajectories for ICL and deriving generalization bounds. 

Based on their insights, they propose a simple algorithm for enhancing ICL inference in downstream tasks.
strength_weakness: strength_weakness: - The theoretical analysis primarily focuses on linear attention, which may not fully capture the complexities of standard Softmax attention used in most transformer models

- The proposed algorithm is derivative-free, but the search for optimal clipping rates may still be computationally expensive for very large models or datasets

- There is a substantial lack of comparison with other pruning methods: the study focuses on SVD-based pruning but doesn't compare it with other pruning techniques, which could provide context for the method's effectiveness

- Poor language, frequent typos, and grammatical errors are significant issues in this paper. This does not help readability, and would likely be a barrier to publication in its current form.

- An essential part of the paper, which is the discussion of related works is not part of the main text. Furthermore, this discussion is prone to criticism. For example, quoting the seminal paper by Frankle and Carbin as an example of low-rank properties of neural networks is clearly misleading. I think that this discussion should be an essential part of the main text, and should also be substantially revised in order to avoid conceptual confusions.
questions: questions: I suggest the Authors to address 1) the concerns regarding a better framing of the work in the current literature. In particular, there is a large body of evidence that is growing on the resilience of LLMs to pruning (see for example "The Unreasonable Ineffectiveness of the Deeper Layers", Gromov et al, https://arxiv.org/pdf/2403.17887, and references therein), and 2) the quality of writings by proofreading it together with a native speaker.
limitations: limitations: Limitations are discussed in the final section (4).
paper_summary: paper_summary: The authors provide a theoretical perspective on the stability of in context learning via implicit gradient descent trajectories. Ultimately, the analysis suggests that high condition numbers of the weight matrices belonging to layers with a high index can be pruned in order to achieve a model which performs better on ICL tasks.
strength_weakness: strength_weakness: - It would be good to define deep and shallow, as these are subjective terms depending on the reference frame.
- Figure 1 cpation says: "We operate on the whole of MLP or ATTN." What does this mean?
- If as figure 1 states, you can clip 99.5% of the original weights, what happens if you just drop that layer entirely? Recent work has shown that the deeper layers can be completely dropped without much effect. [1]
  - I cannot see much benefit gained from pruning part of the weights with SVD when it seems that the in nearly all cases, the benefit can be had by dropping the layer entirely.
  
- Is the mask on L138 supposed to represent a causal mask? If so, I do not think the notation is correct, as the Identity matrix would only have $N$ binary values which is much less than is needed for a causal mask.
- How can equation 1 and 2 use the same mask?
- Example 1 appears to be incorrect:
  - There is no parentheses around $W_{V_r}^k + \delta_V h_i^{k-1}$ in the first line.
  - The triangle inequality seems to say that line 2 $\geq$ line 1
  - Given the above, I do not see what conclusion can be drawn from this equation.
  - Have I missed something here?
questions: questions: - I don't believe the clipping process was adequately explained. Once the SVD operation is done, do you clip starting from the largest singular value? or starting from the smallest?
limitations: limitations: The limitations have been discussed.

Comments: title: Sincerely thank you again
Corrected
Thank you and looking forward to your reconfirmation of the score
Reviewer reply
Thank you for your response
Clarification
Thank you
Thank you for your response
Main focus of our work
Thank you
Thank you
Looking forward to your reply
Thank you
Official Comment by Reviewer Zh6a
comment: > Dear Reviewer Nxqr,
>
> We are deeply grateful for your positive feedback and for the time and effort you have invested in your thoughtful review. Your valuable insights have significantly enriched our work. Thank you once again for your invaluable contribution.
>
> Warmest regards.
Dear Authors, sorry, it was a mistake. The score is now the intended one.
> Dear Reviewer Nxqr,
>
> Thank you sincerely for your time and valuable feedback. We are grateful for your willingness to raise the score following our discussion. However, we noticed that the revised score appears to be recorded as **1 point**. Could there have been an error in this update? We would appreciate your confirmation on this matter.
>
> Best regards.
Dear Authors,
I apologize for being late in my reply.

I sincerely thank the Authors for their clarification. After going through the discussion with the other reviewers, and also taking into account the level of confidence of my rewiew I decided to raise my score.
> Dear Reviewer CqwV,
>
> Thank you for your time and clarifying your points. We deeply apologize for any misunderstanding that may have occurred. We truly value your insightful suggestion, as it provides us with an important opportunity to further clarify this matter. Below, we will address specific questions to ensure comprehensive understanding.

>**Q1:** What would happen if that layer was dropped entirely. It simply may be the case that clipping a layer at 100% (i.e. dropping the layer) may in fact be the **best option**.

>**A:** This is indeed a very meaningful question! You're absolutely right; dropping the layer **could be the best option specifically for optimizing generalization error**. Here's a detailed analysis:
>
>(1) In our theoretical framework, we model **each layer** of the Transformer do a **single iteration of implicit gradient descent** (ICL). This scientific analysis references [1, 2, 3].
>
>(2) In **Theorem 2**,  **L**-layer Transformer: 
>
>Expected generalization error = population risk ($L_{\mu}$) - empirical risk ($L_{H_s}$) 
>
>Expected generalization error = $\sqrt{\frac{R^2}{N}\sum_{t=1}^Ld\log (\lVert\Delta W_t\rVert_F^2 \lVert G_t\rVert_F^2)}$ (show the main part). If you drop the entire layer, it will
>
>change from $\sum_{t=1}^L$ to $\sum_{t=1}^{L-1}$. Therefore, dropping the layer may in fact be the **best option for generalization error**.
>
>(3) However, according to the traditional statistical learning viewpoint, performance can be **bounded by the sum of optimization error and generalization error** (Please see global rebuttal for "How should Theorem 2 be interpreted?"). 
>
>Thus, during the pruning process, there is a **trade-off** between optimization and generalization. Therefore, as pruning increases, the model's performance tends to **first improve and then decline** (the provided reference's method does not harm but also does not improve), as demonstrated in the experiments shown in **Figure 1**.
>
>**In conclusion, your suggestion (dropping the entire layer) can be a potential method (best option for generalization error) in our theoretical framework, but it may not necessarily be the best option for model performance.**
>
>[1] Johannes von Oswald et al. Transformers learn in-context by gradient descent. ICML 2023. 
>
>[2] Ekin Akyiurek et al. What learning algorithm is in-context learning? investigations with linear models. ICLR 2023.
>
>[3] Damai Dai et al. Why can gpt learn in-context? language models secretly perform gradient descent as meta-optimizers. ACL(Findings) 2023. 

>**Q2:** What is the predefined clipping rate set which is mentioned in algorithm 1?

> **A:** Firstly, the details of the algorithm can be reviewed in the **code** provided. Specifically, the set of clipping rate candidates is predefined. In our study, the clipping rate candidates are set as shown in **Figures 1 and 2**: [0, 0.1, 0.5, 0.75, 0.9, 0.95, 0.99, 0.995]. Besides, we analyze the impact of different hyperparameters through comparative experiments (details in **Section 2** and **Section 3.4**), as detailed below (We will provide further explanations in subsequent versions).
>
> - Clipping rate $\xi$：We search for the optimal $\xi$ in the predefined clipping rate candidates.
>
> - Predefined module $\mathcal{O}$ : 
>
>   The module containing the target pruning weights, which can be chosen from 
>
>   ['k_proj', 'q_proj', 'v_proj', 'out_proj', 'fc_in', 'fc_out', 'all', 'mlp', 'attn']
>
> - Selected layer $l$ : The layer containing the target pruning weights. For examlpe: In **Section 2**, we mainly focuse on comparing the impact of weight pruning to the **first two and the last two layers** of the model.
>
> - ICL shot number : The demonstration number in ICL, we analyze the effect of different ICL shot numbers in **Section 2.2**.
I want to be clear in stating that I never asked for a direct comparison with the work I cited. I merely suggested that the results from there may imply that a high clipping rate may suggest that there is limited usefulness in that layer, and it might be dropped entirely without harming performance at all. The work from [1] shows that even with no finetuning, 20-30% of layers can be dropped with virtually no effect on the downstream task ([1] - Figure 1).

> Consequently, we did not compare our approach with other pruning methods

I did not ask for a comparison.

> We sincerely ask, why do you think the remaining rank (i.e., 4096*0.5%>20) after pruning is no longer important.

I never stated it wasn't important, I asked what would happen if that layer was dropped entirely. It simply may be the case that clipping a layer at 100% (i.e. dropping the layer) may in fact be the best option.

What is the predefined clipping rate set which is mentioned in algorithm 1? I do not see it defined anywhere within the paper.
> Dear Reviewer itEx,
>
> Thank you very much for your positive feedback. We truly appreciate your acknowledgment that our responses have successfully addressed your previous questions. Your continued support and positive rating of the paper are immensely encouraging. Thank you once again for your thoughtful review and valuable insights.
I apprecitated the authors' responses have successfully addressed my previous questions. I will hold my positive rating on this paper.
> Additionally, please allow us to underscore the **main focus of our work**, as outlined in the global rebuttal.
>
> Our primary objective is to provide a general **theoretical framework** that reveals the underlying mechanism behind the phenomenon that SVD-based weight pruning can enhance ICL performance.  Based on our theoretical insights, we can **design new algorithms** to enhance ICL performance. Consequently, we did not compare our approach with other pruning methods. Algorithm 1 is presented solely to illustrate how theoretical analysis can guide experimental procedures effectively.
>
> ( **Reviewer Zh6a** precisely pinpointed the key points: theoretical analysis can be applied to design new algorithms like algorithm 1 in this paper）
>Dear Reviewer CqwV,
>
>Thank you for your reply and for resharing the reference.

> Firstly, let's briefly analyze this reference and make some comparisons:
>
> **Abstract:** (you provided) We empirically study a simple **layer-pruning** strategy for popular families of open-weight pretrained LLMs, finding minimal **degradation** of performance on different question-answering benchmarks until after a large fraction (up to half) of the layers are removed. To prune these models, we identify the optimal block of layers to prune by considering similarity across layers; then, **to “heal” the damage, we perform a small amount of finetuning**......
>
> - **Cp1:** The pruning method in the reference you provided is '**layer-based** pruning', while ours is '**weight-based** pruning'.
> - **Cp2:** The pruning method in the reference you provided causes **degradation** in model performance, whereas ours leads to **improvement**.
> - **Cp3:** The pruning method in the reference you provided may require **fine-tuning to to “heal” the damage**, while ours is **gradient-free**.

> Next, let's address your questions:
>
> **Q1:** How would it be challenging to adjust the weights for another downstream task? 
>
>  **Qa:** As the weights are merely skipped and not totally deleted from the model, I do not understand how this would affect other tasks directly, as they could still be used in the future if necessary

> As shown in **Cp1** and **Cp3**, compared to removing layers and fine-tuning, only weight pruning is easier to recover from and adapt to another task. Moreover, without the need for gradient updates, this is a significant advantage of pruning that cannot be overlooked.

> **Q2:** Given the fact that many models use stochastic depth, and the above reference suggests that dropping a significant portion of the layers has a minimal effect. It would seem that pruning 99.5% of the weights might be almost identical to just skipping them altogether.
>
> **Qb:** this is an assumption that has not been backed up by any data as far as I can tell.

> As  shown in **Cp2** , although the above reference suggests that dropping a significant portion of the layers has a minimal effect, **the effect is negtive (degradation & need finetuning to heal damage)**. In contrast, the pruning method under our theoretical framework **does not require fine-tuning and results in performance improvement**.
>
> Regarding what you mentioned about 'It would seem that pruning 99.5% of the weights might be almost identical to just skipping them altogether'.
> Typically, the dimensions of a model number in the thousands (for instance, GPTJ is 4096). **We sincerely ask, why do you think the remaining rank (i.e.,  4096*0.5%>20) after pruning is no longer important**?
>

>Once again, we thank you for your time and comments.
Thank you for the response. I apologize about the missing reference, it must have erroneously been excluded. The reference is:

[1] Gromov, A., Tirumala, K., Shapourian, H., Glorioso, P., & Roberts, D. A. (2024). The unreasonable ineffectiveness of the deeper layers. arXiv preprint arXiv:2403.17887.

I do not understand the following response above:

> (ii) As stated in Theorem 1, from the perspective of ICL (In-Context Learning) and gradient descent, each attention layer corresponds to an implicit gradient descent step (L layers with Implicit Gradient Descent Trajectories $[\Delta W_t]_1^L$/$[G_t]_t^L$).

>Therefore, if we just drop the entire layer, it will result in two main issues: (a) It will become challenging to adjust the weights for another downstream task. (b) The implicit gradient update will be reduced by one step, generally leading to a decline in model performance.

How would it be challenging to adjust the weights for another downstream task? Given the fact that many models use stochastic depth, and the above reference suggests that dropping a significant portion of the layers has a minimal effect. It would seem that pruning 99.5% of the weights might be almost identical to just skipping them altogether. 

for a), As the weights are merely skipped and not totally deleted from the model, I do not understand how this would affect other tasks directly, as they could still be used in the future if necessary.

for b) this is an assumption that has not been backed up by any data as far as I can tell.
> Dear Reviewer Nxqr,
>
> We sincerely appreciate your time and effort in reviewing our manuscript and providing valuable feedback.
>
> As the author-reviewer discussion phase nears completion, we wish to confirm whether our responses have effectively addressed your concerns. We provided detailed responses to your concerns a few days ago and hope they have adequately resolved any issues. If you require further clarification or have any additional concerns, please do not hesitate to contact us. We are more than willing to continue our communication with you.
>
> Best regards.
> Dear Reviewer Zh6a,
>
> Thank you for your thoughtful and constructive feedback regarding our manuscript. We deeply appreciate the time you invested in reevaluating our theoretical analyses and for acknowledging their relevance and insight in explaining the experimental results. Your meticulous review has truly enhanced the value of our work. Additionally, regarding the use of 'gradient quality derived from context' (i.e., $G_t$), please allow us to provide some additional clarifications (We will add a detailed description of this in the next version): 
>
> - (1) In **Theorem 1** (Remark 2), $G_t$ is only dependent on $W_{t−1}$ and $H_s$, this is consistent with gradient descent in terms of relevance (Conventionally, gradients in training are only related to the current parameters and the training samples).
>
> - (2) In real-world training scenarios, SGD computes gradient by selecting a small batch of samples per iteration. This approach approximates the true gradient, inherently introducing noise. Similarly, in In-Context Learning, a small subset of samples (context examples) is used to generate implicit gradient (i.e., $G_t$), which also results in the introduction of noise.
>
> Once again, we thank you for your time and the constructive comments that have greatly enriched our work.
I appreciate the author's efforts to address my concerns, and I apologize for misunderstandings on my part. Upon further reflection, I find the theoretical analyses both reasonable and insightful in explaining the experimental results. While I still consider the use of 'gradient quality derived from context' (i.e., $G_t$) somewhat unconventional, I believe the theoretical contributions of this work are enough to explain the main observations presented in the paper. After reevaluating this study, I have decided to adjust the score I previously assigned.