The Cost of Scaling Down Large Language Models: Reducing Model Size Affects Memory before In-context Learning
Authors: Tian Jin
Nolan Clement
Xin Dong
Vaishnavh Nagarajan
Michael Carbin
Jonathan Ragan-Kelley
Gintare Karolina Dziugaite
Venue: ICLR 2024 poster (2024)
Keywords: large language model
scaling
in-context learning
pruning
Primary area: generative models
TL;DR: Moderate down-scaling harms fact recall, and yet the ability to learn from a few input-output examples from context withstands aggressive down-scaling.

Abstract: We study how down-scaling large language model (LLM) size impacts LLM capabilities. We begin by measuring the effects of weight pruning – a popular technique for reducing model size – on the two abilities of LLMs: (a) recalling facts presented during pre-training and (b) processing information presented in context. Surprisingly, we find that existing pruning techniques affect these two abilities of LLMs differently. For example, pruning more than 30% of weights significantly decreases an LLM’s ability to recall facts presented during pre-training. Yet pruning 60-70% of weights largely preserves an LLM’s ability to process information in-context, ranging from retrieving answers based on information presented in context to learning parameterized functions such as a linear classifier based on a few examples. Moderate pruning impairs LLM’s ability to recall facts learnt from pre-training. However, its effect on model’s ability to process information presented in context is much less pronounced. The said disparate effects similarly arise when replacing the original model with a smaller dense one with reduced width and depth. This similarity suggests that model size reduction in general underpins the said disparity.

Published as a conference paper at ICLR 2024
THE COST OF DOWN-SCALING LANGUAGE MODELS: FACT RECALL DETERIORATES BEFORE LEARNING
Tian Jin
Nolan Clement"2
Xin Dong +3
Vaishnavh Nagarajan' Michael Carbin
Jonathan Ragan-Kelley*
Gintare Karolina Dziugaite
MIT CSAIL
MIT Harvard University
*Google Research
Google DeepMind
ABSTRACT
How does scaling the number of parameters in large language models (LLMs) affect their core capabilities? We study two scaling techniques: weight pruning and dense scaling, which is simply training a smaller or larger model. Our study focuses how these techniques affect two core capabilities of LLMs: (a) recalling facts presented during pre- pre-training and (b) processing information presented in-context during inference. By curating suite of tasks that help disentangle these two capabilities, we find a striking difference in how these two capabilities evolve due scaling. Reducing the model size by more than 30% (via either scaling approach) significantly decreases the ability to (a) recall facts seen in pre-training, leading to more than 5% drop in relative accuracy. Yet, a 60-70% reduction largely preserves the various ways the model can (b) process in-context information, ranging from retrieving answers from long context learning parameterized functions from in-context examples, causing less than 5% drop in relative accuracy. The fact that both dense scaling and weight pruning exhibit this behavior suggests that scaling model size has an inherently disparate effect on fact recall and in-context learning.
INTRODUCTION
Scaling up the size of large language models (LLMs) has yielded impressive performance gains on many natural language tasks (Kaplan et al., 2020; Hoffmann al., 2022; Brown et al., 2020; Wei et al., 2022). On the other hand, to deploy language models sustainably, is also critical to scale them down while preserving their end utility. Naturally, there is growing interest in both scaling up and scaling down language models (Kaplan et al. (2020); Hoffmann et al. (2022); Frantar & Alistarh (2023); Jiang et al. (2023); Kurtic al. (2022); Santacroce al. (2023)). Most of these works evaluate size- size-performance trade-offs of scaling through aggregate performance metrics such as perplexity or downstream accuracy on existing benchmarks.
However, we argue that there exist subtle but important effects of scaling that standard metrics alone cannot capture. Earlier works on image models found, for example, that pruning can bias the model against certain label categories (Hooker et al., 2021) improve robustness against noisy training data (Jin et al., 2022) - these effects do not simply manifest the overall accuracy of the model. Taking inspiration from these works, we identify the subtle effects of scaling LLMs (up down) on their capabilities, the ability to perform fundamental tasks that underpin many applications.
Our approach. In this work, study the effects of scaling the number of parameters in an LLM on two fundamentally dichotomous capabilities put forth by Chan et al. (2022b;a): fact recall - the capability to process information stored in weights (such facts seen during pre-training) and in-contex learning - the capability to process information stored in context (such as hints given in the prompt). It is, however, challenging to isolate the evaluation of these capabilities by simply measuring performance on arbitrary downstream tasks - after all, most tasks require both capabilities
"Correspondence afand ingesa! ..f.tedu, moarb ngces edv, gkd=8goog 1e nom.

---

Published as a conference paper at ICLR 2024
TriviaQA
Linear Classification
Exact Match (%)
Exact Match (%)
60
40
40
20
20
0
40
60
80
20
60
80
Sparsity (%)
Sparsity (%)
OPT-13B
OPT-30B
LLaMA-13B
LLaMA-33B
Average
Figure 1: Pruning to moderate sparsity (> 30% sparse) harms fact recall while in-context learning withstands even aggressive pruning (60% sparse). We plot the accuracy versus sparsity for the OPT-13B, OPT-30B, LLaMA-13B, and LLaMA-33B models on TriviaQA dataset for fact recall evaluation (left) and linear classification dataset for ICL evaluation (right). We plot their average performance in red. We color the range of sparsity where accuracy drop is within (relative) 5% of the dense model in green. Accepting (relative) 5% accuracy drop w.r.t. the dense model, the maximum sparsity on fact recall task is 30%, whereas the maximum sparsity ICL task is 60%.
to varying extents to solve. Therefore, we carefully curate suite of benchmarks that help tease apart the performance of any given model on the two capabilities. We then evaluate the aforementioned two capabilities under two different scaling techniques: pruning and dense scaling. For the former, we consider recent scalable pruning methods that remove weights and subsequently update the remaining weights in one shot (Frantar & Alistarh, 2023; Sun et al., 2024); for the latter, we use (separately-trained) dense models with increased reduced width and depth.
Benchmarks. Our curated suite of benchmarks involve four classes of tasks, featuring a gradation of reliance on information stored in weights and in-context. The first three Q&A tasks: (1) closed book Q&A task that necessitates fact recall from pre-training for which the model needs to recall information from the weights, (2) an open book Q&A task for which it suffices to copy information from the context and (3) an overriding Q&A task, where the information provided in-context overrides facts seen during pre-training (Li et al., 2023; Longpre et al., 2021). Finally, to evaluate more sophisticated in-context learning capabilities, we test the model on (4) learning tasks, where the model must learn parameterized classification functions (e.g., linear classifiers) using input-output pairs presented in-context. Overall, we consider book, overriding Q&A and learning tasks as a sequence of tasks that requires increasing levels of in-context learning (ICL) capabilities (See Section 3 for more discussion). We summarize these tasks in Table 1.
Findings. We find surprisingly dichotomous effects of down-scaling on fact recall and ICL. capabili- ties. For fact recall, minimal pruning significantly degrades performance (Section 4) specifically, for the task in our benchmark that relies most heavily on the fact recall capability, removing more than 30% of weights leads to significant (> 5%. relative) accuracy degradation (Figure 1, left). This is surprising, given that prior pruning literature demonstrates significant down-scaling without noticeable change to accuracy: pioneering work (Dong al., 2017) removes 45% of the weights in a ResNet50 model without noticeable accuracy degradation. In contrast, ICL withstands significant pruning (Section 5). Specifically, for the task that relies most heavily the ICL capability, even after pruning 60% of weights, the relative accuracy decrease is less than 5% (Figure 1, right). Our results challenges the hypothesis presented in the seminal work of Brown al. (2020), that scaling up the number of parameters strongly benefits the ICL capability of LLMs. 2 Furthermore, fact recall and ICL shows the same sensitivity and robustness dense scaling, respectively (Section 6). Our work reveals the disparate effects of scaling on LLM capabilities, which notably holds for two substantially different types of scaling techniques: pruning and dense scaling. We hope that researchers and practitioners may apply our findings to:
Please refer Section 4.1, Figure (.(a) of Dong et al. (2017). Notably, Dong et al. (2017) obtained these results similarly by pruming without retraining.
"Sinnee in-context learning involves ahsorbing many skills and tasks within the parameters of the model, it is plausible that in-context learning abilities might show similarly strong gains with scale." " (Brown al., 2020)
2

---

Published as a conference paper at ICLR 2024
Table 1: Tasks employed in experiments.
Context
Reculling Facts from Pre-training
Retrieving Facts from Contexts
Learning patterns from Contexts
Considered
Task Type
Task
Type
ICL
WebQQuestion
Emply
NaturalQuestions
Factual
Q&A
TriviaQA
Empty
TriviaQA (Filtered)
Factual
DisentQA
Overriding
Lineur CLS
Examples
V
Learning
2-layer NN
Examples
Decision Tree
Examples
V
Improve inference efficiency. Our work reveals that scaling down model size alone has little impact on tasks demanding processing information in the LLM's context. Practitioners may thus use our findings to identify scenarios where decisions could be routed to smaller model instead of larger one without compromising task performance (Chen et al., 2023; Dohan et al., 2022).
Improve assessment of down-scaling techniques. Whether through pruning training distinct, smaller model, our work calls for balanced evaluation of the down-scaled model prior to its adoption. Overemphasis either fact recall ICL risks skewing the assessment of L.LM capabilities.
Improve LLM interpretabiliiyy We find a remarkably small set weights responsible for ICL. This underscores the potential of pruning as tool for isolating neurons responsible for LLM capabilities.
Contributions.
We curate a set of benchmarks for assessing the disparate effects of down-scaling on core capabilities of LLMs, focusing fact recall and in-context learning (ICL).
2. Using these benchmarks, we evaluate two scalable weight pruning algorithms for LLMs. We investigate an extensive set of models comprising six base LLMs, with sizes reaching up to 33 billion parameters. We find that:
even moderate levels of pruning (> 30%) hurt fact recall. However, when the evidence required to solve question-answerigg task is provided in context, the model's ability to correctly answer questions survives higher levels of sparsity;
in contrast, ICL withstands aggressive pruning (up to 60-70%).
3. Similarly for dense scaling, find the same disparate patterns as above for fact recall and ICL capabilities as we vary dense model size. This underscores that the disparate effects on ICL and fact recall are not exclusive pruning but are broader characteristics of scaling in general.
RELATED WORK
LLM scaling, Studies on scaling laws (Hoffmann et al., 2022; Kaplan et al., 2020) suggest a predictable relationship between the quality of language models (i.e., perplexity) and the size of the pre-training corpus and model. Pursuant to this, many have scaled up and discovered remarkable capabilities of language models. Brown al. (2020) discover that LLMs can perform in-context learning (ICL) effectively: the model learns to perform tasks based on few examples of input-output demonstrations in the model's '' context. Other studies (Devlin al., 2019; Wei al., 2022: Liang et al., 2023; Srivastava et al., 2023; Gao et al., 2021; Ganguli et al., 2022; Bubeck et al.. 2023; Biderman al., 2023a) evaluate LLMs across many tasks and metrics and assess the utility of scaling. Our work differs in two ways: instead of measuring model performance on arbitrary tasks, we focus on the foundational capabilities of fact recall and ICI. These capabilities underpin the success of many real world applications of LLMs. Furthermore, while prior work (Kaplan al., 2020) studies joint scaling of both pre-training corpus size and model size, we focus on scaling model size alone.
wn-wh versus in-context learning. For LLMs, learning occurs both during weight updates and context processing (Chan et al., 2022a). Recent research illustrates the similarities and differences between these learning approaches: Von Oswald et al. (2023) demonstrated that ICL can implement an algorithm akin to gradient descent, commonly associated with in-weight learning. Akyürek et al. (2023) revealed that ICL resembles various in-weight learning algorithms, depending model size.
3

---

Published as a conference paper at ICLR 2024
Chan al. (2022b;a) study how the properties of the data distribution disparately affect the two learning approaches. Our work identifies yet another difference: scaling model size has distinct impacts on knowledge learned through model weights and knowledge acquired from model context.
Neural network pruning. Pruning removes unimportant parameters in a model. The origin of pruning traces back to LeCun et al. (1990) and Hassibi al. (1993), with focus reducing the computational footprint. More recently, with the advent of deep learning, pruning research has seen a resurgence (Renda et al., 2020; Han al., 2016; Liu et al., 2017; Frankle al., 2021; Molchanov et al., 2017; Dong et al., 2017: Ma et al., 2023; Tao et al., 2023; Kurtic et al., 2022; Dettmers et al., 2024). Though pruning traditionally focuses on preserving aggregate metrics such as accuracy, the versatility of LLMs calls for a different approach to assessing pruned models. To this end, Frantar & AAlistrr (2023); Sun al. (2024) assessed model size and task accuracy trade-off. Our work continues to expand our toolkits for empirical assessment, proposing to evaluate pruning's effect on fact recall and ICI. Our investigation extends a growing line of studies on effects of pruning beyond aggregate metrics such accuracy and perplexity. Hooker et al. (2021: 2020) show that pruning may harm under-represented categories of examples; Liebenwein et al. (2021) suggest that the pruned models are less accurate than the dense when predicting out-of-distribution examples. Jin et al. (2022) demonstrate that pruning mitigates the harmful effects of noisy data on generalization.
Concurrent to our work, Yin et al. (2023); Jaiswal et al. (2024) remarked on the pronounced negative effect of pruning on mnweleige-intensvee tasks. These works extensively evaluate tasks that require capabilities of fact recall and in-context information retrieval. Our work additionally curates gradation of tasks that further disentangle these abilities - specifically via overriding QA and ICL tasks, which these concurrent works do not cover.
3 PRELIMINARIES
In this section, we introduce methodological details common to all our experiments.
Pruning algorithms. We investigate pruning one possible technique to (down-)scale LLMs. Few pruning algorithms currently scale to LLMs. We use SparseGPT (Frantar & Alistarh, 2023) in the main text and Wanda (Sun et al., 2024) in Appendix D. Both are one-shot pruning algorithms that scale to LLMS and outperform magnitude pruning (i.e., pruning the smallest magnitude weights), without computationally intensive re-training (Frantar & Alistarh, 2023). SparseGPT prunes each layer of the language model by minimizing the ly-distance between the outputs of the original dense layer and the pruned layer. It computes these outputs based on small training dataset. Please refer to Appendix M for implementation details of SparseGPT and Wanda pruning algorithms.
Following standard practice (Frantar & Alistarh, 2023; Frankle & Carbin, 2019), we exclusively prune fully-connected layers. Fully-connected layers contain most of the parameters in attention and feed-forward modules and they account for over 97.5% of the parameters in all models examined. We exclude embedding layers, language modeling heads, and normalization layers from pruning.
Models. We evaluate 6 models from 3 families: OPT (Zhang al., 2022), LLaMA (Touvron et al., 2023) and Pythia (Biderman et al., 2023b). We focus on OPT and LLaMA in our main text and present Pythia results in Appendix E. Pythia family models show consistent results as LLaMA and OPT family models. From the OPT family, we evaluate the two largest models that fit in our hardware setup- - 03B-13B and OPT-30B, with 13 and 30 billion parameters, respectively. From the LLaMA family, we evaluate LLaMA- 13B and LLaMA-33B, with 13 and 33 billion parameters, respectively.
notable difference between OPT and LLaMA families is the ratio of pretraining data to model parameters. Zhang et al. (2022) train the OPT models using 180 billion tokens, resulting in ratios of approximately 14 and 5.5 tokens per parameter for the two OPT models we consider. Touvron et al. (2023) train the LLaMA-13B model with I trillion tokens, resulting in 77 tokens per parameter, and the LLaMA-33B model with 1.4 trillion tokens, resulting in 42 tokens per parameter.
Benchmark design. We evaluate models on two complementary capabilities: the capability to call on information seen during pre- pre-training (and then stored in the model's weights) and the capability to call on information presented in-context. Unfortunately, it is difficult to construct tasks that strictly isolate one of these two capability: on the one hand, for every conceivable natural language task, the model must rely the semantic representations of language it learned from pre-training and
4

---

Published as a conference paper at ICLR 2024
stored in the weights. On the other, every conceivable task necessitates processing model context to understand the instructions in the first place. To help disentangle the two broad capabilities, we focus on two well-defined instantiations of these capabilities: (a) the ability to recall from pre-training to answer questions and, complementing that (b) the ability to learn patterns from context.
We isolate (a) by evaluating the model on recalling facts from pre-training data not provided in-context. We do this using closed-book QA benchmarks.
To evaluate (b), we consider a gradation of tasks that require the increasingly advanced ways of learning from in-context information. Broadly, we consider a series of in-context learning tasks; tasks where the context contains examples of (query, answer) pairs in some form, followed by test query that the model has to answer.
First, as simple ICL task, we consider an Open-book QA (Section 4) counterpart of our QA task. In this task setup, the context includes supporting evidence that directly helps answer the question. Essentially, the ICL task involves a test query that appears within the evidence as an in-context example, complete with its correct answer. For instance, if the test query asks, "Who is the author of the novel The Eagle Has Landed?", the supporting evidence would state, "The author of The Eagle Has Landed is..." and include the answer, "Jack Higgins". Note that in addition to the aforementioned siuddenc--qesttion-eee
triplet, we further aid the model with an independent one-shot triplet to demonstrate how to format its answer.
In the above case, the answer provided in-context may also have been present in pre-training data; thus the model may opt ignore the context, and recall facts from pre-training, and still succeed at the task. Thus, to more rigorously isolate the two mechanisms, we proceed to evaluate on the Overriding QA task (Section 4), where the in-context evidence contradicts a fact present in pre-training data. We present a one-shot evidence-questionannee contextual evidence rather than from pre-training data. Thus, to solve the task correctly, the model must retrieve from the context rather than recall pre-training facts.
triplet, where the correct answer follows from
Next, we investigate ICL capabilities more sophisticated than copying an answer present as an in-context example. To this end, we are interested in tasks where the model must respond to a test query that it has not already seen in-context. A frequently cited example of ICL the English-Freend translation task (Brown et al., 2020), where pairs of English and French phrases serve examples, with the test query being novel English phrase. However, such conventional instances of ICL tasks require significant background knowledge of languages not available in the context, thereby conflating the evaluation of ICL. with the recall of information learned during model pre-training.
To ensure we disentangle the role of knowledge recall from ICL, we consider tasks where the query- answer mapping is a parameterized function (e.g., linear classifiers), Arguably, learning this mapping requires advanced context-processing mechanisms that go beyond information-rettievall mechanisms required for the Open-book and Overriding QA tasks.
Metrics. We reproduce the effects of down-scaling perplexity in Frantar & Alistarh (2023); Sun et al. (2024) in Appendix A. However, our main metric is exact match accuracy (of the answer to each task) because it is direct measure of the model's ability to satisfactorily perform task.
Software and hardware. We perform pruning on 80GB version of 1100 GPUs, using the implemen- tation of SparseGPT (Frantar & Alistarh, 2023). We perform evaluations using TPU v3 running PyTorch (Paszke et al., 2019) with XLA backend. Our research project consumes approximately TPU-month worth of computation. Using TPUs necessitates 6 numerical format. We assess its impact on our experimental results in Appendix I.
4 EFFECT OF PRUNING ON QUESTION ANSWERING TASKS
The ability of LLMs to answer questions underpins many applications of LLMs. Within question- answering evaluation framework, we examine the effect of pruning on its capability to recall facts learnt during pre-training. We also test simple form of ICL capability extracting answers from supporting evidence in the model's ' context, where the answer is unlikely available in pre-training data. Finally, we evaluate interpolation between these two capabilities extracting answers from supporting evidence in the model's 's context, where the answer is likely also available in pre-training data. Our results expose that, with supporting evidence in question-answerinn tasks, the model's
5

---

Published as a conference paper at ICLR 2024
OPT-13B
OPT-308
LLaMA-138
LLaMA-33B
Close-book
Open-book
Overriding
60
TriviaQA
80
Exact Match (%)
WebQA
Exact Match (%)
Exact Match (%)
60
40
40
20
TriviaQA(F)
20
NatQ
DissentQA
25
50 75
25
50
75
25
50 75
Sparsity (%)
Sparsity (%)
Sparsity (%)
Evaluation for question answering tasks under pruning. Each color represents different model; each line-style/marker- shape combination represents a different QA dataset. Moderate pruming harms LLMs' capabiliiy to recall facts learnt during vre-training: in close-book QA tasks, accepting a 5% drop in average accuracy w.r.t. the dense models, we can prune to 30 and 40% on TriviaQA and WehQA dataset. However, when we present the model with the supporting evidence answer the question in its context, the model's ability to answer questions survives to higher sparsity: on open-book QA tasks, accepting the same drop average accuracy, we can prune 50% and 60% on TriviaQA(F) and NatQ dataset, respectively. On the overriding task (DissentQA dataset), we may even prune 70% while maintaining the same acceptable relative accuracy drop.
Figure 2:
ability to answer questions withstands more aggressive pruning than without. With no supporting evidence in context, accuracy rapidly drops as sparsity increases.
Datasets. We use these datasets: (a) TriviaQA. Joshi al. (2017) developed the TriviaQA dataset with questions and supporting evidence. We use its Wikipedia validation partition consisting of 7993 questions.
(b) WebQuestions. Berant et al. (2013) collected question-answer pairs from the Freebase knowledge database. We use its test set consisting of 2032 questions. (c) NaturalQuestions Kwiatkowski al. (2019) compiled the NaturalQuestions dataset from Google search queries. We sampled a 7700-question subset of its validation partition (the same size as the following dataset derived from it), make our evaluation computationally feasible. (d) DissentQA. Neeman al. (2023) constructed the DissentQA dataset from the NaturalQuestions dataset. It contains pairs of questions and evidence for a made-up answer that is different from the factual one. We use its validation partition consisting of 7700 questions.
Evaluation setup. Using the aforementioned datasets, we evaluate the ability of pruned models on the following task setup: (i) Close-book. We feed the question without any supporting evidence to the model. We use the Wikipedia partition of the TriviaQA dataset and the WebQA dataset for this setup. (ii) Open-book. We feed the question with supporting evidence to the model in its context. Notably, this evaluation setup only evaluates whether the answer is right; is however agnostic to the mechanism by which the model retrieves the answer: the model may either generate its answer using the supporting evidence, or by recalling facts from pre-training. To create this dataset, we use a subset of the TriviaQA dataset whose context can fit within the maximum sequence length of the model, consisting of 881 questions. We denote this filtered subset as TriviaQA(F). Additionally, we use the NaturalQuestions dataset for this setup with factual contexts. (iii) Overriding. We present the question accompanied by evidence that deviates from the facts presented during pre-training. Given that the anticipated made-up answers are randomized and different from the factual ones, the model is likely unable to rely on memorization from its pre-training data to generate responses. This evaluation rigorously assesses the model's capability to override its pre-training data with new, context-specific information. We use the DisentQA dataset with supporting evidence for the made-up answer for this setup. Examples of the setups are shown in Appendix G.
To summarize: across these setups, our prompts have three parts to them: first, (1) example evidence question-easeeeeee (2) the supporting evidence for test question (except in the book- setup), and (3) a test question to answer. Answers are the model's prompt completions using greedy decoding. We report the percentage of answers that exactly match ground truth.
triplet (or question-and answer pair in close-boot setting) for demonstration,

---

Published as a conference paper at ICLR 2024
Close-book versus open-book results. Left plots of Figure 2 shows close-book and open- ppen-book results. Notably, for all the models, the pruned model maintains performance on open-book tasks until much higher sparsity levels compared to close-book tasks. In particular, when accept relative decrease of 5% from pruning in the mean accuracy over four models, the highest achievable sparsities for closed-book tasks are 30% TriviaQA and 40% on WebQA.
In contrast, maintaining the same acceptable performance drop, the highest acceptable sparsity levels for open-book tasks are 50% on T(F) and 60% on NaturalQuestions, Our results suggest that while pruning hurts the model's ability to recall information from its pre-training data at moderate sparsity (30-40%), one possible remedy to provide the model with supporting evidence in context. The model's 's ability to answer questions with supporting evidence in its context remains largely intact to higher sparsity levels (50-60%)
Overriding results. The rightmost plot of Figure 2 demonstrates a consistent trend for the overriding tasks. On the DisentQA dataset with supporting evidence for the made-up answers, the highest achievable sparsity is 70% allowing for the same 5% accuracy drop before. Recall that to solve the overriding task, the model must rely on in-context information rather than in-weights information from the pre-training data. Thus, our observations here further substantiate that the model's ability to extracting information from its context remains largely intact at higher sparsity levels.
Accuracy improvements. We note that, surprisingly, on open-book and overriding tasks, there can be small accuracy boosts from pruning. For example, the pruned OPT-30B sees a 9.7% accuracy improvement DissentQA dataset in overriding task setup. The pruned LLaMa- 13B sees 3.5% accuracy improvement on NaturalQuestions in the open-book task setup. Though prior work often reports accuracy improvements with pruning (Frankle & Carbin, 2019; Renda al., 2020), we are, to the best of our knowledge, the first to observe such improvement in the QA setup.
Takeaways. Our findings suggest that scaling via pruning has much higher impact on the model's capability to retrieve facts learnt during pre-training than on its capability to retrieve information from the context. Pruning may even improve model's capability to answer questions when its context includes the necessary information to answer the question.
SOPHISTICATED IN-CONTEXT LEARNING
Section 4 demonstrates that moderate pruning preserves question answering task accuracy when supporting evidence available context. In this section, we show that even with more sophisticated ICL tasks than previously studied, moderately pruned models maintain their ICL abilities.
Typical ICL tasks require models to have extensive knowledge about language and the world learnt during pre-training. However, Section 4 shows that pruning likely impairs such knowledge. So, to isolate the LLMs' ICL capability from its fact recall capability, we select tasks that require learning a parameterized function (e.g., linear model) through input-output examples present in-context.
Method. We evaluate LLMs with multiclass classification problems of the form f R. > (0, 1, 1.... K }} from in-context examples, where 1) refers the input dimensionality and K the number of label categories. To generate a sequence consisting of context, query and an answer, we first pick a random parameterized function from one of three function classes: a). 2-way lincar classifiers, b). 2-way neural network classifiers with 2 layers, where the hidden dimension size equals the input dimension size, and c). 4-way decision tree classifiers. For instance, to pick a random 2-way linear classifier, we sample a random classification hyperplane in the input space; see Appendix B for more details regarding the construction of f. Then, in the context, we provide N examples of the form (x, / (xr),, followed by novel query Fquery- The goal of the model is to correctly predict the answer as / (x Please refer to Appendix G for an example of the full prompt.
We construct M instances of parametric function f for each function class. For each instance, we generate N in-context examples split evenly across all possible labels. We generate the test example randomly and we ensure equal probabilities for its classification into all possible label categories. We do not use any advanced prompting technique when solving ICL tasks. The LLMs only generate answer tokens corresponding directly classification labels, they do not generate any other tokens during evaluation. We draw all random numbers uniformly from integers between -10 and 10 inclusive. We set M 2048, D = 4, N 32.
7

---

Published as a conference paper at ICLR 2024
OPT-13B
OPT-30B
LLaMA-133 138
LLaMA-338
Linear Classification
2-layer Neural Net
Decision Tree
Exact Match (%)
60
Exact Match (%)
Exact Match (9)
40
40
40
20
20
20
0
0
0
25
75
25
50 75
25
50 75
Sparsity (%)
Sparsity (%)
Sparsity (%)
Figure 3: ICL withstands aggresive pruning (60-70% spasity). Accepting relative average (over models) accuracy drop of 5% w.r.t. the dense models, we can prune to 60%, 60% and 70% on linear classification, 2-layer NN, decision tree tasks, respectively.
Pruning results. Figure 19 shows the accuracy of pruned models versus sparsity. We can remove substantial number parameters from the models without affecting their ICL capability. Specifically, the average accuracy of the pruned models on linear, 2-layer NN, decision tree classification tasks is within 5% of the dense models up to 60%, 60% and 70% sparsity, respectively.
Task difficulty. One trivial explanation for our observation is that perhaps these tasks are easy that they are solvable even by severely pruned model. eliminate this hypothesis, in Appendix C, increase the input dimension (D) of the linear classification task. Even increase this to significant extent (say, to an extent that the task almost, but not totally, unlearnable), we find that the model maintains its ICL performance even under aggressive pruning.
Conclusion. Our results suggest that when pruning LLMs moderately (i.e., to < 70% sparsity), the model's ICL capability stays largely intact. Thus, unlike fact recall tasks, can prune models significantly more while maintaining ICL performance.
DENSE SCALING
For vision models, pruned models behave similarly to smaller dense models trained with similar hyperparameters (Jin et al., 2022). Here, examine the effects of dense scaling (i.c., choosing from a suite of independently trained models of various sizes) on fact recall vs. ICL capabilities of LLMs.
Pruned OPT-30B
Pruned OPT-13B
Dense
ICL
Close-book book
Open-book & Counterfactual
40
Exact Match (%)
TriviaQA
Exact Match 180
40-
50
20
Exact
20-
TrivaaA(F)
30
DisentQA
linear --.- nn
dt
0
10
20
30
10
20
30
20
10
20
30
Param Count (B)
Param Count (B)
Param Count (B)
Figure Like pruning, moderate dense down-scaling preserves ICI. while harming fact recall. Gray lines show the sequence of dense down-scaled OPT models; we also show the sequence of pruned OPT-30B and OPT-13B models in red and yellow for comparison. Each line style and marker shape combination corresponds to unique tasks. For the left plot: fact recall is highly sensitive to dense scaling: gray lines shows that, moving from the 30B model to the next largest 13B model leads to than 5% relative accuracy drop close-book TriviaQA (solid line/circle marker) and WebQA task (dashed line/square marker), respectively. For the middle plot, open-bookkoverriiing QA accuracy is much less sensitive dense down-scaling; maintaining the same 5% acceptable accuracy drop, one may down-scale 30B model to a 6.7B model on TriviaQA(F) (gray, solid line/circle marker), NaturalQuestions dataset (gray, dashed line/square marker) and DisentQA (gray, dotted line/triangle marker). For the right plot, model performance on sophisticated ICL tasks (linear classification, 2-layer NN and decision tree) are highly robust dense down-scaling.
8

---

Published as a conference paper at ICLR 2024
With the same benchmarks in Sections 4 and 5, evaluate the accuracy of dense models with different parameter count, but the same pre-training data size. The OPT family of models provides an ideal collection of dense models for our evaluation: Zhang et al. (2022) trained all OPT models with the same number of tokens (300b).
Method.
Results. Figure 4 shows that fact recall capabilliy is highly sensitive to dense scaling e.g.. focusing on scaling downwards, moving from the 30B model to the next largest 13B model leads to more than 5% relative task accuracy degradation on close-book TriviaQA and WebQA task. However, open- book/overriding QA accuracy is much less sensitive to dense scaling. Specifically, maintaining the same 5% acceptable relative accuracy degradation with respect to the 30B models, one may replace a 30B model on (F))
and DisentQA dataset with a 6.7B model. Figure 4 also shows that sophisticated ICL. capability similarly robust to dense scaling. Specifically, the accuracy difference between the largest (30B) and the smallest (1.3B) dense OPT model we consider 1.8%, 1.4% and 0. % on linear, 2-layer NN and decision tree classification tasks, respectively.
Conclusion. Like pruning, changing a dense model's size more readily affects its capability to retrieve facts from pre-training than to process information from context. We hypothesize that this effect stems from scaling in general be it pruning or using dense model of different size.
7 CLOSING DISCUSSION
We study the effects of scaling model size via pruning and dense scaling on two core capabilities of LLMs that underpin their practical success: the capability to recall facts and the capability to perform in-context learning (ICL). In both cases, we find disparate effects the two capabilities. Moderate pruning (> 30% sparsity) harms fact recall, and yet the capability to learn from a few input-output examples from context withstands aggressive pruning (up to 60-70% sparsity). The same disparity arises when changing the width and depth of dense (independennly-traine
models.
What could explain this disparity? We conjecture that the number of parameters required to store a set of facts must scale in proportion to the number of independent facts. On the other hand, a smaller set of parameters acting as universal gradient descent module (Von Oswald et al., 2023) may accomplish many kinds of in-context learning tasks. Verifying these hypotheses theoretically is an important direction for future work. Our findings also invite various research directions:
Pruning& & interpretability. Our findings suggest, remarkably, relatively small fraction of weights are responsible for ICL. performance. This observation could prove to be valuable for enhancing the interpretability of LLMs, reviving decades-old motivation behind work in pruning (Mozer & Smolensky, 1988). In particular, pruning may help better localize the weights responsible for ICL and fact recall capabilities, and complement recent approaches such identifying circuits consisting of attention heads responsible for specific capabilities (Wang et al., 2023), and explaining neuron functionalities using large language models themselves (Bills al., 2023). In Appendix F, we present preliminary results comparing the importance of feedforward (FFW) versus attention layers for maintaining ICL and fact recall capabilities. Our evaluation shows that while FFW and attention layers show similar importance for ICL, FFWs layers show greater importance for fact recall.
Memory augmentation. Our observations advocate for memory augmentation as a promising way to improve the trade-off' between computational cost and task accuracy. Memory augmentation techniques present helpful facts to the model by augmenting them directly the context. Thus, rather than having the LLM to rely on fact recall from pre-training downscaling - we can delegate fact-retrieval to a separate retrieval model (Borgeaud al., 2022; Guu et al., 2020), and allow the LLM to focus on retrieving the fact from the context-whie is an ability that is preserved under -caa-nc
an ability that is degraded under
Limitations. We validated claims on a large number of benchmarks (a total of 8 tasks and 6 models). However, since our work is empirical in nature, our observations may not generalize to all tasks and LLMs. Furthermore, we focused only on pruning algorithms that scale to the size of contemporary LLMs, as we believe that scalability is essential for the wide adoption of any pruming algorithm. Even though more sophisticated pruning algorithms exist (Renda et al.. 2020), they typically require re- re-training - re-runming the training of these large-scale language models for every sparsity level we examine. The cost of such experiment is therefore beyond our means.

---

Published as a conference paper at ICLR 2024
Acknowledgements. We thank Zack Ankner, William Brandon, Ellie Y. Cheng, Jesse Michelle, Nicole Mitchell, Aniruddha 'Ani' Nrusimha, Alex Renda, Daniel M. Roy, Logan Weber for their helpful feedback on this work. This work was supported in-part by the Sloan Foundation, Google Research, the MIT-IBM Watson Al Lab, Apple, and SRC JUMP 2.0 (CoCoSys).
REFERENCES
Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, and Denny Zhou. What learning algorithm is in-context learning? investigations with linear models. In International Conference on Learning Representations, 2023.
Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang, Semantic parsing Freebase from question-answer pairs. In Conference on Empirical Methods in Natural Language Processing, 2013.
Stella Biderman, USVSN Sai Prashanth, Lintang Sutawika, Hailey Schoelkopf, Quentin Anthony, Shivanshu Purohit, and Edward Raff. Emergent and predictable memorization in large language models. arXiv preprint arXiv:2304.11158 2023a.
Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar van der Wal. Pythia: A suite for analyzing large language models across training and scaling, arXiv preprint arXiv:2304.01373, 2023b.
Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. Language models can explain neurons in language mod- els. https: / aepenaapubllii blob. core. windows net / -ew.onneeppainerr paper/index htm. 2023.
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, bean-Baptiste Lespiau. Bogdan Damoe, Aidan Clark, Diego De Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving, Oriol Vinyals, Simon Osindero, Karen Simonyan, Jack Rae, Erich Elsen, and Laurent Sifre. Improving language models by retrieving from trillions of tokens. In International Conference on Machine Learning, 2022.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Advances in Neural Information Processing Systems, 2020.
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12777 2023.
Stephanie C. Y. Chan, Ishita Dasgupta, Junkyung Kim, Dharshan Kumaran, Andrew K. Lampinen, and Felix Hill. Transformers generalize differently from information stored in context vs in weights. arXiv preprint arXiv:2210.05675. 2022a.
Stephanie C. Y. Chan, Adam Santoro, Andrew K. Lampinen, Jane X. Wang, Aaditya Singh, Pierre H. Richemond, Jay McClelland, and Felix Hill. Data distributional properties drive emergent in- context learning in transformers. In Advances in Neural Information Processing Systems, 2022b.
Lingjiao Chen, Matei Zaharia, and James Zou. Frugalgpt: How to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305..0 05 176, 2023.
10

---

Published as a conference paper at ICLR 2024
Damai Dai, Li Dong, Y. Hao, Zhifang Sui, and Furu Wei. Knowledge neurons in pretrained transformers. arXiv preprint arXiv:2104.08666 2021.
Tim Dettmers, Ruslan A. Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Salch Ashkboos, Alexander Borzunov, Torsten Hoefler, and Dan Alistarh. SpQR: A sparse-quantized representation for near-lsssless LLM weight compression. International Conference on Learning Representations, 2024.
Jacob Devlin, Ming-Wer Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies), 2019.
David Dohan, Winnie Xu, Aitor Lewkowycz, Jacob Austin, David Bieber, Raphael Gontijo Lopes, Yuhuai Wu, Henryk Michalewski, Rif A. Saurous, Jascha Sohl-dickstein, Kevin Murphy, and Charles Sutton. Language model cascades. arXiv preprint arXiv:2207.10344 2022.
Xin Dong, Shangyu Chen, and Sinno Pan. Learning to prune deep neural networks via layer-wise optimal brain surgeon. In Advances in Neural Information Processing Systems, 2017.
Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. In International Conference on Learning Representations, 2019.
Jonathan Frankle, Gintare Karolina Dziugaite, Daniel Roy, and Michael Carbin. Pruning neural networks at initialization: Why are we missing the mark? In International Conference on Learning Representations, 2021.
Elias Frantar and Dan Alistarh. SparseGPT: Massive language models can be accurately pruned in onc-shot. In International Conference on Machine Learning, 2023.
Elias Frantar, Eldar Kurtic, and Dan Alistarh. M-fac: Efficient matrix- free approximations of second-order information. In Advances in Neural Information Processing Systems, 2021.
Elias Frantar, Salch Ashkboos, Torsten Hoefler, and Dan Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers, 2023.
Deep Ganguli, Danny Hernandez, Liane Lovitt, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova Dassarma, Dawn Drain, Nelson Elhage, Sheer El Showk, Stanislav Fort, Zac HatfieldDoodds Tom Henighan, Scott Johnston, Andy Jones, Nicholas Joseph, Jackson Kernian, Shauna Kravec, Ben Mann, Neel Nanda, Kamal Ndousse, Catherine Olsson, Daniela Amodei, Tom Brown, Jared Kaplan, Sam McCandlish, Christopher Olah, Dario Amodei, and Jack Clark. Pre- dictability and surprise in large generative models. In ACM Conference on Fairness, Accountability, and Transparency, 2022.
Leo Gao, Jonathan Tow, Stella Biderman. Sid Black, Anthony DiPofi, Charles Foster. Laurence Gold- ing, Jeffrey Hsu, Kyle McDonell, Niklas Muennighoff, Jason Phang, Laria Reynolds, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. A framework for few-shot language model evaluation, September 2021. URL https::////...........ttt /oiii.... org/ 10. 0.5. 5281 1 zenodo. 5371 1777628. 8.
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming- Wei Chang. Realm: Retrieval- augmented language model pre- mre-training. In International Conference Machine Learning, 2020.
Song Han, Huizi Mao, and William J. Dally. Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding. In International Conference on Learning Representations, 2016.
B. Hassibi, D.G. Stork, and G.J. Wolff. Optimal brain surgeon and general network pruning. In IEEE International Conference on Neural Networks, 1993.
Jordan Hoffmmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and Laurent Sifre. Training compute-optimal large language models. In Advances Neural Information Processing Systems, 2022.
11

---

Published as a conference paper at ICLR 2024
Sara Hooker, Nyalleng Moorosi, Gregory Clark, Samy Bengio, and Emily Denton. Characterising bias in compressed models. arXiv preprint arXiv:2010.03058, 2020.
Sara Hooker, Aaron Courville, Gregory Clark, Yann Dauphin, and Andrea Frome. What do com- pressed deep neural networks forget? arXiv preprint 1910:2248,, 2021.
Ajay Kumar Jaiswal, Zhe Gan, Xianzhi Du, Bowen Zhang, Zhangyang Wang, and Yinfei Yang. Compressing LLMs: The truth is rarely pure and never simple. In The Twelfth International Conference on Learning Representations, 2024.
Ting Jiang, Deqing Wang, Fuzhen Zhuang, Ruobing Xie, and Feng Xia. Pruning pre-trained language models without fine-tuning. In Annual Meeting of the Association for Computational Linguistics, 2023.
Tian Jin, Michael Carbin, Dan Roy, Jonathan Frankle, and Gintare Karolina Dziugaite. Pruning's effect on generalization through the lens of training and regularization. In Advances in Neural Information Processing Systems, 2022.
Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.0836/ 2020.
Eldar Kurtic, Denis Kuznedelev, Elias Frantar, Michael Goin, and Dan Alistarh. Sparse finetuning for inference acceleration of large language models. Advances in Neural Information Processing Systems, 2022.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Matthew Kelcey, Jacob Devlin, Kenton Lee, Kristina N. Toutanova, Llion Jones, Ming-Wo Wei Chang, Andrew Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. Natural questions: benchmark for question answering research. In Transactions of the Association of Computational Linguistics, 2019.
Yann LeCun, John Denker, and Sara Solla. Optimal brain damage. In Advances in Neural Information Processing Systems, 1990.
Daliang Li, Ankit Singh Rawat, Manzil Zaheer, Xin Wang. Michal Lukasik, Andreas Veit, Felix Yu, and Sanjiv Kumar. Large language models with controllable working memory. In Findings of the Association for Computational Linguistics, 2023.
Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Alexander Cosgrove, Christopher D Manning. Christopher Re, Diana Acosta-Navas, Drew Arad Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue WANG, Keshav Santhanam, Laurel OIT, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri S. Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Andrew Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, and Yuta Koreeda. Holistic evaluation of language models. Transactions on Machine Learning Research, 2023.
Lucas Liebenwein, Cenk Baykal, Brandon Carter, David Gifford, and Daniela Rus. Lost in pruning: The effects of pruning neural networks beyond test accuracy. In Conference Machine Learning and Systems, 2021.
Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui Zhang. Learning efficient convolutional networks through network slimming. In IEEE International Conference on Computer Vision, 2017.
12

---

Published as a conference paper at ICLR 2024
Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. Entity-based knowledge conflicts in question answering. In Conference on Empirical Methods in Natural Language Processing, 2021.
Xinyin Ma, Gongfan Fang, and Xinchao Wang. LLM-pruner: On the structural pruning of large language models. In Advances in Neural Information Processing Systems, 2023.
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. In Advances in Neural Information Processing Systems, 2022.
Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, and Jan Kautz. Pruning convolutional neural networks for resource efficient inference. In International Conference on Learning Repre- sentations, 2017.
Michael C Mozer and Paul Smolensky. Skeletonization: A technique for trimming the fat from network via relevance assessment. In Advances in Neural Information Processing Systems, 1988.
Ella Neeman, Roee Aharoni, Or Honovich, Leshem Choshen, Idan Szpektor, and Omri Abend. DisentQA: Disentangling parametric and contextual knowledge with counterfactual question answering. In Annual Meeting of the Association for Computational Linguistics, 2023.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang. Zach De Vito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang. Junjie Bai, and Soumith Chintala. Pytorch: an imperative style, tegh-peeformance deep learning library. In Advances in Neural Information Processing Systems, 2019.
Alex Renda, Jonathan Frankle, and Michael Carbin. Comparing rewinding and fine-tuning in neural network pruning. In International Conference on Learning Representations, 2020.
Michael Santacroce, Zixin Wen, Yelong Shen, and Yuanzhi Li. What matters in the structured pruning of generative language models? arXiv preprint Stt1:23022.037733 2023.
Aarohi Srivastava, Abhinav Rastogi. Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garrig--lonss Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazary, Alice Xiang, Alicia Parrish, Allen Nie, Aman Hussain, Amanda Askell, Amanda Dsouza, Ambrose Slone, Ameet Rahane, Anantharaman S. Iyer, Anders Johan Andreassen, Andrea Madotto, Andrea Santilli, Andreas Stuhlmüller, Andrew M. Dai, Andrew La, Andrew Lampinen, Andy Zou, Angela Jiang, Angelica Chen, Anh Vuong, Animesh Gupta, Anna Gottardi, Antonio Norelli, Anu Venkatesh, Arash Gholamidavoodi, Arfa Tabassum, Arul Menezes, Arun Kirubarajan, Asher Mullokandov. Ashish Sabharwal, Austin Herrick, Avia Efrat, Aykut Erdem, Ayla Karakaş, B. Ryan Roberts, Bao Sheng Loe, Barret Zoph, Bartlomiej Bojanowski, Batuhan Özyurt, Behnam Hedayatnia, Behnam Neyshabur, Benjamin Inden, Benno Stein, Berk Ekmekci, Bill Yuchen Lin, Blake Howald, Bryan Orinion, Cameron Diao, Cameron Dour, Catherine Stinson, Cedrick Argucta, Cesar Ferri, Chandan Singh, Charles Rathkopf, Chenlin Meng, Chitta Baral, Chiyu Wu, Chris Callison-Burch, Christopher Waites, Christian Voigt, Christopher D Manning, Christopher Potts, Cindy Ramirez, Clara E. Rivera, Clemencia Siro, Colin Raffel, Courtney Ashcraft, Cristina Garbacea, Damien Sileo, Dan Garrette, Dan Hendrycks, Dan Kilman, Dan Roth, C. Daniel Freeman, Daniel Khashabi, Daniel Levy, Daniel Moseguí González, Danielle Perszyk. Danny Hernandez, Danqi Chen, Daphne Ippolito, Dar Gilboa, David Dohan, David Drakard, David Jurgens, Debajyoti Datta, Deep Ganguli, Denis Emelin, Denis Kleyko, Deniz Yuret, Derek Chen, Derek Tam, Dieuwke Hupkes, Diganta Misra, Dilyar Buzan, Dimitri Coelho Mollo, Diyi Yang, Dong- Ho Lee, Dylan Schrader, Ekaterina Shutova, Ekin Dogus Cubuk, Elad Segal, Eleanor Hagerman, Elizabeth Barnes, Elizabeth Donoway, Ellie Pavlick, Emanuele Rodolà, Emma Lam, Eric Chu, Eric Tang, Erkut Erdem, Ernie Chang, Ethan A Chi, Ethan Dyer, Ethan Jerzak, Ethan Kim, Eunice Engefu Manyasi, Evgenii Zheltonozhskii, Fanyue Xia, Fatemeh Siar, Fernando Martínez-Plumee, Francesca Happé, Francois Chollet, Frieda Rong, Gaurav Mishra, Genta Indra Winata, Gerard de Melo, Germán Kruszewski, Giambatista Parascandolo, Giorgio Mariani, Gloria Xinyue Wang, Gonzalo Jaimovitch- Lopez, Gregor Betz, Guy Gur-Ari. Hana Galijasevic, Hannah Kim, Hannah Rashkin, Hannanch Hajishirzi, Harsh Mehta, Hayden Bogar, Henry Francis Anthony Shevlin, Hinrich Schuetze, Hiromu Yakura, Hongming Zhang, Hugh Mee
13

---

Published as a conference paper at ICLR 2024
Wong, Ian Ng, Isaac Noble, Jaap Jumelet, Jack Geissinger, Jackson Kernion, Jacob Hilton, Jaehoon Lee, Jaime Fernández Fisac, James B Simon. James Koppel, James Zheng, James Zou, Jan Kocon, Jana Thompson, Janelle Wingfield, Jared Kaplan, Jarema Radom, Jascha Sohl-Dickstein, Jason Phang, Jason Wei, Jason Yosinski, Jekaterina Novikova, Jelle Bosscher, Jennifer Marsh, Jeremy Kim, Jeroen Taal, Jesse Engel, Jesujoba Alabi, Jiacheng Xu, Jiaming Song, Jillian Tang, Joan Waweru, John Burden, John Miller, John U. Balis, Jonathan Batchelder, Jonathan Berant, Jörg Frohberg, Jos Rozen, Jose Hernandez-Orallo,, Joseph Boudeman, Joseph Guerr, Joseph Jones, Joshua B. Tenenbaum, Joshua S. Rule, Joyce Chua, Kamil Kanclerz, Karen Livescu, Karl Krauth, Karthik Gopalakrishnan, Katerina Ignatyeva, Katja Markert, Kaustubh Dhole, Kevin Gimpel, Kevin Omondi, Kory Wallace Mathewson, Kristen Chiafullo, Ksenia Shkaruta, Kumar Shridhar, Kyle McDonell, Kyle Richardson, Laria Reynolds, Leo Gao, Li Zhang, Liam Dugan, Lianhui Qin, Lidia Contreras-Ochandd. Louis-Philippe Morency, Luca Moschella, Lucas Lam, Lucy Noble, Ludwig Schmidt, Luheng He, Luis Oliveros-Coldd Colón, Luke Metz, Lütfi Kerem Senel, Maarten Bosma, Maarten Sap, Maartje Ter Hoeve, Maheen Farooqi. Manaal Faruqui, Mantas Mazeika, Marco Baturan, Marco Marelli, Marco Maru, Maria Jose Ramirez-Quintana, Marie Tolkiehn, Mario Giulianelli, Martha Lewis, Martin Potthast, Matthew L. Leavitt, Matthias Hagen, Mátyás Schubert, Medina Orduna Baitemirova, Melody Arnaud, Melvin McElrath, Michael Andrew Yee, Michael Cohen, Michael Gu, Michael Ivanitskiy, Michael Starritt, Michael Strube, Michal Swedrowski, Michele Bevilacqua, Michihiro Yasunaga, Mihir Kale, Mike Cain, Mimee Xu, Mirac Suzgun, Mitch Walker, Mo Tiwari, Mohit Bansal, Moin Aminnaseri, Mor Geva, Mozhdeh Gheini. Mukund Varma T, Nanyun Peng. Nathan Andrew Chi, Nayeon Lee, Neta Gur-Ari Krakover, Nicholas Cameron, Nicholas Roberts, Nick Doiron, Nicole Martinez, Nikita Nangia, Niklas Deckers, Niklas Muennighoff, Nitish Shirish Keskar, Niveditha S. Iyer, Noah Constant, Noah Fiedel, Nuan Wen, Oliver Zhang, Omar Agha, Omar Elbaghdadi. Omer Levy, Owain Evans, Pablo Antonio Moreno Casares, Parth Doshi, Pascale Fung, Paul Pu Liang, Paul Vicol, Pegah Alipoormolabashi, Peiyuan Liao, Percy Liang, Peter W Chang, Peter Eckersley, Phu Mon Htut, Pinyu Hwang, Piotr Milkowski, Piyush Patil, Pouya Pezeshkpour, Priti Oli, Qiaozhu Mei, Qing Lyu, Qinlang Chen, Rabin Banjade, Rachel Etta Rudolph, Raefer Gabriel, Rahel Habacker, Ramon Risco, Raphaël Millière, Rhythm Garg, Richard Barnes, Rif A. Saurous, Riku Arakawa, Robbe Raymaekers, Robert Frank, Rohan Sikand, Roman Novak, Roman Sitelew, Ronan Le Bras, Rosanne Liu, Rowan Jacobs, Rui Zhang, Russ Salakhutdinov, Ryan Andrew Chi, Seungjae Ryan Lee, Ryan Stovall, Ryan Techan, Rylan Yang. Sahib Singh, Saif M. Mohammad, Sajant Anand, Sam Dillavou, Sam Shleifer, Sam Wiseman, Samuel Gruetter, Samuel R. Bowman, Samuel Stern Schoenholz. Sanghyun Han, Sanjeev Kwatra, Sarah A. Rous, Sarik Ghazarian, Sayan Ghosh, Sean Casey, Sebastian Bischoff, Sebastian Gehrmann, Sebastian Schuster, Sepideh Sadeghi, Shadi Hamdan, Sharon Zhou, Shashank Srivastava, Sherry Shi, Shikhar Singh, Shima Asadii Shixiang Shane Gu, Shubh Pachchigar, Shubham Toshniwal, Shyam Upadhyay, Shyamolima Shammie Debnath, Siamak Shakeri, Simon Thormeyer, Simone Melzi, Siva Reddy, Sneha Priscilla Makini, Soo- Hwan Lee, Spencer Torene, Sriharsha Hatwar, Stanislas Dehaene, Stefan Divic, Stefano Ermon, Stella Biderman, Stephanie Lin, Stephen Prasad, Steven Piantadosi, Stuart Shieber, Summer Misherghi, Svetlana Kiritchenko, Swaroop Mishra, Tal Linzen, Tal Schuster, Tao Li, Tao Yu, Tariq Ali, Tatsunori Hashimoto, Te-Lin Wu, Théo Desbordes, Theodore Rothschild, Thomas Phan, Tianle Wang, Tiberius Nkinyili, Timo Schick, Timofei Kornev, Titus Tunduny. Tobias Gerstenberg, Trenton Chang. Trishala Neeraj. Tushar Khot, Tyler Shultz, Uri Shaham, Vedant Misra, Vera Demberg, Victoria Nyamai. Vikas Raunak, Vinay Venkatesh Ramasesh. vinay uday prabhu, Vishakh Padmakumar, Vivek Srikumar, William Fedus, William Saunders. William Zhang, Wout Vossen, Xiang Ren, Xiaoyu Tong, Xinran Zhao, Xinyi Wu, Xudong Shen, Yadollah Yaghoobzadeh, Yair Lakretz, Yangqiu Song, Yasaman Bahri, Yejin Choi, Yichi Yang. Yiding Hao, Yifu Chen, Yonatan Belinkov, Yu Hou, Yufang Hou, Yuntao Bai, Zachary Seid, Zhuoye Zhao, Zijian Wang, Zijie J. Wang, Zirui Wang, and Ziyi Wu. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transactions on Machine Learning Research, 2023.
Mingjie Sun, Zhuang Liu, Anna Bair, and Zico Kolter. A simple and effective pruning approach for large language models. In International Conference on Learning Representations, 2024.
Chaofan Tao, Lu Hou, Haoli Bai. Jiansheng Wei, Xin Jiang, Qun Liu, Ping Luo, and Ngai Wong. Structured pruning for efficient generative pre-trained language models. In Findings of the Association for Computational Linguistics, 2023.
14

---

Published as a conference paper at ICLR 2024
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix. Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. 13971, 2023.
Johannes Von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, and Max Vladymyrov. Transformers learn in-context by gradient descent. In International Conference on Machine Learning. 2023.
Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Inter- pretability in the wild: a circuit indirect object identification GPT-2 small. In International Conference Learning Representations, 2023.
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yo- gatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang. Jeff Dean, and William Fedus. Emergent abilities of large language models. Transactions on Machine Learning Research, 2022.
Lu Yin, Shiwei Liu, Ajay Jaiswal, Souvik Kundu, and Zhangyang Wang. Junk dna hypothesis: A task-centric angle of Ilm pre-trained weights through sparsity. arXiv preprint 1rXiv:2310.02277, 2023.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. Opt: Open pre-trained transformer language models. arXiv preprint arXiv: 01066, 2022.
15

---

Published as a conference paper at ICLR 2024
Name/Sparsity
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
OPT-13B
11.5
11.5
11.6
11.6
11.9
12.4
13.7
17.8
45.3
40362.3
OPT-30B
10.9
10.9
10.9
11.0
11.1
11.4
12.2
14.6
35.4
4187.5
LLaMa-13B
6.6
6.6
6.7
6.8
7.1
7.8
10.0
20.0
59.9
822.1
LLaMa-33B
5.9
6.0
6.0
6.1
6.4
7.0
8.5
13.7
36.1
355.9
Table 2: Pruning's 's effect on perplexity.
OPT-13B
OPT-30B
LLaMa-13B
LLaMa-33B
22.5
20.0
12-
20.0
10-
Perplesty
17.5
Perplesity Septestt
17.5
Perplexity
10
Perplesity
15.0
15.0
8-
12.5
6-
10.0
10.0
0
25
50
75
0
25
50
75
25
50
75
25
50
75
Sparsity (%)
Sparsity (%)
Sparsity (%)
Sparsity (36)
Figure 5: Pruning's effect on model perplexity.
A PRUNING PERPLEXITY EVAL
In Table 2 present the full range of pruming's effect the perplexity of models consider in this work. We obtain perplexity results by running the pruned model on a randomly sampled subset of C4 validation set, following the precedent of Frantar al. (2023).
B CONSTRUCTING PARAMETERIZED FUNCTIONS IN ICL EVALUATION
While it may be unreasonable to expect the model to infer any arbitrary f. we focus on three natural classes of functions from which we pick f: the class of linear, 2-layered neural network, and decision tree models. These classes are defined as follows: (a) Linear. For each task instance, we generate distinct random hyperplane in the D-dimensional input space the decision boundary. We label each example as positive/negative depending the side of the decision boundary it falls on. (b) 2-layer NN. For each task instance, we generate a 2-layer neural network mapping D-dimensional input vector to binary label y with the following form: W20
where W1, W2 are D x D, D x 2 matrices correspondingly. We draw elements within these two matrices from independent, Gaussian distributions with zero-meat meea and unit variance. We ReLU the activation function σ. (c) Decision tree. For each task instance, we construct a full depth-2 binary decision tree, with each of its four leaf nodes representing one possible label. This maps D-dimensional input vectors to the said labels. We assign each non-leaf node an index from 0 to - 1. We evaluate the tree by traversing from the root, comparing the input vector's value at each node's 's associated index determine the next step: if negative, we go left; if zero or positive, we right. Reaching a leaf node marks the end of traversal, and the corresponding label represents the evaluation result.
C INCREASING INPUT DIMENSION OF ICL EVALUATION
In this section, we further evaluate the effect of pruning on ICL tasks with different input dimensions.
Method. In Section 5, we evaluate pruning's effect on learning linear classification tasks in-context. The task to classify 4-dimensional input vectors. Here, we increase the input vector dimensions to 8 and 16, and observe whether our observation about pruning's 's effect generalize to larger input dimenions.
Results. Figure presents the relationship between task accuracy and sparsity for linear classifica- tion task. We increase the input dimension from 4 to 8 and 16. We observe that tolerating the same 5% relative drop in the average accuracy among all four models with respect to the dense model, we prune to 60%, 80% sparsity linear classification task with 4, 8 dimensional inputs respectively.
16

---

Published as a conference paper at ICLR 2024
OPT-13B
OPT-30B
LLaMA-138
LLaMA-33B
Linear Classification, Dim= 4
Linear Classification, D
Linear Classification, D6m=16
70
70
70
Exact Match (%)
Exact Match (%)
Exact Match (%)
60
60
60
50
50
50
40
40
40
20
40
60
80
20
40
60
80
20
40
60
80
Sparsity (%6)
Sparsity (%6
Sparsity (%)
Figure 6: Pruning's effect on ICL linear classification tasks with input dimension of 4, 8 and 16. We observe that even with increased input dimension, ICL ability of LLMs continue to be robust to aggresive pruning.
Dense models perform close to chance 16-dimensional inputs, thus refrain from drawing conclusions based on these results.
Conclusion. Even with higher dimensional inputs, model's ICL ability remains resilient aggres- sive pruning (>= 60% sparsity).
D ADDITIONAL PRUNING ALGORITHM EVALUATION
our main paper, we focus on single pruning algorithm SparseGPT (Frantar Alistarh, 2023). In this section, we study whether our observation that scaling down LLM size affects fact recall more readily than ICL generalize to another pruning algorithm.
Method. We repeat key experiments with another pruning algorithm called Wanda (Sun al., 2024), Notably, unlike SparseGPT, Wanda does not update the remaining weights after weights removal. The author of Wanda shows that at 50% sparsity, Wanda achieves accuracy that is competitive with SparseGPT.
Results. We observe the same disparate effects from pruning fact recall versus ICL: while moderate pruning hurts fact recall, ICL survives to higher sparsity. Specifically, accepting the same 5% relative drop in accuracy and 50% weights on TriviaQA(Closebookk)) TOieiaQA(Opeenboo)) and Linear Classification tasks. Unsurprisingly, given that Wanda is computationally less expensive, underperforins SparseGPT high sparsities on ICL tasks.
did for SparseGPT results, one may remove 30%, 40%
With our experiments repeated with another pruning algorithm, show that our observation is general.
Conclusion.
E PYTHIA MODELS EVALUATION
this section, we study whether our observation that scaling down LLM size affects fact recall more readily than ICL generalize to another model family: Pythia (Biderman et al., 2023b).
Method. We repeat key experiments with Pythia model family. With SparseGPT, we pruned two variants of the Pythia-120 12B model, the original and "deduped" variant where the model is trained on a deduplicated dataset. They're 're the most capable models in the Pythia family.
Results. Figure 8 shows consistent disparate effect of pruning fact recall versus ICL. Specifically, accepting the same 5% accuracy drop as in all our results, one can remove 20%. 30% and 50% weights for TTivia@A(Closehook), TriviaQA(Openhook) and in-context linear classification tasks.
17

---

Published as a conference paper at ICLR 2024
TriviaAA(ClocBBoo))
TriviaAA(OpenBook)
Lincar Class
Accuracy 18a
Accuracy (%6
Accuracy (%6
40
40
40
20
Hamaa-30b
20
lama-30b
lama-13
20
llama-30b
opt-1 Nb
apt-1.3
apt-1 Mb
20
40
80
40
60
80
Sparsity (%6)
Sparsity (%)
Sparsity (%)
Figure 7: An alternative pruning algorithm (Wanda) shows the same patterns of accuracy drop as findings in our original paper: moderate pruning hurts fact recall (e.g., left, TriviaQA in Closebook setting) while ICL (c.g., right, in-context linear classification) survives to higher sparsity. Specifi- cally, accepting 5% relative drop in accuracy. may remove 30%, 40% and 50% weights TriviaQA(Closeboooo TriviaQA(Openbook) and Linear Classification tasks, respectively.
TrviaQAA/loseedbbok
TriviaoAiF/Oppe-bbook
Linear Classification/CI
Accurasy (86)
20
Accurary
Accurary
40
10
pythia 12b
pythia-122 12b
20
pythia 12b
pythia-11 1ee-112bddee deduped
pythia12bb dodpedd
pythia deduped
10 20 30 40 50 60 70
80
90
10 20 30 40 50 60 70
80
90
10 20 30 40 50 60
70 80 90
Sparsity
Sparsity (%)
Figure 8: Two variants of Pythia- 12B model shows the same patterns of accuracy drop findings in our original paper: moderate pruning hurts fact recall (e.g., left, TriviaQA in Closebook setting) while ICL (e.g., right, in-context linear classification) survives higher sparsity. Specifically, accepting 5% relative drop in average accuracy, one may remove 20%, 30% and 50% weights on (Closeboo,,,,,,o TriviaQA(Openboook and Linear Classification tasks, respectively.
Conclusion. Our observation robust to the choice of model families.
18

---

Published as a conference paper at ICLR 2024
Answer these questions: Q: In Scotland a bothy/bothie is a?
A: House
Q: Which 90s sci fi series with James Belushi was based on Bruce Wagner's comic strip
of the same name?
A:
Figure 10: Examples Prompt for Question-answering Task without Context.
F PRUNING ATTENTION AND FEED FORWARD LAYERS
A growing line of work (Dai et al., 2021; Meng et al., 2022) suggests that feed forward (FFW) layers and the attention layers are responsibly for distinct capabilities considered in this paper - namely, fact recall and in-context learning. To test this, we exclusively prune either type of layers and examine how the model's capabilities deteriorate on fact recall and ICL tasks.
Method. We prune LLaMA-13B model. We either exclusively prune attention layers or ex- clusively prune FFW layers and observe their effects on accuracy of TriviaQA task (fact recall) and linear classification task (ICL). We plot the accuracy as a function of module sparsity. Mod- ule sparsity refers to the fraction of parameters pruned from a specific type of layers (either at- tention FFW), with respect to the total number of parameters of that type.
LLaMa
70.0
50
Exart Match 199
67.5
40
65.0
March
30
20
60.0
Attention
10
Attention
FFW
FFW
57.5
60
Sparsity (N)
Module Sparsity
Results. Figure 9 shows that while attention and FFW layers appear equally important for ICL, FFW layers appear more important for fact recall. For example, pruning 60% FFW layers lead 14% more accuracy degradation than pruning 60% of attention layers.
(a) Linear Classification (b) TriviaQA Close-book Figure 9: Effects of pruning only attention layers versus pruming only FFW layers on fact recall (Triv- iaQA) versus ICL (Linear Classification). Module sparsity computes the fraction parameters the specified module (attention or FFW) that pruning removes. Attention and FFW layers appear equally important for ICL. FFW layers appear more impor- tant than attention layers for fact recall.
Conclusion. Attention and FFW layers show similar importance for ICL; and FFW layers show greater importance for fact recall.
G EXAMPLE PROMPTS
In this section, we present example prompts for Q&A tasks in Section 4 and ICL tasks Section
G.1
QUESTION-ANSWERING WITHOUT CONTEXT.
Our prompt follows the precedent of Touvron al. (2023). We show prompt for Q/A tasks without context in Figure 10.
G.2
QUESTION-ANSWERING WITH CONTEXT.
We show an example prompt we use evaluate Q/A task with the necessary information to answer the question available in-context in Figure 11.
G.3
ICL TASK.
We present the prompt we use to evaluate ICL on linear classification task in Figure 12. Evaluations for other ICL tasks uses analogous prompts.
19

---

Published as a conference paper at ICLR 2024
Answer these questions:
Context: Scattered across the Highlands and rural areas of Scotland, a bothy is small
house that can be used by anyone.
Q: In Scotland bothy/bothie is a?
A: House
Context: "Roses Are Red (My Love)" is popular song composed by Al Byron and Paul Evans. It was recorded by Bobby Vinton and was his first hit.
The song released in April 1962. It reached No. 1 in Australia, New Zealand, Norway, the Philippines, South Africa, and the United States, and was major hit in many other countries well. The song topped the Billboard Hot 100 singles chart on July 14, 1962, and remained there for four weeks. The single was also the first number-one hit for Epic Records. Billboard ranked the record as the No. song of 1962.
Vinton found the song in reject pile at Epic Records. He first recorded it as an R&B number, but was allowed re-record it in slower more dramatic arrangement, with strings and a vocal choir added. Ronnie Carroll version
In the UK, cover version by Ronnie Carroll reached No. on the Record Retailer chart on August 8, 1962, the same week that the Bobby Vinton record peaked No. 15. It peaked No. 7 in the very first Irish Singles Chart published in October 1962.
Other versions
The song was recorded by Jim Reeves in 1963 and released the album Gentleman Jim, one of the last albums released while he was still alive. While it did not chart in the US, it became minor hit in Norway and Germany.
The song was covered by Singaporean female artist Zhuang Xue Fang. in edited Standard Chinese lyrics written by Suyin under title name of, with Ruby Records 1967. In 1962, an answer song, entitled "I "Long As The Rose Is Red", was recorded by Florraine Darlin. The song spent seven weeks on the Billboard Hot 100, reaching No. 62, while reaching No. 15 Billboards Easy Listening chart. It was released by Epic Records (single #9529) and was also produced by Robert Morgan.
Charts
Bobby Vinton version
Ronnie Carroll version
Q: Which singer had big 60s No with Roses Are Red?
A:
Figure 11: Examples Prompt for Question- answering Task with Context.
20

---

Published as a conference paper at ICLR 2024
[5
[3,
--005,-9.-100=
[7.0,4,-7]==1
[2,-2,9,-3]=
-9,-5,-5--]==
[4,-1,-10,-8]
[-10--1,336] --
[3,6,-5.-8]=
[1,7,-2,6]
--
[-6.-1,-6-9]
[1,0,-10.-9]
[-5,5.336]]
[4,6,-10,-7]
[4.5.-6.110]
[-10 =
[-7--2,-7,99]
[=,1,-7,4)==
[2,22,33-100
[1,44,4.9]=
[7.0.6.-10]
[2.7,66.1]==
-1.
[-8,3,7,-7==1
[0,7,-3,-8]=
[0,2,0.0]=-1
[-7,9,28]
[1,-8,-9,6]=-1
6,
-
2-,,,,
(33
Figure 12: Examples Prompt for ICL on Linear Classification Task.
21

---

Published as a conference paper at ICLR 2024
H ANALYSIS ON ACCURACY IMPROVEMENT
On many tasks, such as ICL on algorithmic tasks in Section 5, we observe accuracy improvement from pruning. In this section, we provide further analysis this phenomenon.
Method. We focus on the ICL task of check-sorting and check-even from Section 5, where the pruned LLaMA models see significant accuracy improvement. We analyze the prediction of the pruned model, in relation the following two answers: 1). Answer without context. We compute the classification output with the highest log likelihood, without all the in-context examples. The prompt thus looks like "[1, 0, 3, -5]=" without any other information such in-context training examples. The LLM thus classifies this input array exclusively based on likelihood of classification label as a continuation of the prompt. We denote this classification outcome as the answer without context. 2). Ground-truth answer.
Results. In Figure 13, we present the percentage time when the prediction of the pruned model matches the answer without context versus the ground truth. We observe that accuracy improvement often mirrors a departure of the model's prediction from answer without context (the solid line dropping whilst the dashed line raising). This suggest that for some pruned models (for example LLaMA-30B at 60% sparsity), the presence of in-context examples changes the prediction of the LLM more than the dense models.
Conclusion. We hypothesize, without testing, that pruning may improve task accuracy by enhancing the effect of contextual information its prediction.
LLaMa-13B without-context
LLaMa-33B without-context
LLaMa-13B ground-truth
LLaMa-33B ground-truth
Results for Check Sorting
Results for Check Even
100
100
80
80
d
g
60
60
40
40
0
20
40
60
80
0
20
40 Sparsity
60
80
Sparsity %
Figure 13: Analyzing the accuracy improvement of pruned model on ICL tasks.
22

---

Published as a conference paper at ICLR 2024
I CHARACTERIZING ERRORS FROM NUMERICAL PRECISION
All our experiments run on TPU v3. The TPU hardware does not support float16 (IEER 754 half precision floating point format), which are the formats used for training the OPT/LLaMA models. We thus use closely related floating point format called bfloat (Brain Float) instead to evaluate these models. Both bfloat 16 and float16 are floating point numbers requiring 16 bits to represent, the key difference is that 6 format can represent numbers in a slightly wider range of values, with slightly reduced precision compared with float16. For this reason, we expect our results to contain small systematic errors. In this section, we characterize the difference between our results in b and standard results float16.
Perplexity and accuracy results. In Figure 14, we plot the perplexity and accuracy in next token prediction in 6 on TPU v3 and f on A A100. We observe that the average accuracy difference is 1.0%, 0.89%, 0.70% and 0.68% for OPT-13B, OPT-30B, LLaMA-13B and LLaMA-33BB The perplexity grows to very large numbers at high sparsity, so we report the average perplexity difference for sparsity level less than or equal 50%. The average perplexity difference 0.69, 0.19, 0.13. We note that 1). the difference in accuracy and perplexity due to numerical precision systematic. The accuracy of next token prediction raises by a similar amount across all sparsity levels. 2). the difference in next token prediction accuracy is small (<1%).
0.86,
ОРТ. OPT models see higher difference in accuracy and perplexity. This is because its GPU implementation uses mixed precision arithmetic,, where the attention score computes in float32. Our compiler and runtime, however, does not support mixed precision arithmetic, we therefore compute entirely within bfloat16.
brloat16 ppl --.
flot116 pp!
brooat16 accuracy
--.. float16 accuracy
opt-13b
opt-30b
llama-133
llama-30b
22.5
50
50
60
60
20.0
Perpeeddy
20.0
17.5
30
40
Accuracy
Perplesdy
17.5
15.0-
2o S (60
30
40
Parplexky
50
40
Accurads
Perplexky
10.
40
Accuracy
12.5
20
30
12.5
6
100
10.0
10
10.0
50 Sparsity (36)
50
50
Sparsity
Sparsity
Sparsity
Figure 14: Characterizing the difference between f and 6
J BEAM SEARCH + TOLERANT ANSWER MATCHING
23

---

Published as a conference paper at ICLR 2024
TriviaQA Closebook
TriviaQA(F) Openbook
100
100
Accuracy (96)
60
(9)
60
40
33B (BeamSearch)
33B (BeamSearch)
33B (Greedy)
33B (Greedy)
20
13B (BeamSearch)
13B (BeamSearch)
13B (Greedy)
13B (Greedy)
20
40
60
80
20
40
60
80
Sparsity (%)
Sparsity (%)
Figure 15: Reproducing TriviaQA Closebook and TriviaQA(F) Openbook results using beam search (beam size=3) and more tolerant answer matching technique on LLaMA-13B/33 13B/33B: we generate 32 tokens and check the correct answer appear anywhere in the generated response. Our conclusion remains unaffected: using our current experimental design, averaging over two LLaMA models, one can prune 30% and 40% weights on Closebook and Openbook TriviaQA task, respectively. Using beam search, one can prune 30% and 50% weights on Closebook and Openbook TriviaQA tasks, respectively. The ability to retrieve answers from provided context remains more resilient to pruning than the ability to recall information learnt during pre-traie pre-training.
24

---

Published as a conference paper at ICLR 2024
K
MORE TASKS
Lambada
Translation (WMT14 EN-FR)
80
60
30
Accuracy (%)
BLEU Score
40
20
20
LLaMA 13B
llama-13 13b
opt-13b
10
llama-300
opt-30b
LLaMA 30B
20
40
60
80
0.0
0.2
0.4
0.6
Sparsity (%)
Sparsity (%)
Figure 16: Lambada
Figure 17: Translation
L IN-CONTEXT LEARNING ON ALGORITHMIC TASKS
We use the following ICL task and for each task, generate total of N task instances. Each task instance consists of a set of in-context examples along with a test input. a). Check Sorting. The task is determine, whether D-element input array is sorted. We label each example as positive/negative depending on whether the input array is sorted or not.b). Check Contains Even Number. The task is determine, whether a D-element input array contains each example as positive/negative depending on whether the input array contains an even number. We present the pseudo code solution these algorithmic tasks in Figure 18.
Method.
number. We label
All task instances consist of K positive in-context examples and K negative in-context examples, We draw all random numbers from integers between-Jy 10 inclusive. We set N=2048, D=4, K=16.
Sparsity results. Figure 19 shows the accuracy of pruned models versus sparsity. We can remove substantial number of parameters from the models without affecting their ICL ability. Specifically, the average accuracy of the pruned models on check sorting and check contains even number is within 5% of the dense models up to 60% and 70% sparsity respectively.
Conclusion. Our results suggest that when pruning to moderate sparsity levels (60-70%) the model's ICL ability stays largely intact. Compared with pruning on fact recall tasks, one can prune to much higher sparsity levels while maintaining even improving ICL performance.
def contains _eterrd_eeeeddd (A)
se_sorted4d (A)
False
2
s Truc
or A[0
$
= and A[O] <= A[1]
or A [1
8
and A[1] <= A[2]
c = C or A
8 == 0
return A[2] A[3]
or A
$
= 0
return
7
Figure 18: Algorithmic tasks for in-context learning evaluation. (Left) Check if a 4-D vector is sorted. (Right) Determine whether 4-D input vector contains an even number.
25

---

Published as a conference paper at ICLR 2024
OPT-13B
OPT-30B
LLaMA-13B
LLaMA-33B
Check Sorting
Contains Even
70
70
Exact Match (%)
Exact Match (%)
60
60
50
50
40
40
30
30
0
25
50
75
25
50
75
Sparsity (%)
Sparsity (%)
Figure 19: ICL withstands aggresive pruning (60-70% spasity). Accepting a relative average (over 4 models) accuracy drop of 5% w.r.t. the dense models, we can prune to 60% and 70% check sorting and check contains even number tasks, respectively.
M PRUNING ALGORITHMS DESCRIPTION
In this section, we present implementation details of pruning algorithms we evaluate in this work.
M.1 SPARSEGPT.
The SparseGPT algorithm prunes weights in language model layer by layer. For each layer's weight W,, where is the layer index, it produces a binary mask Mt and updated of weights Wi to minimize the layerwise reconstruction loss |WiX (M Wi) 1|. where @ denotes element-wise multiplication, and X correspond to inputs to layer 1. Such inputs are computed using a small set of calibration data, randomly drawn from the training dataset. Note that rows in Wi do not interact with one another, SparseGPT thus prunes each row independently.
Weight selection and repair. SparseGPT builds on optimal brain surgeon (OBS) (Hassibi et al., 1993; Frantar et al., 2021), classical approach in weight pruning. For given weight row upon removing an element at position m, OBS suggests the optimal weight update 8n to compensate for the effects of removed weights, and characterize the associated reconstruction error em follows:
8m
Wm
H. S,m
Cy
Wm
[
H
mo
mm
The SparseGPT thus prunes rows of Wi independently and in parallel. Within each row, the algorithm prunes weights vectors of size 128. The algorithm selects weights for removal based on the lowest reconstruction еттог After removing weights. SparseGPT itcratively applies the OBS weight update. An important difference between SparseGPT and OBS is that while OBS adjusts all remaining weights to compensate for removed weights, SparseGPT only updates subset of remaining weights to reduce the computational complexity associated with computing the Hessian inverse. This complexity reduction enables SparseGPT to scale to very large language models (175B), See pseudocode below for details about the exact subset of remaining weights updated by SparseGPT.
Pseudocode The following pseudocode is simplified version of SparseGPT largely based Algorithm. of Frantar & Alistarh (2023). This algorithm prunes the layer weight matrix to p% unstructured sparsity given inverse Hessian H-1 (XXT)-1, weight vector size B.
26

---

Published as a conference paper at ICLR 2024
Algorithm The SparseGPT algorithm.
1: M -- drww Xdent
> binary pruning mask block pruning errors
2: < 0d B
3: for i =0 B, 2B..... do
4: M.j.(jB..) + mask of (1 - -%%) weights we in W W.(++B) with largest w 'Icc
5: Apply the OBS update
subset of remaining weights.
6: for j i,.... i+ B do
7:
E.j-i / 5j E.j-i t (1 M. E.j-i
8:
9:
W..++i) f- m.)s(im)) E.j-i (H"-la-m
10: end for
11: W,, < W W.(+(BB) - F. -E- FB.H.(4B).04 (((+
12:
end for
W W.
set pruned weights
M.2 WANDA
Wanda(Sun et al., 2024) similarly prunes a language model layer by layer. Within each layer, it independently prunes individual rows in the weight matrix. The inputs layer, denoted as X, have the shape (N, L. C.n). Here, N, L, and Cin refer to the batch size, sequence length, and input hidden dimension size, respectively. The inputs to the layer W derive from small set of calibration data. The weight matrix to prune has shape (Cin: Cou)). Cin and Cout refer to the input and output hidden dimension sizes. Wanda scores each weight W., using the following equation:
Si = [ |X
where denotes the absolute value function. Wanda removes p% of the weights with the lowest scores in each row W, Notably, unlike SparseGPT, there is no weight update after weights removal.
N STRUCTURED PRUNING (LLM-PRUNER) RESULTS
In this section, we test our findings using structured pruning algorithm LLM-Pruner (Ma et al., 2023).
Configuration. We use LLaMA7B/13B instead of 13B/30B in the main paper because appears to consume significantly more GPU memory than SparseGPT, and 30B model does not fit within our hardware setup.
Following the recommended setup of Ma et al. (2023) in their code repository, we skip the first 4 layers and last 2 layers for all LLaMA models we test. Following Frantar Alistarh (2023), we sample calibration data of maximum sequence length (2048 tokens) from C4 dataset. The number of samples in the calibration data configured to maximum allowed by our system memory size (167 GB CPU memory 80GB GPU memory), which is for 7B and for 13B LLaMA model.
Since Ma et al. (2023) focuses on the low-sparsity regime (0-20%) in the paper, we similarly focus on this regime of sparsity. Specifically, we prune each of the prunable layers (which excludes 4 initial and 2 final layers) to 5%, 10%, 15%, 20%, and 25% sparsity. We refer to this sparsity ratio as local sparsity, since it refers the sparsity ratio within prunable layers only. In Figure 20, report global sparsity to remain consistent with our main paper global sparsity refers to the number of remaining parameters divided by the total number of parameters of the full model.
Results. Figure 20 shows that our main conclusion that fact recall deteriorates faster than ICL under down-scaling remains valid with the structured pruning algorithm. Specifically, averaging over two models. we can prune to 0%, 5% and 20% local sparsity Closebook (TriviaQA), Openbook (TriviaQA(F)) and ICL linear classification tasks, respectively, without dropping more than 5% relative accuracy.
27

---

Published as a conference paper at ICLR 2024
TriviaQA Closebook
TriviaQA(F) Openbook
ICL Linear Classification
13B LLaMA
7B LLaMA
0.6
Accurscy 186)
Accurrry 146
40
Accuray
13B LLaMA LLaMA
10
LLaMA
15
10
Weights
Weights
Figure 20: Structured pruning results.
28


Reviews:
[{"review_id": "5piCQJh0Sc", "reviewer": "Reviewer_ecKe", "summary": "The authors study the effects of weight pruning, a popular technique for reducing model size, on the two core capabilities of LLMs: (a) recalling facts presented during pre-training and (b) processing information presented in context. They find that existing pruning techniques affect these two abilities of LLMs quite differently. The paper presents a detailed analysis of the experimental results, which show that the effects of down-scaling LLMs depend on the specific pruning technique used. The authors conclude that there is a trade-off between model size and performance, and that the optimal model size depends on the specific task and dataset.", "questions": "How the structured pruning methods, e.g., LLM-Pruner, performs on these two LLM capabilities?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The paper investigates the impact of down-scaling large language models on their capabilities, which is an important topic in the field of natural language processing. \n- The authors provide a detailed analysis of the experimental results, which can help researchers and practitioners better understand the trade-offs between model size and performance. \n- The paper provides insights into the development of more efficient language models, which are becoming increasingly important for a wide range of natural language processing tasks.", "weaknesses": "- The paper is empirical in nature, and the authors acknowledge that their observations may not generalize to the full spectrum of tasks and large language models. \n- The study focuses on evaluating two pruning algorithms that are unstructured pruning, evaluation on structured pruning methods are expected. \n- The study could include other types of tasks, like NLI, classification, summarization, to make the study more solid.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "N371zoQP03", "reviewer": "Reviewer_xD3m", "summary": "Pruning parameters from large language models can affect aspects of model performance differently. The authors strive to characterize these effects by separating fact recall from in context learning. They explore the relative impact of pruning on several different tasks using several base models and multiple pruning techniques. Overall they find that even moderate pruning can degrade fact recall settings, here in-context learning seems more robust.", "questions": "It seems that Dense Pruning of 30B -> ~13B underperforms the unpruned 13B param model, right? I'd love to see more discussion here about what is going on there.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The settings for evaluating fact recall and in context learning seem useful in general.\n\nMultiple settings for pruning to push for more robust result interpretations\n\nMultiple model families were used in evaluation.\n\nA range of tasks were presented.", "weaknesses": "Fact Recall and In Context Learning are some reasonable aspects, but the authors could have considered more. Detailed Instruction Following, and Heavy Reasoning feel like other key aspects, as well as the ability to learn from Few Shot inline. I would have loved to see some more details.\n\nAre all In Context Learning tasks equally difficult? Could a few more gradations be helpful here?\n\nAre there a few more settings that one could use for evaluating model performance? The set of tasks seems rather small.\n\nI'm assuming that pruning is primarily used to increase inference speed, right? If that's the case, I'd like to see tradeoffs between accuracy and inference speed be presented here.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IgDHerI2sU", "reviewer": "Reviewer_2TYK", "summary": "Edit - the authors have addressed concerns within my review and also contrasted the novelty of their work with previous pruning/ICL studies.  I'm increasing my score accordingly.\n\nContinuing off recently proposed LLM pruning methods (i.e., SparseGPT and Wanda), the authors explore the effect of LLM pruning on both parametric knowledge (i.e., knowledge memorized by the model) and knowledge learned via ICL.  Towards this end, the authors explore the effect of pruning for pretrained OPT-13B/30B and LLamA-13B/33B models.  The authors main contributions are experiments evaluating the pruned-sparsity levels of the aforementioned models versus (a) parametric knowledge and (b) ICL knowledge, as measured via several Q&A tests; parametric knowledge is measured using a closed-book Q&A test, while ICL knowledge is measured using an open-book Q&A test where the in-context prompt is contains a counter-factual answer compared to the training data.  Furthermore, the authors also test ICL knowledge by providing in-context information detailing parametric functions and measure the models predictive accuracy in this case.", "questions": "-Why did the authors not consider the GINC dataset for ICL, from Xie et al's \"An Explanation of In-context Learning as Implicit Bayesian Inference?\"\n\n-\"From the OPT family, we evaluate the two largest models that fit in our hardware\nsetup\" <- Please state the hardware setup\n\n-In Table 1, please define what is meant by \"Context Type\"", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The topic is important and interesting and, at a high-level, the experimental design make sense.  Furthermore, as LLM parameter sizes continue to grow, the question of how new pruning methods (i.e., LLM-Pruner, SparseGPT, and Wanda) affect a model's parametric and ICL knowledge is important.  However, more care is required to ensure model performance is accurately being measured.", "weaknesses": "# Evaluation\nThe major weakness of the paper is the effective testing of LLM parametric and ICL knowledge.  In particular, how do the authors verify that the in-context evidence contradicts a fact present in training data?  Although LLama was trained on\npublicly available data, it is not a simple matter to verify that the answers in the Q&A datasets align or misalign with the massive dataset used to train LLamA (note that, the dataset itself was never released, so all public datasets would have to be evaluated in their entirety for verification).  In this case, it is not clear how Verifying that \"answers do not exist in the pre-training corpus\" is possible.  While the authors discuss previous work which has explored LLMs' abilities to override in-memory/parametric knowledge, such works set up measurement guardrails to do so through fine-tuning, e.g.:\n- \"DissentQA. Neeman et al. (2022) constructed the DissentQA dataset from the NaturalQuestions dataset. It contains pairs of\nquestions and evidence for a made-up answer that is different from the factual one. It assesses\nwhether the model can override its memory formed during pre-training with new context.\"\n\"Given that the anticipated made-up answers are randomized and\ndifferent from the factual ones, the model cannot depend on memorization from its pre-training\ndata to generate responses. This evaluation framework rigorously assesses the model’s ability to\noverride its pre-training data with new, context-specific information.\" <- Neeman et al. (2022) fine-tune their evaluated T5 models, thus ensuring that the parameterized answers are learned and relevant questions and answers are actually counter-factual (relatedly, gold passages are considered in Longpre et al. (2021)). This work does not fine-tune the evaluated LLMs, thus relating to the earlier criticism on the validity of the presented results.\n\nFurthermore, the exact measurement of accuracy used in the paper is potentially incorrect and too conservative for recently release instruction-tuned LLMs like LLaMA and OPT.  From the text:\n- \"Answers are the model’s prompt completions produced by greedy decoding. We report the\npercentage of answers that exactly match ground truth.\" <- Two important remarks: greedy is known to be extremely suboptimal for recent\ninstruction-tuned LLMs, and an exact match is not necessarily a fair metric.  Such chat models are known to be extremely wordy, so if the model produces some lead up text followed by the correct answer, this metric discounts such a correct response.  For the former, it makes sense as a fair, reproducible benchmark across different sparsity percentages per model (e.g., nucleus sampling would produce differing results between runs), please include in the text why greedy is used.  However, note that the latter is an extremely important problem which biases all related results.\n\n# Claims\nSeveral claims require revision or further discussion.  In general, wrt to key contributions, it is necessary to discuss how the presented methodology differs from previous work.  E.g., the SparseGPT paper itself reports zero-shot performance for different datasets at different sparsity levels (which effectively tests parametric knowledge), how does the presented benchmark differ from this?  Why does the presented work differ in conclusions wrt parametric knowledge compared to the SparseGPT paper, i.e., SparseGPT showed high sparsity while retraining zero-shot performance.  Why is this not the case in the presented work?  These types of questions, and their ensuing answers/justifications, require significant discussion.  More examples from the paper:\n- \"From work on image classification, however, we know that down-scaling neural networks affects more than just top-line metrics or task accuracy. Pruning, for example, can introduce biases (Hooker et al., 2019) or disproportionate affects on certain subsets of the data (Jin et al., 2022).\" <- This claim is too strong, it makes it seem as though it is a certainty that such effects occur given down-scaling.  However, a significant amount of work has shown that pruning is an effective tool for vision models.\n- \"It is difficult to assess these abilities in isolation, as a standard downstream task needs to process the\ninformation provided in context as well as access the information stored in weights.\" <- Please contrast related work which has previously explored zero-to-many shot ICL performance (across different target applications); see the following for an extensive overview:\nDong et al, \"A Survey on In-context Learning\", https://arxiv.org/pdf/2301.00234.pdf\n- \"Improve inference efficiency. Our work reveals that scaling down model size alone has little impact\non tasks demanding processing information in the LLM’s context. Practitioners may thus use our\nfindings to identify scenarios where decisions could be routed to a smaller model instead of a larger\none without hurting task performance (Chen et al., 2023; Dohan et al., 2022).\" <- The latter work already explores how the parameter size affects performance.  In particular, the Wanda paper already tackles the question of how pruning affects ICL performance (and compares to SparseGPT)\n- \"Our work differs from prior scaling studies in two ways: while prior work (Kaplan et al., 2020b) studies\njoint scaling of both pre-training corpus size and model size, we focus on scaling model size alone.\nFurthermore, instead of measuring task performance, we focus on foundational capabilities of LLMs–fact recall and ICL. These capabilities drive the success for many real world applications of LLMs\" <- This is wrong for a number of reasons.  Firstly, \"we focus on scaling model size along\" is not a valid contribution, as this would, by definition, be provided in the study of \"joint scaling of both pre-training corpus size and model size.\"  Secondly, the work of Kaplan does not study pruning, but rather LLM model size->training->resulting performance.  It is necessary to demarcate the difference between these two paradigms.\n-\"In-weight versus in-context learning\" <- Please explain how the considered work differs from Longpre et al 2022, which extensively explores In-weight versus in-context learning.\n- \"the versatility of LLMs calls for a different approach to assessing pruned models. Our work begins to fill this gap, proposing to evaluate pruning’s effect on fact recall and ICL.\" <- As previously mentioned, what the authors define as fact recall is equivalent to the task of zero-shot question answering; the effect of pruning on various tasks has been explored, e.g., within the papers of the pruners specifically used within this work (SparseGPT and Wanda), as well as in the LLM-Pruner paper.  Furthermore, the effect of pruning an LLM to various sparsity levels on ICL was extensively explored in the Wanda paper.  Please revise your contributions, and position them within the context of previous works.\n-\" In all the above settings, as a simple point of comparison, we measure the effect of downscaling on perplexity\" <- Please note in the paper that this was previously considered in both the SparseGPT and Wanda papers.\n-\"  we focus on foundational capabilities of LLMs – fact recall and ICL\" <- Fact recall is zero-shot Q&A, which may be thought of as a specific task.  Please adjust this claim.\n\n# Presentation\nOverall, the writing and presentation of the discussed work could be significantly improved.  E.g.:\n- \"removing more than 30% of weights leads to significant (> 5%, relative) accuracy degradation on fact recall related tasks (Figure 1, left). Fact recall suffers similarly from dense down-scaling.\" <- Please have some text which segways from the first paragraph (page 2) to the list of 3 bold-faced items.  The intro currently reads like a collection of text/paragraphs which do not blend into one another.  E.g., combine all the bold-faced-starting paragraph in page 2 into a single paragraph, which: -States the paper shows the following dichotomy wrt pruning LLMs.  For fact recall/parametricknowldge, minimal pruning significantly degrades performance.  [insert your bold-faced-starting text here] In stark contrast, large-scale  pruning does not significantly degrade ICL performance. [insert your second bold-faced-starting text here]. [insert your third bold-faced-starting text here]\n- Same comment for italicized-starting-text, which proceed bold-starting-text; please segway the various paragraphs together.  It is very difficult for a reader to understand the point that is trying to be made when sentences exist independently.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LhxH9IY9kz", "reviewer": "Reviewer_B31e", "summary": "The paper investigates the effects of pruning and down-scaling large language models (LLMs) on the model capabilities. Specifically, the authors focus on two abilities of modern LLMs: (1) the ability to process information stored in the weights (fact recall) and (2) the ability to process information that is available in context. To evaluate down-scaled models on these capabilities, they use a suite of benchmarks covering four tasks, open-book QA, closed-book QA, overriding QA, and learning tasks (i.e., model needs to understand underlying function based on examples given as in-context learning). Experiments on 6 base LLMs, each with 9 different sparsity levels demonstrate different model behavior in terms of the two capabilities. Model ability to process information in weights degrades with moderate level of pruning (>30%), while model ability to process information in context does not really degrade even with aggressive pruning (up to 70%).", "questions": "- For learning tasks evaluation, why only consider task with scalar values as labels? I understand this needs to be something that model can generalize through the examples, but if we focus on language capability of the model, I would expect that a natural language task is used instead.\n- For ICL results, it seems the performance drops significantly (not gradually) from 70% above, do you have intuition why?\n- As the paper only use some particular pruning methods, do you have any opinion on whether the same findings will hold for other pruning methods? It would be good to have a short discussion on the difference between them.\n\n**Things to improve the paper**\n- Section 5 is hard to follow without examples, there are multiple notations without explanation, e.g. K (page 7), x, D=4, N=32, etc\n- Would be good to release the version of datasets that are used for benchmarking", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "Overall, I think the paper discusses an important question regarding the trade-offs of having smaller-scaled models and its impact to model capabilities. The experiments are well-thought, with the use of different benchmark tasks to isolate different model capabilities being tested and the use of different base LLMs to see that the effects are similar across different model families. I think the main findings of this paper will be useful for future work in this area. The paper is well-written.", "weaknesses": "- Although down-scaling and pruning are the main topic of the paper, the technical details on methods used is very limited (even in the Appendix too). If space is an issue, I would suggest to cut down the paper motivation which is repeated multiple times throughout the paper.\n- Relatedly, there is very little discussion regarding down-scaling vs. pruning. For general readers it would be helpful to understand what are the difference between the two, and is one a specific version of the other?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)