Paper: (Dynamic) Prompting might be all you need to repair Compressed LLMs (ICLR-2024)

Reviewer comment (Presentation):
The paper's clarity and organization require improvement. In particular, the flow of the paper can be challenging to follow, particularly in the experiment section. Section 3.3.1 is titled "IDP results," while 3.3.2 is titled "More Comparisons and Ablation Studies," but it is not clear what differentiates these experiments. Additionally, the discussion of Table 1 and Table 2 occurs after the discussion of Table 3, which can be confusing for readers. The suffixes /short and /large in Table 1 and Table 2 are not adequately explained in the main paper. Furthermore, the details of Table 3 and Table 4, specifically the 3 settings of parameter sizes for LoRA, prompt, and ptune, are not provided in either the captions or the main paper.

Author response:
**Second**, regarding our manuscript's writing and organization, we will implement the suggested revisions. Notably, sections 3.1.1 and 3.1.2 will be renamed to more clearly delineate their content. In line with your recommendation, we will reorder the Table to match the sequence of discussion. For hyper-parameter clarity, we will update the relevant section to state the numbers explicitly. 
For context, "small" prompts comprise 26 tokens, and "large" prompts comprise 100 tokens. The case of OPT-6.7b and Llama-7b models, the embedding size "e" is 4096. Regarding "m," the number of distinct prompts used, as mentioned in section 3.3.1, is 2, with "n" being 50 and 100. The three prompt sizes employed for the prompting baseline are 25, 50, and 100, consistent across Ptune's three separate settings. LoRA is fine-tuned with hidden dimensions of 2, 3, and 4, and the alpha parameter is set at 36. As suggested, we will also include the baseline uncompressed performance of Llama-7b and OPT-6.7b in table 1 and 2 as well

Reviewer comment (Reproducibility):
There are some critical details missing in the paper, particularly concerning hyperparameters. For example, the paper does not specify the prompt length and number, which are denoted as n and m in Section 3.2. Furthermore, there is a lack of information regarding the settings for LoRA, ptuning, and soft prompts. Moreover, it would be helpful if the authors presented the results of the original LLMs to demonstrate the performance drop resulting from compression algorithms.

Author response:
**Second**, regarding our manuscript's writing and organization, we will implement the suggested revisions. Notably, sections 3.1.1 and 3.1.2 will be renamed to more clearly delineate their content. In line with your recommendation, we will reorder the Table to match the sequence of discussion. For hyper-parameter clarity, we will update the relevant section to state the numbers explicitly. 
For context, "small" prompts comprise 26 tokens, and "large" prompts comprise 100 tokens. The case of OPT-6.7b and Llama-7b models, the embedding size "e" is 4096. Regarding "m," the number of distinct prompts used, as mentioned in section 3.3.1, is 2, with "n" being 50 and 100. The three prompt sizes employed for the prompting baseline are 25, 50, and 100, consistent across Ptune's three separate settings. LoRA is fine-tuned with hidden dimensions of 2, 3, and 4, and the alpha parameter is set at 36. As suggested, we will also include the baseline uncompressed performance of Llama-7b and OPT-6.7b in table 1 and 2 as well

Reviewer comment (Evaluation):
The performance enhancement achieved by IDP, in comparison to the baseline models, tends to be relatively modest. Across Table 1 to Table 4, the observed improvement in performance typically falls within the range of 1%. In cases where the performance gap is quite narrow, it may be helpful to conduct a significance test to assess the statistical significance of the results.

Author response:
**Fourth**, regarding the significant improvement over baselines, following your suggestion, we have rerun our experimentation using compressed GPTQ Llama-7b for five trials and conducted a T-statistical significant test. The table below marks the T statistical differences between our baselines and IDP and our p-value. Note that since our p-value is below 0.05 and the t-statistic indicates a lower overall sample mean, IDP does produce statistically significant results. In our revision, we will include this ablation study in our result section.
| model   | type   | size | mean               | std                 | T-stat                 | p-value                |
|---------|--------|------|--------------------|---------------------|------------------------|------------------------|
| llama-7b| prompt | 0.1  | 57.454722222222216 | 0.7293593302514931  | -4.132946245610187     | 0.0016425536487376573  |
| llama-7b| prompt | 0.2  | 57.89944444444445  | 0.7231221120492015  | -1.9204822699592363    | 0.04552389734223814    |
| llama-7b| prompt | 0.4  | 57.932500000000005 | 0.6070659356954821  | -1.9092908112014755    | 0.04631619380664712    |
| llama-7b| lora   | 4.4  | 57.89407407407407  | 0.29266610291661366 | -2.602985906153462     | 0.015735718097273196   |
| llama-7b| lora   | 6.7  | 57.88333333333334  | 0.4033454568766604  | -2.5069461400629938    | 0.01827219202740582    |
| llama-7b| lora   | 8.9  | 57.91481481481482  | 0.36280647361976315 | -2.36792071959218      | 0.022696752987284078   |
| llama-7b| IDP    | 0.6  | 58.28222222222222  | 0.6857403424711355  | None                   | None                   |

Reviewer comment (Evaluation):
One of the claimed contributions of this paper pertains to the exploration of the reasons behind the performance enhancement achieved by IDP. It introduces hypotheses, denoted as H0 and H1, and conducts hypothesis testing. It's worth noting, however, that this process doesn't adhere to the conventional standard of hypothesis testing, which typically involves the rejection of the null hypothesis (H0). Instead, the authors undertake an analysis of the behavior of IDP and the baseline models, drawing the conclusion that H1 is the more plausible explanation.

Author response:
**Fifth** Regarding the reviewer's criticism of our hypothesis:
* Firstly, we acknowledge that our use of the term "hypothesis" might be misconstrued as a reference to a statistical hypothesis. To avoid this confusion, we will revise our introduction and discussion, opting for the term "argument" as a more appropriate descriptor.

* Secondly, addressing the distinction between "data attrition" and "knowledge rechanneling," and the potential impact of external data sources on self-attention mechanisms, we reference the example of LoRA, showcased in Figure 6, as a counterpoint. LoRA modifies the weight representation within the self-attention module, a change that should theoretically result in notable alterations in attention patterns. However, our findings reveal that even when LoRA is trained on the same data source as Prompt and Ptune, it exhibits a remarkably high self-attention similarity. Figure 6 presents the average cosine similarity across various size settings as outlined in Tables 1 and 2 for LoRA, Prompt, and Ptune. This consistency in behavior across different sizes suggests that introducing a new data source does not necessarily necessitate a change in attention patterns. Our conclusion is based on the observation that different attention behaviors exist between the two methods. This indicates an alternative approach to recovering performance and output, one that does not rely solely on restoring self-attention patterns.  As a final point, we use Figure 7 to illustrate that reducing the number of tokens can still effectively recover average performance compared to the baseline. This provides further evidence to suggest the possibility of recovering output without necessarily restoring self-attention and that this phenomenon is not strongly related to the number of parameters or data sources.

Reviewer comment (Evaluation):
The distinction between "data attrition" and "knowledge rechanneling" is somewhat subtle and challenging to precisely define. The evidence presented in the paper, including the shift in attention patterns, may not be compelling enough to definitively support H1, as factors such as the incorporation of new data sources could also lead to changes in attention patterns, making them difficult to reject.

Author response:
**Fifth** Regarding the reviewer's criticism of our hypothesis:
* Firstly, we acknowledge that our use of the term "hypothesis" might be misconstrued as a reference to a statistical hypothesis. To avoid this confusion, we will revise our introduction and discussion, opting for the term "argument" as a more appropriate descriptor.

* Secondly, addressing the distinction between "data attrition" and "knowledge rechanneling," and the potential impact of external data sources on self-attention mechanisms, we reference the example of LoRA, showcased in Figure 6, as a counterpoint. LoRA modifies the weight representation within the self-attention module, a change that should theoretically result in notable alterations in attention patterns. However, our findings reveal that even when LoRA is trained on the same data source as Prompt and Ptune, it exhibits a remarkably high self-attention similarity. Figure 6 presents the average cosine similarity across various size settings as outlined in Tables 1 and 2 for LoRA, Prompt, and Ptune. This consistency in behavior across different sizes suggests that introducing a new data source does not necessarily necessitate a change in attention patterns. Our conclusion is based on the observation that different attention behaviors exist between the two methods. This indicates an alternative approach to recovering performance and output, one that does not rely solely on restoring self-attention patterns.  As a final point, we use Figure 7 to illustrate that reducing the number of tokens can still effectively recover average performance compared to the baseline. This provides further evidence to suggest the possibility of recovering output without necessarily restoring self-attention and that this phenomenon is not strongly related to the number of parameters or data sources.

Reviewer comment (Presentation):
In Table 3, what do /small and /large mean? Do they correspond to "individual prompts"? How long are they?

Author response:
**Question 2**: Long refers to 100 tokens and shorts 50 tokens, and yes, they correspond to the individual prompts.

Reviewer comment (Experiments):
In all experiments, is the performance of the original models comparable? By showing the performance of un-compressed models, we can clearly see how much compression impairs the model performance.

Author response:
**Question 3**: We will include the performance of the baseline non-compressed model in Tables 1 and 2 for future reference.

Reviewer comment (Theory):
The paper could have benefitted from a more detailed explanation or a separate section dedicated to the underlying theory or intuition behind why IDP works the way it does.

Author response:
We thank the reviewer for their support of our paper. As per your advice, we will revise the manuscript to include a section explaining the theory and intuition behind IDP works. But in short,  IDP’s mean attention metric captures the average token-to-token attention between the input with any given prompt. We assume that prompts eliciting higher levels of activity in response to a given input are likely to be more contextually relevant to that input compared to those that do not. Essentially, what we are measuring is a rough proxy for sequence-to-sequence attention, where higher mean attention values indicate a stronger and potentially more relevant interaction between the prompt and the input sequence.

Reviewer comment (Theory):
Could the authors elaborate on the foundational theory behind IDP's ability to rechannel inherent knowledge? Is there any theoretical upper limit to its efficiency?

Author response:
**First** Regarding the foundational theory behind IDP/Prompt’s ability to rechannel inherent knowledge, this is an attribute that we observe and show through deduction.  For instance, we reference the example of LoRA, showcased in Figure 6. LoRA modifies the weight representation within the self-attention module, a change that should theoretically result in notable alterations in attention patterns. However, our empirical findings reveal that even when LoRA is trained on the same data source as Prompt and Ptune, it exhibits a remarkably high self-attention similarity. Figure 6 presents the average cosine similarity across various size settings as outlined in Tables 1 and 2 for LoRA, Prompt, and Ptune. This consistency in behavior across different sizes suggests that introducing a new data source does not necessarily necessitate a change in attention patterns. Our conclusion is based on the observation that different attention behaviors exist between the two methods. This indicates an alternative approach to recovering performance and output, one that does not rely solely on restoring self-attention patterns. 

As a final point, we use Figure 7 to illustrate that reducing the number of tokens can still effectively recover average performance compared to the baseline. This provides further evidence to suggest the possibility of recovering output without necessarily restoring self-attention and that this phenomenon is not strongly related to the number of parameters or data sources.

Reviewer comment (Evaluation):
Are there any domains or tasks where IDP might not be as effective, and if so, what are the challenges faced in those scenarios?

Author response:
**Second** In addressing whether there are domains where IDP is insufficient, Tables 1 and 2 demonstrate that IDP underperforms in tasks classified under Common Reasoning. These tasks demand the ability to solve problems by comprehending provided information, as opposed to merely recalling information or identifying likely words. The primary challenge here is that the limited number of parameters in the prompts is sufficient only to rechannel attention, thereby recovering information hidden by the compression process. However, these parameters are insufficient to supplement new knowledge that would effectively utilize these facts. This limitation suggests that while IDP can aid in revealing underlying information, its capacity to enhance reasoning or comprehension in more complex task scenarios is constrained.

Reviewer comment (Experiments):
Given the effectiveness of IDP in re-channeling latent knowledge within the model, how does it fare in terms of transfer learning across different domains?

Author response:
**Third** For the final question, given that IDP essentially measures the proxy between input and prompt sequences, it is highly applicable that prompts adapt to different tasks. It can be used together in a mix-domain input for increased efficiency. This is something we are actively exploring.

Reviewer comment (Evaluation):
For most of the experimental results, IDP outperforms baselines only marginally, and the authors have not conducted any significant tests, which makes the results unconvincing.

Author response:
Regarding the “marginal” performance improvement as the reviewer put it, we highlight several key efficiency factors that we keep pointing out shall be taken into the context. Firstly, compared to LoRA, particularly in quantized settings, IDP is 2x more efficient. This efficiency is due to IDP's elimination of the need for an additional residual path across linear layers. Concerning parameter size, IDP is at worst 14x smaller than LoRA and 21x smaller than Ptune. Moreover, IDP demonstrates enhanced efficiency. For instance, as shown in Figure 7, our IDP method outperforms a 100-token single prompt, which has an average accuracy of 58.42, using what is effectively a single prompt of 38 tokens. This represents a 2.6x improvement in computational efficiency, emphasizing the **very significant gains** IDP offers on efficiency dimension while being better or comparable on task performance dimension.

In summary,  as we indicated in the last reply, the reviewer was confused on the impression that our performance gain is “trivial” over other methods but seems to overlook the right context that **such performance gain is achieved with 14 times less parameter size and 2 times less latency** (compared to the standard LoRA or (q)LoRA). **This together shall be treated as the highly nontrivial accuracy-efficiency gain of our method**. In order to dive into how much the gain comes up, we dive into the above analysis on several potential factors (e.g., choosing which metric, how long prompt), confirming that a mixture of short prompts is the best option compared to one much longer prompt, and further that such mixture can be cleverly implemented with no overhead at all - that makes the logic line of our paper.

We hope our explanation now has completely clarified any confusion and we’re thankful for your very prompt response!