Paper: SuperPos-Prompt: Enhancing Soft Prompt Tuning of Language Models with Superposition of Multi Token Embeddings (ICLR-2024)

Reviewer comment (Evaluation):
the paper presents experimental results from the innovations without sufficient justification and explanation, e.g., what are the key explainable innovations that contribute to the performance improvement over similar techniques IPT, ATTEMPT, and Residual Prompt Tuning.

Author response:
Regarding the weaknesses highlighted:

Clarity on Innovations: In addition to eliminating dropout during training, another key innovation of our method is the use of multiple token embeddings for each prompt token, different from the standard practice in the field that typically employs a single token embedding per prompt token. This approach enables a more complex and nuanced initialization vital for effective learning and better adaptation across a wide range of tasks. While our method may not be complex, our experiments demonstrate its consistent improvement over benchmark datasets, with minimal overall variance compared to other soft-prompt tuning approaches. It also shows competitiveness with the full fine-tuning approach, often considered the upper bound.

Reviewer comment (Experiments):
Why does the author only choose T5 as the model for evaluation? Especial when the authors already noticed that T5 has checkpoints that pretrained on GLUE and SuperGLUE datasets. Therefore, I doubt the contribution of proposed method if the method does not lead to a progress on the SOTA results compared to either pretraining or other fine-tuning approaches.

Author response:
In response to the question regarding our choice of the T5 model:
We selected the T5 model due to its widespread recognition and frequent use in prompt tuning research, as evidenced by its application in foundational studies like the original prompt tuning research and the 'ATTEMPT' paper. Our focus on T5 aligns with prevalent practices in the field and allows for a focused, in-depth analysis within our study's scope. It's important to note that the objective of parameter-efficient fine-tuning, as demonstrated by our method, is not necessarily to surpass state-of-the-art (SOTA) results achieved by full fine-tuning methods. Instead, our aim is to achieve significant improvements with minimal parameter modifications. Our method demonstrates its effectiveness by showing strong results with only 10 tokens tuned. This is a significant achievement in parameter-efficient tuning, offering performance and consistency on par with full fine-tuning method. While exploring a wider range of models would enrich our research, our current resources limited us to T5. We acknowledge the value of testing our method on other models and will consider this for future work.

Here are some examples of previous recent work on T5:

The Power of Scale for Parameter-Efficient Prompt Tuning (Lester et al., EMNLP 2021)

SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer (Vu et al., ACL 2022)

Residual Prompt Tuning: Improving Prompt Tuning with Residual Reparameterization (Razdaibiedina et al., ACL 2023)

Reviewer comment (Reproducibility):
Unclear presentation. For example, if you finally decided to freeze E_{freeze}, how did you initialize it? Is it just randomly initialized or intialized with some pretrained token embeddings?

Author response:
**1.** Regarding the initialization of $\( E_{\text{freeze}} \)$. In our method, $\( E_{\text{freeze}} \)$ is initialized by selecting $\( m \)$ random unique token embeddings from the language model's embedding layer. We believe the key factor in our method's success and robustness is its ability to utilize more than one token embedding for each prompt embedding. The purpose of $\( \Delta E \)$ in our methodology is to accumulate gradients during training, preserving the norm of the original $\( E \)$ matrix from being affected by weight decay. This reparameterization technique is utilized to maintain the effectiveness of $\( E_{\text{freeze}} \)$ without directly modifying it during the optimization process. We hope this clarification addresses the raised concern.

Reviewer comment (Experiments):
Missing baselines. For example, the authors should at least compare the proposed method with the similar methods they mentioned themselves, such as IPT and ATTEMPT, in a fair setting. “Through our experiments, we noticed that utilizing superposition is more efficient than softmax weighting.” Where is this experiment?

Author response:
**2.** In line with your suggestion, we have included ATTEMPT in our set of baselines, enabling a more comprehensive comparison. We also tested our method using T5-Base and compared our results with their reported number in the appendix (Table A3). Regarding the comparison with IPT, we encountered a methodological limitation. The IPT approach utilizes the BART model, whereas our work is based on the T5 model. To conduct a fair comparison, we would need access to a similar number of pretrained source prompts (around 100) to train an equivalent autoencoder. Unfortunately, our resources did not extend to this number of source prompts, and training an autoencoder with a small amount of data would not provide optimal results. We have explained this limitation in our revised manuscript for transparency. Furthermore, as discussed, we have added the experiment comparing superposition efficiency with softmax weighting in the appendix. This experiment provides empirical evidence supporting our claim about the efficiency of superposition over softmax weighting in our context.

Reviewer comment (Presentation):
The writing requires improvement. For instance, (a) the authors introduce various methods under three categories: 1) Prompt layers reparameterization, 2) Pre-trained prompts as initial states, and 3) Combined approach. Presenting this multitude of works in the introduction might overwhelm and confuse the reader. As an alternative, the authors could cite one representative work for each category and discuss the limitations of these works in the introduction. Additionally, a dedicated ‘Related Works’ section could be included, where the authors provide a more comprehensive overview of the numerous cited works, allowing readers to delve into further details. This structure would enhance the paper’s readability.

Author response:
We appreciate your suggestion regarding the structure of the Introduction and Related Works sections. However, we believe that the inclusion of related work in the introduction is largely a matter of preference. Given this, we prefer to maintain the original structure, as it effectively presents the context and relevance of our work within the field.

Reviewer comment (Experiments):
The authors deliberately choose not to include IPT and ATTEMPT in their comparative analysis, explaining that these methods depend on pretrained source prompts. This is at odds with their primary goal, which is to enhance soft prompt tuning without the need for pre-trained soft prompts. However, the rationale behind excluding these methods from the comparison is not entirely transparent. To bolster the validity of their approach, incorporating a comparison with these baseline methods would lead to more insights on the utility of this method. Utilizing the pretrained soft-prompts has a similar analogy as utilizing the pretrained weights for finetuning on a downstream task. There does not seem to be a limitation in directly utilizing the pretrained soft prompts.

Author response:
**IPT Exclusion Rationale:** We acknowledge the importance of comparing our method with IPT. However, IPT uses BART as its backbone, in contrast to our T5-based approach. A direct comparison would necessitate training an autoencoder on T5 prompts over numerous source tasks, a condition we couldn't replicate due to limited access to an adequate number of source prompts. We have emphasized this methodological limitation in our manuscript.

Reviewer comment (Experiments):
I am not convinced with the result that, Superpos PT without dropout outperforms full finetuning by such a large margin of 16.4% on CB task on T5 small, same goes with T5 base on COPA task. It is not convincing that PEFT method is able to beat full finetuning by such a large margin. As claimed by the authors even LoRA and adapters that perform better than soft prompt based methods, occasionally beat full finetuning by very small margin. Could you please shed some light on it? Also, are the hyperparameters used to tune full model parameters consistent with the original paper.

Author response:
**Question on Performance Margin:** Regarding the significant performance margin in smaller datasets like CB and COPA, we attribute this to the reduced risk of overfitting in PEFT methods compared to full finetuning. Full finetuning on such small datasets often leads to overfitting, while PEFT methods, by modifying a limited subset of parameters, mitigate this risk.

Reviewer comment (Writing):
The authors mention this in the introduction “Soft prompt is renowned for its exceptional parameter efficiency.” However it also mentions “finetuning soft prompts is optimization-intensive, particularly with limited data and smaller model sizes in T5 family between 50 to 300 million parameters (Lester et al., 2021);” As far as I understand the model weights are frozen. Are they contradictory statements made independently or are the authors mentioning that the convergence rate is slow for soft prompt based methods?

Author response:
Question on **Parameter Efficiency and Optimization Intensity:** The statement about parameter efficiency refers to the effectiveness of soft prompts in adapting large models like T5 with fewer parameters. However, optimizing these prompts, especially in limited data scenarios, requires more training steps or epochs, thus resulting in a slower convergence rate, a trade-off that soft prompt tuning is dealing with.

Reviewer comment (Evaluation):
The paper lacks sufficient analysis of SuperPos-Prompt learned soft prompts on what semantic information the prompt tokens indicate and how it contributes the results, at least some case study is needed.

Author response:
Regarding the analysis of SuperPos-Prompt from a semantic perspective, we agree that this is an important aspect. In the current revision, we've included a preliminary visual analysis for task similarities. However, a more comprehensive semantic dissection requires sophisticated methodologies which we aim to explore in our future work.  In the revised paper, we introduced a visual analysis of task similarities using SuperPos-Prompt weights in Figure 2.d, providing a clearer understanding of how taks prompts are related to each other.

Reviewer comment (Experiments):
Have you considered more recent LLMs and includes more baselines like few-shot results on ChatGPT/GPT4 for reference?

Author response:
In response to the comment about including OpenAI LLMs, we have added an analysis of GPT-3.5's one-shot performance in the appendix. GPT-3.5 achieved an average score of 67.1, while our approach achieved an average score of 81.2 (Table A3).