Context-aware Prompt Tuning: Advancing In-Context Learning with Adversarial Methods

Authors: Tsachi Blau, Moshe Kimhi, Yonatan Belinkov, Alexander Bronstein, Chaim Baskin
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: Large Language Models, In-Context Learning, and Adversarial Attacks

Abstract:
Large Language Models (LLMs) can perform few-shot learning using either optimization-based approaches or In-Context Learning (ICL). Optimization-based methods often suffer from overfitting, as they require updating a large number of parameters with limited data. In contrast, ICL avoids overfitting but typically underperforms compared to optimization-based methods and is highly sensitive to the selection, order, and format of demonstration examples. To overcome these challenges, we introduce Context-aware Prompt Tuning (CPT), a method inspired by ICL, Prompt Tuning (PT), and adversarial attacks. CPT builds on the ICL strategy of concatenating examples before the input, extending it by incorporating PT-like learning to refine the context embedding through iterative optimization, extracting deeper insights from the training examples. Our approach carefully modifies specific context tokens, considering the unique structure of the examples within the context.In addition to updating the context with PT-like optimization, CPT draws inspiration from adversarial attacks, adjusting the input based on the labels present in the context while preserving the inherent value of the user-provided data.  To ensure robustness and stability during optimization, we employ a projected gradient descent algorithm, constraining token embeddings to remain close to their original values and safeguarding the quality of the context. Our method has demonstrated superior accuracy across multiple classification tasks using various LLM models, outperforming existing baselines and effectively addressing the overfitting challenge in few-shot learning.

Decision: reject
Reviewer scores: 3, 5, 6, 5

Reviews and discussion:
<thread>
creation date:
1730496468546

thread_type:
official_review

<thread_object>
creator: reviewer_iktw

<content>
summary:
This paper proposes context-aware prompt tuning (CPT), a few-shot learning method that optimizes the in-context example using projected gradient descent (PGD). The authors conduct experiments on various models and benchmarks, finding CPT outperforms other baseline methods (prompt tuning, LoRA, instruction prompt tuning).

strengths:
The idea of optimizing the context using PGD is novel. If one needs to optimize the in-context example, it makes sense to use PGD so that the examples will not change by too much and the optimization does not overfit the few-shot examples.

weaknesses:
1. It is unclear what is the motivation for optimizing the in-context example. Does that make more sense than optimizing the prompt in instruction prompt tuning (IPT)? There is one possibility that the performance should be attributed to the use of PGD, which might prevent from overfitting. It will be beneficial if there is abation study on that, e.g., uses other regularized optimizers or use PGD in baselines (IPT, PT).

2. Lack of analysis on the experimental results. Again, it might make the results easier to analysize if the authors disentangle these two components: (i) train on in-context samples; (ii) PGD as the optimizer. 

3. Some paragraphs are not clearly written. For example, in section 4.3, the authors introduce another variant $\dagger$ which initializes trainable tokens with human engineered instructions. It is unclear what is the difference between $PT^\dagger$ and $IPT^\dagger$ for few-shot setting since they both have few-shot examples and trainable instruction-initialized tokens. 

4. The Set Classification dataset is confusing. I feel the only mechanism that works for an input sequence from that dataset is to search in the context to see if the test example appears and copy the cooresponding label (if any) as the output (please correct me if I am wrong), which is simply an induction head,  and the loss design of CPT seems to be encouraging the forming of induction head. Therefore is it a good benchmark to measure few-shot learning?

5. In section 5.1 the title "Better with Harder Tasks" for that paragraph seems to be overclaiming.

questions:
Please see weakness.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730644983876

thread_type:
official_review

<thread_object>
creator: reviewer_desr

<content>
summary:
This paper proposes a few-shot method called context-aware prompt tuning. This method is inspired by in-context learning and prompt tuning, concatenating examples before the input as in ICL and learning these examples as in prompt tuning. The authors verify the effectiveness of the proposed context-aware prompt tuning method on several classification tasks using three models.

strengths:
The proposed method is straightforward and easy to understand.
The experiments in this paper are comprehensive, which use three models and test on several classification datasets.
The results show that the proposed method outperforms LoRA, PT, IPT and ICL in most cases.

weaknesses:
1. The proposed method integrates ICL and PT intuitively, thus the novelty is not very significant.
2. Since the proposed context-aware prompt tuning is an optimization-based method, it also has the overfitting problem, just like fine-tuning and PT. Thought authors tried to mitigate overfitting by incorporating context labels into the loss function and applying projected gradient descent, the overfitting problem still exists in context-aware prompt tuning.

questions:
1. Why ICL cannot fully extract the information that exists in the training examples?
2. In Table 1, CPT† (incorporating instructions) outperforms CPT on some datasets/models (for example SST-2 with BLOOM 1.7B), but underperforms CPT on other settings (for example DBpedia with BLOOM 1.7B, decreasing from 58.85 to 33.80 using 2 shots). How to explain this phenomenon?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730681507477

thread_type:
official_review

<thread_object>
creator: reviewer_cols

<content>
summary:
This paper introduces an approach to mitigating overfitting in few-shot learning scenarios, where traditional fine-tuning often leads to overfitting, and in-context learning (ICL) performance is highly sensitive to the choice of demonstration examples. The authors propose a context-aware prompt tuning strategy that integrates elements of ICL and prompt tuning. In this method, the demonstration examples provided to the model are treated as tunable parameters, while the associated labels remain fixed. By tuning the context of these demonstrations rather than the entire model, the approach leads to improved generalization across multiple benchmarks.

strengths:
- Proposes a simple extension of prompt tuning by combining it with in-context learning. 
- Evaluates the model on different open-source datasets like AGNews, SST-2, DBpedia and TREC.
- The figures (fig 2) in the paper are really helpful in understanding key differences between their method and various baselines they implemented.

weaknesses:
- The idea introduced is very similar to prompt tuning and the whole paper is just about applying it in-conjunction with in-context learning making it harder to find novelty in the approach. 
- The writing of the paper can be improved by quite a bit eg: abstract of this paper is hard to understand talking about different approaches rather than making it more crux on the solution/method and same applies to description provided on the set classification dataset.

questions:
- Was any hyperparameter tuning or prompt optimization conducted for the baseline models? The reported performance of Llama-3 seems surprisingly low on straightforward tasks like SST-2, where zero-shot models typically perform well. This raises questions about whether the baseline results fully reflect the model's potential or if they were hindered by suboptimal configurations. Clarification on any tuning efforts for the baselines would be helpful to ensure a fair comparison with the proposed method.
</content>
</thread_object>

</thread>
<thread>
creation date:
1731041772301

thread_type:
official_review

<thread_object>
creator: reviewer_29ww

<content>
summary:
This paper proposes Context-aware Prompt Tuning (CPT) as a new method to address the overfitting problem of prompt tuning. The optimization of CPT consists of a novel loss design and controlled token embedding optimization. Extensive experiments on various datasets using different LLMs demonstrate that CPT achieves better performance than baselines.

strengths:
1. The research problem is interesting, and the authors proposed a new method for better LLM performance on several benchmark classification tasks. 
2. The authors have provided detailed literature studies and discussed the motivation of their method, 
3. The authors have done extensive experiments to demonstrate the effectiveness of CPT.

weaknesses:
1. The paper is not clearly written and well-organized. It is hard to understand the authors' ideas and the proposed methods. For example, the motivation for presenting Figure 1 and Figure 2 before the introduction is not clear. Although the authors show some input presentation in Sec.3.1, it is still unclear what the inputs look like. The authors should have provided detailed examples in this section. 
2. The novelty if this paper is limited, it is a simple combination of several existing works, such as prompt tuning and adversartial training. Besides the empirical results, the paper also lacks theoretical analysis. 
3. The figures, such as Figure 2 and Figure 4, are not clear and hard to understand. For example, why are there two x_emb2 in Figure 4. What do different colors mean in these figures? 
4.  Some concepts need further clarification. For example, how to do projected gradient descent as presented in Lines 272-273. 
5. The authors only work on simple classification tasks and lack experiments on more general generation tasks, such as summarization and machine translation.

questions:
1. Does the final loss have any tuning parameters, as presented in Line 252?
2. See weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732089974390

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Discussion Period

comment:
We have provided detailed comments addressing each of the reviewers' points, and to fully benefit from the discussion period, we would kindly like to remind the reviewers that we are awaiting their responses during this time. Your feedback is invaluable in ensuring the clarity and quality of the final version, and we greatly appreciate your continued engagement in this process.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732103978469

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
All Reviewers [1/2]

comment:
We would like to extend our deepest gratitude to all the reviewers for their time and constructive feedback. Your insights have been included in our paper, and we truly appreciate your contributions. Throughout this response, we will refer to reviewers 29Ww, CoLs, DESR, and iktw as R1, R2, R3, and R4, respectively.


You described this as an “interesting research problem” (R1) and acknowledged that we have “proposed a new method for better LLM performance on several benchmark classification tasks” (R1). Additionally, you noted that we propose an “extension of prompt tuning by combining it with in-context learning” (R2) and highlighted that our “method is straightforward and easy to understand” (R3).
The clarity of our presentation was further commended, as you mentioned that “the figures (Fig. 2) in the paper are really helpful in understanding key differences between their method and various baselines they implemented” (R2). You also pointed out that we “provided detailed literature studies and discussed the motivation of their method” (R1), particularly in revealing and addressing the overfitting problem.


Moreover, you recognized the novelty of our approach, stating that “the idea of optimizing the context using PGD is novel” (R4) and that “it makes sense to use PGD so that the examples will not change by too much and the optimization does not overfit” (R4).


Finally, you acknowledged the robustness of our evaluation, emphasizing that “the authors have done extensive experiments to demonstrate the effectiveness of CPT” (R1). You described the experiments as “comprehensive, using three models and testing on several classification datasets” (R3) and noted that our work evaluates “the model on different open-source datasets like AGNews, SST-2, DBpedia, and TREC” (R2). Importantly, you highlighted that “the proposed method outperforms LoRA, PT, IPT, and ICL in most cases” (R3).


As for the points that required further elaboration, we have responded to each of you individually, addressing all of your concerns thoroughly. Additionally, we would like to highlight your remarks that necessitated modifications to the paper or were shared among some of you. For your convenience, all changes made to the paper have been marked in red in the revised version.





(R1) “Fig. 2 and Fig. 4 are not clear” - We revised Fig. 2 and added additional explanations for Fig. 4 to improve clarity (in App. G). 


(R1, R4) “it is still unclear what the inputs look like” and “It is unclear what is the difference between PT \dag and IPT \dag.” - This was addressed in App. G.


(R1) “Some concepts need further clarification” - We included a detailed explanation of how PGD works to provide further clarity (in App. H). 


(R2,R3) “abstract of this paper is hard to understand” and "Why ICL cannot fully extract the information that exists in the training examples" - We updated the abstract.

(R2, R4) "description provided on the set classification" and "Set Classification dataset is confusing" - As requested by R4, we have removed the Set Classification dataset from the paper. 



(R4) “There is one possibility that the performance should be attributed to the use of PGD” - We added additional experiments (in App. I), as requested, to confirm that the improvements are not solely due to PGD. Additionally, we emphasized that the ablation of the original version (Tab. 2) already demonstrates the critical role of the novel loss function in our method.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper introduces Context-aware Prompt Tuning (CPT), a method combining In-Context Learning (ICL), Prompt Tuning (PT), and adversarial optimization to address overfitting in few-shot learning tasks. The approach is validated on multiple classification datasets and different LLMs, demonstrating improved performance over baselines. However, concerns about the novelty, dataset relevance, and generalization to broader tasks were widely raised among the reviewers, and the authors, though actively engaging in the discussion, do not fully address the reviewers' concerns. 

I think this work has its methodological merits and practical significance, but it needs a revision to better illustrate these points to the readers. So given that most reviewers find these problems are not fully unaddressed in the current form, I recommend rejection but recommend the authors to submit it to a future venue taking these comments into consideration.