Paper: Meta-Knowledge Extraction: Uncertainty-Aware Prompted Meta-Learning (ICLR-2024)

Reviewer comment (Experiments):
While the motivation for introducing the Bayesian meta-prompt is clear and reasonable, the current experimental results cannot fully support it due to the lack of a large meta-feature extractor network (backbone). The main goal of this work is to leverage prompt tuning to avoid inefficient nested gradient calculation on large models; yet the backbones used in the experiment are too small to validate the effectiveness of the proposed method. *Table 2* also exacerbates the above concern, where all the methods actually report a comparable training/adaption time.

Author response:
That is quite a good question. We would like to clarify that the reason why we do not post the result on the large-scale (e.g., vit) is that the inefficiency issue of the traditional methods (e.g., MAML) prevents their application on the large-scale architecture. Instead, we have conducted the experiments on vit-based-patch16-224 with the efficient meta-learning methods (i.e., ours and ANIL), and found that our proposed two methods achieve an increase of 3%/3.6% (52.07%/52.67%  v.s. 49.01%) with a similar computational consumption. As for the result of table2, the decreased gap (compared to table1) of the consumption between ours and the baselines is due to the decreased scale of the few-shot tasks, from 5-shot to 1-shot. We also conducted the 5-way 10-shot experiment and the experimental results demonstrate that our method achieves an increase of 1.1% and 1.6% while only requiring 40% and 23% computation consumption for training when compared to MAML. Another experiment under 20-way 1-shot also confirms the efficiency of our methods, where the result demonstrates that our method requires 62% consumption of MAML for training but with an increase of 4% on performance.

Reviewer comment (Theory):
The prompt poster is inconsistent through Eqs (1), (4), and (6). It remains confusing 1) where the meta prompt is sampled from and 2) what parameters are used in optimizing the prompt posterior. Is $p(s)$ in (1) the prompt prior or the posterior, or should it be written as $p(s|D)$? In meta-training, should the method sample prompt from the prior or the learned posterior? Comparing (4) and (6), is $q(s)$ optimized through $\phi$ or $\theta$?

Author response:
We thank again for the reviewer’s detailed review, which really help to improve the quality of our paper. We would like to clarify that in our assumptions, we consider the distribution of s as a Gaussian distribution, as explicitly stated in A.1 of the original submission. Accordingly, the $q(s)$ is parameterised by $\phi = \{ \mu, \sigma \}$. To prevent any potential misunderstanding, we relocated the details pertaining to $q(s)$ from the appendix to the main text in the revised version.

Reviewer comment (Experiments):
It lacks strong empirical evidence to indicate the effectiveness of the proposed Bayesian meta-prompt. From Table 3, the probabilistic formulation of the meta-prompt did not show a significant improvement. How about directly learning $s$ through a MAML approach?

Author response:
The results obtained by directly learning the meta-prompts through a MAML approach are denoted as "w/o Bayes" in Table 3. Our experimental findings indicate that the non-Bayesian method experiences a decrease of nearly 1.5% in average performance across the four datasets. It is essential to note that the reported performance represents the average result across these datasets. For a more detailed breakdown of results on each dataset, please refer to Table. 8 and 9, which demonstrate that the Bayesian treatment consistently improves performance across all four datasets. Furthermore, the advantage of the Bayesian treatment extends beyond the experimental results. It encompasses the capability to handle noise and uncertainty when applying the method in real-world scenarios. This underscores the broader applicability and robustness of our Bayesian meta-prompt in addressing challenges that may arise in practical applications.

Reviewer comment (Novelty):
The lit review is insufficient. Some relevant works, e.g., Probabilistic Model-Agnostic Meta-Learning, should be clearly discussed in the paper.

Author response:
The paper [3] mentioned by the reviewer focuses on the problem of Bayesian meta-learning, which casts the task-specific parameters as the deterministic value and only operates the Bayesian treatment on the meta-parameters. We follow a similar idea but only use the Bayesian treatment on our proposed meta-prompt instead of all the meta-parameters. We will add this paper to the discussion about the Bayesian meta-learning in our paper.

Reviewer comment (Experiments):
Can the proposed method be applied to large models? Such as ResNet101, ViT-L/14, etc.

Author response:
It is definite that our method can be used on the large models. As we mentioned in the response to Q1, we have conducted the experiments on VIT-base-patch16-224, and found that our method achieves an improvement of 3%/3.6% (52.07%/52.67%  v.s. 49.01%) with a similar computational consumption, when compared to ANIL. Unfortunately, according to the inefficiency issue, it is difficult to train a vit-based model with MAML in an acceptable time.

Reviewer comment (Experiments):
How does the proposed method choose the prompt prior? Is the proposed method sensitive to prior choice?

Author response:
To answer this question, we conduct the experiments with different priors, i.e., the mixture Gaussian distribution used in our paper, and the standard normal distribution (the mean is 0 and the standard deviation is 1). The experimental results show that the meta-prompt with a standard normal distribution achieves a comparable performance under 5-shot setting while seeing a decrease of 0.7%-1.8% under 1-shot, compared to the meta-prompt with a mixture prior(our reported method in the original submission). It is reasonable that the mixture prior helps to focus more on the long-tail data, especially under 1-shot setting where it is easily affected by some outliers, according to the existing work [4].

Reviewer comment (Evaluation):
Can the paper show some visualizations of how the prompt uncertainty changes across different tasks?

Author response:
We guess the reviewer would like to see the task-specific prompts across different tasks. We have visualised the task-specific prompts across different 5-way 5-shot tasks in Figure 6 of the appendix.

Reviewer comment (Novelty):
The technical novelty might not be enough. Although the paper claims that the idea is inspired from prompt tuning of LLM, it is still to meta-learn part of model parameters (the majority of model parameters are freezed), so that the training becomes efficient. Similar papers are:  
[1] Rapid learning or feature reuse? towards understanding the effectiveness of maml, ICLR 2019  
[2] Boil: Towards representation change for few-shot learning, ICLR 2020

Author response:
In addition to the meta-knowledge extraction paradigm mentioned above, we also provide two other contributions. Firstly, we theoretically analyse the role of the learned uncertainty of our learnable Bayesian meta-prompt. This exploration contributes a foundational understanding of the mechanisms for constructing the task-specific prompts in our methodology. Secondly, we propose two modulation methods to construct the task-specific prompt from the meta-prompt. These methods utilise the learned posterior distribution of the meta-prompt to modulate the knowledge transfer among tasks. The aim of our work is to address the efficiency challenges that have impeded the application of meta-learning in real-world scenarios. By combining these contributions, we aspire to provide valuable insights to enhance the applicability of meta-learning in practical settings.

Reviewer comment (Writing):
Minor issues: one ")" is missed in Eq.10.

Author response:
Moreover, the mentioned minor typos are revised in the updated version.

Reviewer comment (Novelty):
I am still not convinced by the connection between prompt tuning and the idea in the paper. The idea of this paper is more close and similar to ANIL and BOIL [1,2], and I did not find a similarity with prompt tuning, technically speaking. Since the authors use many spaces to explain prompt tuning, I assume I missed their connection. If the authors could explain this connection and key difference from ANIL and BOIL in the response, I am open to raising the rating.  
[1] Rapid learning or feature reuse? towards understanding the effectiveness of maml, ICLR 2019  
[2] Boil: Towards representation change for few-shot learning, ICLR 2020

Author response:
Prompt tuning has recently emerged as a parameter-efficient fine-tuning technique within the pre-training followed by fine-tuning paradigm. Its fundamental principle involves knowledge extraction from a pre-trained model, as opposed to the conventional approach of adapting this knowledge for downstream tasks through full fine-tuning. On the other hand, meta-learning entails acquiring meta-knowledge from existing training tasks, thereby facilitating quick adaptation to few-shot tasks. The prevailing gradient-based meta-learning method, designed to acquire an optimal initialisation, formulates meta-knowledge as neural network weights, akin to the pretraining-finetuning paradigm. Building upon the insights from prompt tuning, our research endeavors to employ task-specific prompts for efficient guidance of a fixed backbone (shared meta-knowledge) in handling few-shot tasks. Notably, it stands in contrast to the resource-intensive paradigm of adapting meta-knowledge that is akin to the fine-tuning process applied to the entire model.

One of our contributions is to change the costly meta-knowledge adaption paradigm in traditional meta-learning into efficient meta-knowledge extraction. As we claimed in the introduction, our work employs the fixed backbone, which is supported by the previous ANIL that confirms the effect of the shared features (feature reuse in the original paper). However, ANIL faces limitations, particularly in its inability to address the complexities of certain few-shot tasks, as highlighted by BOIL. For example, the shared feature extractor cannot deal with heterogeneous scenarios where different tasks might require different features. Instead, BOIL proposes adapting the feature extractor for specific tasks with a fixed classification head, yet it incurs a computational cost similar to that of MAML, as evidenced in our experiments. Differently, our work seeks to employ a small number of parameters (task-specific prompts in our work) to prompt the fixed feature to deal with different few-shot tasks in a more efficient way. **The experimental results confirm the superior performance of our method compared to BIOL, while requiring a similar consumption as ANIL.**

Reviewer comment (Theory):
The fact that the two update rules take different inputs from the posteriors of the meta-prompts (the hard modulation takes the means and the standard deviations, but the soft modulation takes only the standard deviations) makes both update rules look more ad hoc.

Author response:
We would like to clarify that the difference in using the mean or not for constructing task-specific prompts depends on the modulation mechanism. Specifically, the soft way employs the input directly to modulate the learning rate, while the hard way utilizes the input to decide whether to update a certain dimension or not. Consequently, the soft way is more sensitive to the value of the input. easily influenced by some outliers. The unlimited scale of the mean value limits its application on the direct modulation in the soft way, although the mean value indicates the role of the dimension where the larger means more important or effective the dimension is to the prediction given a specific task (claimed in our original submission according to [1]).

We also conducted the experiment (see the following table) and found that the input of the mean leads to a decrease of 0.5%-2% (especially for the 1-shot setting), which also confirms the negative use of the mean value in the soft way. We will clarify this point in the revised paper.

|    |Conv4|    |Resnet12||
|-|-|-|-|-|
|    |1-shot|5-shot|1-shot|5-shot|
|Reference| 56.84$\pm$0.73| 71.96$\pm$0.59 | 58.52$\pm$0.77| 73.77$\pm$0.57|
|Soft Modulation with Mean|56.35$\pm$0.73 | 71.26$\pm$0.81 | 56.70$\pm$0.77| 73.07$\pm$0.81|

Reviewer comment (Writing):
The manuscript could be improved in presentation due to missing details and typos in the main text.

Author response:
Thank you again for pointing out the typos in our original version. We moved the definition of the variational distribution into the main text. In addition, the incorrect references in RQ2, the missing reference for Fig.1 and the incorrect definition in eq.5 and 6 mentioned in the minor comments are fixed in the updated version.

Reviewer comment (Presentation):
The exact definition of the Gaussian distributions for the meta-prompts is not given in the main text, including the definition of the standard deviation of the d-th dimension of the meta-prompt, \(\sigma_d\). Although some descriptions are present in the appendix, they should be included in the main text for easier reading.

Author response:
Thank you again for pointing out the typos in our original version. We moved the definition of the variational distribution into the main text. In addition, the incorrect references in RQ2, the missing reference for Fig.1 and the incorrect definition in eq.5 and 6 mentioned in the minor comments are fixed in the updated version.

Reviewer comment (Writing):
In Section 5.2 (RQ2), the figure and table are referenced incorrectly (Figure 1 and Table 1?). Additionally, the exact definition of the (weighted) KL-term considered in this section is unclear (could it be eq. (4)?). The exact meaning of the removal of the Bayesian treatment is also unclear.

Author response:
The KL-term explicitly corresponds to the latter terms in Equation (4), and we have incorporated the relevant reference in the updated version. The meaning of the removal of the Bayesian treatment means that we model the meta-prompt in a deterministic way instead of a probabilistic way. In other words, we treat the meta-prompt as a deterministic vector instead of a probabilistic distribution. This Bayesian treatment enables to provide a flexible initialisation for constructing task-specific prompts.

Reviewer comment (Theory):
The symbol \(\mathcal{L}\) was introduced as a loss function but was also used as the lower bound (which should be maximized) in eq. (6). There is also a need to check eq. (5) regarding the relationship between the loss function and the likelihood.

Author response:
Thank you again for pointing out the typos in our original version. We moved the definition of the variational distribution into the main text. In addition, the incorrect references in RQ2, the missing reference for Fig.1 and the incorrect definition in eq.5 and 6 mentioned in the minor comments are fixed in the updated version.

Reviewer comment (Evaluation):
The manuscript states that one of the main motivations (of using the prompt learning) is to improve the computational inefficiency of the meta-knowledge adaptation. However, in the experimental result section, the improvements in the computation time do not look significant compared to the baseline (e.g., MAML) for the choice of the backbone network (e.g., Conv4). This could lead to the misunderstanding that the improvements reported in the experimental results are a matter of implementation (e.g., code optimization). The datasets may not be large enough to contrast the improvements in the computation time of the methods.

Author response:
The efficiency of our methods mainly results from the fixed backbone when dealing with specific tasks, as analysed in the appendix A.3 of the original submission. Since Conv-4 is just a simple backbone, the computational consumption incurred by the backbone remains relatively modest but our method still achieves a 58% decrease compared to MAML (refer to Table 1). In the case of more complex backbones, such as Res12, our approach demands only 24% of the computational resources required by MAML. Moreover, we have also conducted the experiments on a larger backbone (i.e., vit_base_patch16_224), and our proposed two methods achieve an increase of 3%/3.6% (52.07%/52.67%  v.s. 49.01%) on the performance with a similar computational consumption, when compared to ANIL, an efficient baseline. As for MAML, the meta-knowledge adaption, we found it difficult to train such a large architecture in an acceptable time.

Reviewer comment (Evaluation):
In Section 5.2 (RQ2), it is unclear what the authors intended to show with the performance changes in the dimension of the meta prompts. The text does not clearly explain this, nor how to understand this pattern in terms of Bayesian learning. In the current version of the manuscript, the figures just convey that the dimension of the meta prompts is also a hyperparameter to be tuned by the users.

Author response:
In the RQ2, our objective is to investigate the impact of meta-prompts on performance. One notable attribute of the meta-prompt under consideration is its number of dimensions, a factor intricately linked to computational consumption, particularly in terms of the Bayesian treatment. Our experimental findings consistently reveal an anti-intuitive trend: larger prompts do not necessarily yield superior performance. Such an anti-intuitive conclusion offers an insight that finding an appropriate number of dimensions is more important than just simply adding the dimension.