Few-shot Unified Question Answering: Tuning Models or Prompts? (2023)

Abstract: Question-answering (QA) tasks often investigate specific question types, knowledge domains, or reasoning skills, leading to specialized models catering to specific categories of QA tasks. While recent research has explored the idea of unified QA models, such models are usually explored for high-resource scenarios and require re-training to extend their capabilities. To overcome these drawbacks, the paper explores the potential of two paradigms of tuning, model, and prompts, for unified QA under a low-resource setting. The paper provides an exhaustive analysis of their applicability using 16 QA datasets, revealing that prompt tuning can perform as well as model tuning in a few-shot setting with a good initialization. The study also shows that parameter-sharing results in superior few-shot performance, simple knowledge transfer techniques for prompt initialization can be effective, and prompt tuning achieves a significant performance boost from pre-training in a low-resource regime. The research offers insights into the advantages and limitations of prompt tuning for unified QA in a few-shot setting, contributing to the development of effective and efficient systems in low-resource scenarios.

## introduction question answering (qa) is a pivotal area of research in nlp that evaluates the language understanding and reasoning capabilities of language models. to this end, the nlp community has developed numerous qa datasets that span various domains, question-answer formats, and reasoning skills @xcite . consequently, there is an increasing demand for a unified qa system that can manage mixed batches of instances from different datasets and tasks during training and inference @xcite . such a system would eliminate the need for manual tuning or per-task adjustments, * work done during internship at salesforce research figure 1 : comparison of multi-task model-tuning (ft-mt), prompt-tuning with a format based (pt-f) and a format agnostic (pt-c), and attempt (att-mt) @xcite (a complex prompt transfer learning approach) for unified qa on target qa datasets in a 32-shot scenario using t5-base as the backbone model. the results show that prompt-tuning with prior outperforms multi-task full-model fine-tuning and attempt does not provide any additional advantage, especially for out-of-domain target tasks. here iid has been used to refer to in-distribution while ood refers to out-ofdistribution. the success of transformer-based models in textto-text generation has led to a growing interest in unified qa systems. @xcite proposed unified-qa, a single qa model pretrained on diverse datasets that outperforms formatspecialized models. while prompt-tuning methods @xcite have emerged as a promising alternative to fine-tuning, @xcite proposed to model the commonal-ities and distinguish task differences through a structurally designed prompt-based input schema. however, these approaches have limitations related to scalability, expensive pre-training requirements, and the need for tens of thousands of training examples for each task. moreover, the performance of pre-trained qa models significantly degrades when only a few question-answering examples are available @xcite . while unified qa approaches have shown success in high-data scenarios, their efficacy in more practical scenarios with limited training examples remains unexplored. this paper aims to explore the potential of two different paradigms of tuning, model, and prompts, for unified question answering under a low resource setting. despite the importance of this problem, there have been no previous studies investigating the effectiveness of these paradigms for this task. in response, we conduct an exhaustive analysis of the applicability of these two paradigms to a unified question-answering system. to do so, we evaluate their promise, effectiveness, and trade-offs using a set of 16 qa datasets, covering diverse domains and a wide range of skills and formats. our empirical study reveals several key findings, including (i) prompt tuning with good initialization can outperform model tuning under a low resource regime for out-of-distribution tasks, (ii) parameter-sharing results in superior few-shot performance, but the trends are reversed in the fullshot setting, (iii) simple knowledge transfer techniques for prompt initialization can be as effective as more complex methods in the few-shot setting, without introducing additional parameters, and (iv) prompt tuning achieves a significant performance boost from pre-training in a low resource regime while increasing model size does not significantly affect prompt tuning with initialization. in addition, we perform a systematic quantitative and qualitative study to provide insights into the advantages and limitations of prompt tuning for unified qa with an emphasis on the behaviors in the few-shot setting. overall, our research aims to contribute to the development of effective and efficient unified question-answering systems in low-resource scenarios. multi-task transfer learning. efficient task transferability in nlp has been extensively studied @xcite @xcite . with t5 @xcite demonstrating the capabilities of using existing downstream task datasets to learn a new task, proposing efficient methodologies for unifying nlp models has become a promising research paradigm in the community. following this, @xcite proposed unifiedqa, a single qa model pre-trained on datasets involving diverse formats and reasoning skills. transfer learning has been demonstrated to be effective from rich data sources @xcite , between similar target tasks @xcite , and for tasks that require similar reasoning skills @xcite . however, this approach would require updating/retraining the model on a new task or a different domain, which could lead to catastrophic forgetting @xcite . moreover, @xcite showed approaches towards unifying nlp models suffer from negative interference to less represented tasks and between dissimilar tasks. most recently, @xcite validates that parameter-efficient tuning methods can perform well with mixed task batches. @xcite takes the first step towards building unified qa models utilizing structural prompt tuning. along these lines, @xcite ; @xcite integrates both the paradigms of parameter-efficient tuning and unifying nlp models to propose a single pre-trained model for different downstream tasks by learning target task-specific prompts from the source task prompts. @xcite demonstrates transfer using the attention module, while @xcite facilitates prompt transfer by learning the target prompt initialized from similar source prompts. these approaches require fewer than 0.1% of trainable lm parameters with little tradeoff in performance. few-shot question answering. @xcite has identified a discrepancy between current pretraining objectives and qa, as standard models perform poorly when fine-tuned with few examples. they propose recurring span selection as a pretraining scheme tailored for question answering. @xcite , on the other hand, proposes a fine-tuning framework aligned with the pretraining framework. however, there have been no studies focusing on the viability of prompt tuning for unified qa under low-resource settings. to address this gap, we follow prior works, @xcite @xcite , and extensively study the viability and trade-offs of prompt tuning and prompt-based transfer learning in comparison to approaches that involve full-model fine-tuning for few-shot unified qa. as a result of our comprehensive experiments, we offer essential guidelines in the form of valuable insights into the advantages and limitations of prompt tuning with respect to model tuning for unified qa in both full and fewshot scenarios. approach paradigm sm kt nt ex ft model-tuning limited high ft-mt model-tuning ✓ high limited pt-r prompt-tuning ✓ limited limited ✓ pt-f/pt-c prompt-tuning ✓ high limited ✓ att-mt prompt-tuning ✓ high high ✓ model-tuning this paradigm involves the finetuning of all the parameters of a language model to cater to a specific task or a set of tasks. although fine-tuning (ft) on a particular dataset is an effective strategy, it is not suitable for unified qa because it requires specialized models for each dataset during inference, which is counter-intuitive to the concept of a unified qa model. in contrast, multi-task learning via fine-tuning (ft-mt) @xcite involves the joint learning of a single model on multiple datasets by sharing all the trainable model parameters across different tasks. by training on multiple datasets, ft-mt allows for knowledge transfer from relevant tasks during inference. however, sharing all the parameters often leads to negative transfer from unrelated tasks. incorporating additional tasks into existing models requires retraining the model with all previous tasks and the new ones, making them computationally expensive to scale and more prone to negative interference. prompt-tuning this paradigm involves learning soft-prompt tokens added to the input while the backbone language model remains frozen. we follow the approach proposed by @xcite to train soft prompts for each task, where prompts are initialized from random words in the vocabulary (pt-r). this vanilla prompt-tuning approach is parameter-efficient and easy to scale. since taskspecific knowledge is captured in a different set of parameters (i.e., the prompts), this approach avoids negative interference to a great extent. with a single backbone model, we can use these prompts for different tasks. however, this approach does not leverage knowledge from other tasks not already captured in the backbone model. prompt initialization is a technique that addresses the issue of knowledge transfer from source tasks in vanilla prompt-tuning while retaining the benefits of a single model, minimal negative transfer, and extensibility. previous studies @xcite @xcite have shown that prompt-tuning methods are often sensitive to initialization, particularly in low data settings. however, the impact of different initialization methods on qa datasets has not been well studied. inspired by @xcite , we initialize the target prompt by taking the average of the top-3 source task prompts most similar to the prompt trained on the target dataset. we employ two distinct approaches to this initialization process: (i) selecting source task prompts with the same answer format as that of the target dataset (pt-f), and (ii) selecting source task prompts from the complete set of source prompts (pt-c). apart from prompt initialization, another way to transfer knowledge from multiple tasks is through the composition of their corresponding prompts. to this end, @xcite proposes at-tempt, a transfer learning method that learns new task-specific target prompts by computing weighted combinations of source prompts using a sub-network-based attention module trained on a single or set of tasks. we distinguish between two settings: att-mt, where attention modules are shared across tasks and trained in a multi-task manner, and att-st, where attention module parameters are not shared. while att-mt provides a single model for transferring knowledge from source prompts and is easily scalable to new target tasks, sharing attention modules across tasks may result in some negative transfer, compared to more straightforward prompt-tuning methods. ## datasets in their recent study, @xcite highlight a significant increase in the number of questionanswering and reading comprehension datasets, spanning various domains, formats, and reasoning abilities. this study aims to evaluate and finetune a range of models, leveraging a collection of datasets referred to as "source datasets" for pretraining, and a distinct set of datasets known as "target datasets" for evaluation. this paper includes datasets that cover a wide range of reasoning skills and complex linguistic phenomena, including conversational, temporal, causal, and coreference reasoning, among others, enabling a more comprehensive evaluation of training paradigms on question-answering datasets and facilitating analysis of cross-skill transfer. this broader coverage across reasoning skills not only enables a more thorough evaluation of training paradigms on qa datasets but also facilitates analysis of cross-skill transfer. source datasets. this study leverages source datasets for two primary purposes: pre-training models through model tuning and training source prompts via prompt-tuning approaches. the source datasets employed in our research comprise over 30,000 training instances. they aim to encompass essential reasoning skills such as reading comprehension, conversational and commonsense reasoning, as well as discrete and numerical reasoning necessary for question answering. source datasets cover a wide range of domains, including knowledge bases, news, web documents, and wikipedia. target datasets. we employ target datasets to fine-tune models using the model-tuning paradigm, or to train target prompts for prompt-tuning approaches. target datasets are typically small in size, containing fewer training instances. in-distribution : includes datasets that share the sourcesplittargetsplitreasoningskilldatasetdomainformatreasoningskilldatasetdomainformatreadingsquad(rajpurkaretal.,2016)wikipediaextreadingtweetqa(xiongetal.,2019)twitter10.6kcomprehensionsearchqa(dunnetal.,2017)j!archive140kcomprehensioniirc(fergusonetal.,2020)wikipedia13k(rc)newsqa(trischleretal.,2017)newsarticles76k(rc)mctest( richardsonetal.,2013)stories1.4ktriviaqa(joshietal.,2017)web,wikipedia96krc(inferential)boolq(clarketal.,2019)wikipedia9knaturalquestions(kwiatkowskietal.,2019)wikipedia104krc(logical)reclor(yuetal.,2020)web,book5knqopen(leeetal.,2019)wikipedia79kconversationaldream(sunetal.,2019)toefl6krace(laietal.,2017)exams87ksharc(saeidietal.,2018)lawrulebooks4kduorc(sahaetal.,2018)movieplots130kco-referencequoref(dasigietal.,2019)wikipedia22krc(scientific)pubmedqa(jinetal.,2019)pubmed211kcommonsensecosmosqa(huangetal.,2019 )spinn3r/personalnarratives25krc(long)narrativeqa(koˇciskýetal.,2018)books,movies65kpiqa(bisketal.,2020)news,encyclope-dia16.1krc(multihop)hotpotqa(yangetal.,2018)wikipedia90kcommonsenseqa(talmoretal.,2019)conceptnet9.7kconversationalcoqa(reddyetal.,2019)news,wikipedia,books,etc.120ktemporalcom-monsensemctaco(zhouetal., 2019)multiple13kquac(choietal.,2018)wikipedia83kcausalropes(linetal.,2019)sciencetext/wikipedia10kcommonsenserecord(zhangetal.,2018)news101kobqa(mihaylovetal.,2018)sciencebooks6ksiqa(sapetal.,2019)commonsenseknowledgebase33.4kquarel(tafjordetal.,2018)science,eco-nomics,etc.2.2kdiscretedrop(duaetal.,2019)wikipedia77kcopa(gordonetal.,2012)personalstories400 narrativeqa @xcite books, movies abs rc (multihop) hotpotqa @xcite wikipedia ext conversational coqa @xcite news,wiki,books abs quac @xcite wikipedia ext commonsense record @xcite news clozeqa siqa @xcite knowledge base mcq discrete drop @xcite wikipedia abs out-of-distribution : includes datasets that lack domain knowledge and reasoning skills found in one or more of the source datasets. this subset includes datasets that involve intricate and specialized reasoning abilities such as temporal commonsense, causal reasoning, logical and inferential reasoning, as well as datasets specific to domains like twitter, toefl, law books, and personal narratives. these target datasets can benefit from generic source tasks. in some contexts, certain tasks require multiple types of reasoning. for instance, the sharc dataset necessitates a combination of conversational and causal reasoning, while the copa dataset entails the application of commonsense causal reasoning. therefore, natural language processing models may face additional challenges in performing these tasks due to the integration of multiple reasoning skills. to assess the effectiveness of a unified qa system, we perform experiments on the test set of the target datasets. reasoning skill dataset domain format in-distribution (iid) reading iirc (ferguson et al., 2020) wikipedia ext comprehension mctest (richardson et al., 2013) stories mcq (rc) boolq (clark et al., 2019) wikipedia mcq reclor (yu et al., 2020) web ,book mcq co-reference quoref (dasigi et al., 2019) wikipedia ext commonsense mctaco (zhou et al., 2019) multiple mcq out-of-distribution (ood) -domain reading tweetqa (xiong et al., 2019) twitter abs commonsense cosmosqa (huang et al., 2019) spinn3r/ personal narratives mcq piqa (bisk et al., 2020) news, encyclopedia mcq commonsenseqa (talmor et al., 2019) concept net mcq conversational dream (sun et al., 2019) toefl sharc (saeidi et al., 2018) law rule books abs out-of-distribution (ood) -skill causal ropes (lin et al., 2019) science/ wiki ext obqa @xcite science books mcq quarel @xcite science, etc. mcq copa @xcite personal stories mcq ## experiments we employ the t5-base model for all our experiments, unless stated otherwise. source prompts are trained independently for each task, while the pre-trained language model (prlm) and attention modules for attempt are trained jointly on all the source tasks. for target datasets, we randomly select a small number of instances for few-shot training and evaluation. the hyperparameters for training are presented in section 5.1. in cases where a test set is unavailable, we use the development set to report our model's performance and create a small subset from the training set for hyperparameter tuning and checkpoint selection. we report the aggregate results of three seeds. ## conclusion in this work, we explore the viability of prompttuning as a solution to unified qa and conduct a thorough analysis of its promise, effectiveness, and trade-offs compared with the model-tuning paradigm on a set of 16 qa datasets, focusing particularly on several few-shot scenarios. as a result, we obtain several key findings and insights that hopefully will inform which paradigm to prefer under which scenarios. prompt tuning is quite competitive with model-tuning in the lower extreme of the few-shot scenarios, given a good initialization. while parameter-sharing leads to superior performance in the few-shot setting, the trends flip in the full-shot setting, a simple knowledge transfer approach (i.e., an average of relevant prompts) is as effective as complex methods without introducing additional parameters. pre-training the backbone model on the source tasks significantly benefits prompt tuning. while initializing from a strong prior is very helpful for prompt tuning, its benefit is not as substantial when using a larger backbone model, especially when the number of training examples exceeds a certain threshold.