PADA : Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domains (2022)

Abstract: Natural Language Processing algorithms have made incredible progress, but they still struggle when applied to out-of-distribution examples. We address a challenging and underexplored version of this domain adaptation problem, where an algorithm is trained on several source domains, and then applied to examples from unseen domains that are unknown at training time. Particularly, no examples, labeled or unlabeled, or any other knowledge about the target domain are available to the algorithm at training time. We present PADA: An example-based autoregressive Prompt learning algorithm for on-the-fly Any-Domain Adaptation, based on the T5 language model. Given a test example, PADA first generates a unique prompt for it and then, conditioned on this prompt, labels the example with respect to the NLP prediction task. PADA is trained to generate a prompt that is a token sequence of unrestricted length, consisting of Domain Related Features (DRFs) that characterize each of the source domains. Intuitively, the generated prompt is a unique signature that maps the test example to a semantic space spanned by the source domains. In experiments with 3 tasks (text classification and sequence tagging), for a total of 14 multi-source adaptation scenarios, PADA substantially outperforms strong baselines.1

## introduction natural language processing (nlp) algorithms are gradually achieving remarkable milestones @xcite @xcite . however, such algorithms often rely on the seminal assumption that the training set and the test set come from the same underlying distribution. unfortunately, this assumption often does not hold since text may emanate from many different sources, each with unique distributional properties. as generalization beyond the training distribution is still a fundamental challenge, nlp algorithms suffer a significant degradation when applied to out-of-distribution examples. domain adaptation (da) explicitly addresses this challenge, striving to improve out-ofdistribution generalization of nlp algorithms. da algorithms are trained on annotated data from source domains, to be effectively applied in a variety of target domains. over the years, considerable efforts have been devoted to the da challenge, focusing on various scenarios where the target domain is known at training time (e.g., through labeled or unlabeled data) but is yet underrepresented @xcite @xcite @xcite . still, the challenge of adaptation to any possible target domain, which is unknown at training time, is underexplored in da literature. 2 in this work, we focus on adaptation to any target domain, which we consider a ''holy grail'' of da ( §3). apart from the pronounced intellectual challenge, it also presents unique modeling advantages as target-aware algorithms typically require training a separate model for each target domain, leading to an inefficient overall solution. intuitively, better generalization to unseen domains can be achieved by integrating knowledge from several source domains. we present pada: an example-based autoregressive prompt learning algorithm for on-the-fly any-domain adaptation ( §4), which utilizes an autoregressive language model (t5; @xcite , and presents a novel mechanism that learns to generate human-readable prompts that represent multiple source domains. given a new example, from any unknown domain, the model first generates properties (a sequence of tokens) that belong to familiar (source) domains and relate to the given example. then, the generated sequence is used as a prompt for the example, while the model performs the downstream task. foot_1 pada implements a specialized two-stage multi-task protocol that facilitates model parameter sharing between the prompt generation and the downstream tasks. ultimately, pada performs its adaptation per example, by leveraging (1) an example-specific prompting mechanism and (2) a two-stage multi-task objective. in order to generate effective prompts, we draw inspiration from previous work on pivot features @xcite @xcite to define sets of domain related features (drfs, §4.2). drfs are tokens that are strongly associated with one of the source domains, encoding domain-specific semantics. we leverage the drfs of the various source domains in order to span their shared semantic space. together, these drfs reflect the similarities and differences between the source domains, in addition to domain-specific knowledge. consider the task of review sentiment classification (figure 1 ). the model is familiar with four source domains: restaurants, home-furniture, electronic-devices, and movies. when the model encounters a review, this time from the airlines domain, it uses drfs to project the example into the shared semantic space, via the prompting mechanism. in the given example the drfs marked in blue and green relate to the restaurants and the home-furniture domains, respectively. the drf-based prompt is then used in classification. we evaluate pada in the multi-source da setting, where the target domain is unknown during training @xcite . we consider two text classification tasks (rumour detection and multi-genre natural language inference @xcite ), and a sequence tagging task (aspect prediction), for a total of 14 da setups. pada outperforms strong baselines, yielding substantial error reductions. ## related work we first describe research in the setting of unsupervised da with a focus on pivot-based methods. we then continue with the study of da methods with multiple sources, focusing on mixture of experts models. finally, we describe autoregressive language models and prompting mechanisms, and the unique manner in which we employ t5 for da. ## example-based adaptation through prompt learning in this work we propose a single model that encodes information from multiple domains. our model is designed such that test examples from new unknown domains can trigger the most relevant parameters in the model. this way we allow our model to share information between domains and use the most relevant information at test time. our model is inspired by recent research on prompting mechanisms for autoregressive language models. we start ( §4.1) by describing the general architecture of our model, and continue ( §4.2) with the domain related features that form our prompts. ## ablation analysis in this section, we analyze several unique aspects of pada. we first evaluate the prompts generated by pada, to gain further insight into its generative capabilities. we then analyze the impact of the number of source domains on pada's performance. finally, we examine performance drops due to domain shifts, in order to evaluate pada's adaptation stability across domains. for the sake of clarity and concision, analyses will henceforth focus on the rumour detection task. ## discussion we addressed the problem of multi-source domain adaptation when the target domain is not known at training time. effective models for this setup can be applied to any target domain with no data requirements about the target domains and without an increase in the number of model parameters as a function of the number of source or target domains. pada, our algorithm, extends the prompting mechanism of the t5 autoregressive language model to generate a unique textual prompt per example. each generated prompt maps its test example into a semantic space spanned by the source domains. our experimental results with three tasks and fourteen multi-source adaptation settings demonstrate the effectiveness of our approach compared to strong alternatives, as well as the importance of the model components and of our design choices. moreover, as opposed to the moe paradigm, where a model is trained separately for each source domain, pada provides a single unified model. intuitively, this approach also seems more cognitively plausible-a single model attempts to adapt itself to examples from new incoming domains, rather than employing an independent model per domain. the prompt generation mechanism of pada is naturally limited by the set of source domains it is trained on. this might yield sub-optimal drfs in prompts generated for examples stemming from target domains which are semantically unrelated to any of the source domains. to alleviate this issue, we allow pada to generate non-drf words. still, our prompt generation training process does not directly optimize for the downstream prediction task's objective, which might also contribute to sub-optimally generated prompts. in future work, we hope to improve these aspects of our approach and explore natural extensions that accommodate multiple tasks and domains in a single model.