Learning Label Modular Prompts for Text Classification in the Wild (2022)

Abstract: Machine learning models usually assume i.i.d data during training and testing, but data and tasks in real world often change over time. To emulate the transient nature of real world, we propose a challenging but practical task: text classification in-the-wild, which introduces different non-stationary training/testing stages. Decomposing a complex task into modular components can enable robust generalisation under such non-stationary environment. However, current modular approaches in NLP do not take advantage of recent advances in parameter efficient tuning of pretrained language models. To close this gap, we propose ModularPrompt, a label-modular prompt tuning framework for text classification tasks. In ModularPrompt, the input prompt consists of a sequence of soft label prompts, each encoding modular knowledge related to the corresponding class label. In two of most formidable settings, ModularPrompt outperforms relevant baselines by a large margin demonstrating strong generalisation ability. We also conduct comprehensive analysis to validate whether the learned prompts satisfy properties of a modular representation.

## introduction while nlp research has received a significant boost in performance by employing large-scale pretrained language models (plms), finetuning an entire dedicated model for each task is not always practical or even feasible, especially as model size continues to grow. to alleviate this, recently there has been increased interest in parameter-efficient methods such as adapter @xcite and prompt-based tuning methods @xcite @xcite . when training on a downstream task, these methods keep the plm frozen and only 1 our code is available at https://github.com/ salesforce/modularprompt update a small set of parameters that are added to the network. among these methods, @xcite ing @xcite proposes tunable prompts -a sequence of learnable soft tokens, and show impressive results, even competitive to full-model finetuning with a large plm. while these prompt-based methods have proven quite effective on popular benchmarks, these standard tasks typically assume only independently and identically distributed (i.i.d) data during training and testing. however, practical cognitive tasks in the real world are usually more complex involving changing contexts or non-stationary environments. talking about what is next for nlp, kathleen mckeown in a recent interview (source) said: "most models are static. but the world changes every minute, every second. dealing with a dynamic world is a new area that's up and coming." our work in this paper particularly concerns text classification settings where a model is trained on sequence of tasks and evaluated on an arbitrary subset of seen labels of interest. we formalize this as a novel text classification in-the-wild task (defined in §3), which emulates the transient learning environment of real world, e.g., for a service requiring classification, the label set might gradually change over time to include new labels or remove obsolete ones. such scenarios typically result in a sequence of non-stationary low-resource training and evaluations over different label sets (e.g., train on {chemistry, physics} and {basketball, football} in succession and then test on {physics, football}). this requires handling non-stationary data distribution which humans are quite adept at, partly because we can decompose a complex task in a modular fashion @xcite . for example, when learning to classify objects, we acquire modular knowledge exclusive to each class. this allows us to robustly classify irrespective of any label space manipulations such as label omission or learning over new label spaces. this notion of modularity at the level of each class label is what we call label modularity and is a desirable quality for nlp models to generalize to practical non-stationary classification settings. contemporary modular model designs for complex nlp tasks typically use a routing network or programmer @xcite @xcite @xcite @xcite @xcite which learns a meaningful decomposition of the task into sub-tasks and executes it by applying a chain of specialised modules designed for each sub-task. for classification tasks, this can entail learning a specialized module for each label in the target labelspace. while there has been limited research on utilizing modular architectures for plms @xcite , the main research gap that we explore in this work is that of a modular design of parameter efficient tuning of large plms, in particular, prompttuning. although prompttuning can be considered modular at task level in that it learns soft-prompts for each task to support multitasking, it is not able to learn modular decomposition within a particular task. for non-modular designs like prompttuning or model-finetuning, text classification in-the-wild is challenging to handle, as it requires combining partial information from different label spaces. in contrast, a label-modular approach should learn ex-clusive knowledge for each label and generalise to any subset of the label set. we thus postulate two main objectives of a label-modular model: objective 1. separable label representation: each class label should have its own representation which compactly encodes the information from the data belonging to that label. objective 2. prediction over controllable label space: models should perform robustly over any subset of the learnt label space during inference. to meet these objectives, we propose a modular design of prompt tuning -label-modular prompt tuning (modularprompt). it decomposes the prompt sequence into label-modular components called label prompts, each encoding specific knowledge corresponding to a class label. thus in each forward pass, we can select desired label prompts to construct the input prompt, based on the target label-set. to ensure that the learned knowledge is encoded in a modular fashion during training, we introduce a novel subset-invariant loss over dynamic label-sets. to evaluate generalizability of modularprompt we construct some practical scenarios of text classification in-the-wild. we train in multiple stages over non-overlapping label spaces and evaluate the model on label-sets that (i) correspond to each training stage (stage-specific), (ii) is accumulated over all learned labels (stage-agnostic), and (iii) is comprised of labels across multiple training stages (stage-fused). we show an example of modu-larprompt on stage-fused ner setting in figure 1 . the stage-agnostic and stage-fused settings are the most challenging scenarios for typical finetuned or prompt-tuned models, and specifically on those settings we find that modularprompt outperforms all relevant baselines by a significant margin. this alludes towards its ability to learn robust prompt representations that is generalizable to different non-stationary learning environments. we further empirically justify that modu-larprompt indeed showcases modular properties by analyzing its behavior when either the ground truth or other random labels are removed from the input or the order of label prompts is permuted. ## related work we now review methods from the literature that are relevant to ours from different perspectives. first, many parameter efficient tuning methods have been proposed such as adapter @xcite , prompttuning @xcite , prefixtuning (li and liang, 2021), bitfit @xcite , lora @xcite and compacter @xcite . they either finetune a parameter subset or introduce new layers or embeddings. transfer learning over prompts @xcite and adapters @xcite and multi-tasking over the latter @xcite have also been explored, but they are not comparable to ours due to difference in task/problem settings. second, continual learning (cl) methods are relevant as they also have sequential training stages. architecture based cl methods adjust model architecture for each task @xcite @xcite , but require task identities for inference. regularization based cl methods restrain updating parameters critical to previous tasks @xcite @xcite . memory based cl methods retain key examples from prior tasks @xcite @xcite while memory generator models learn to generate and use pseudo-data from prior tasks @xcite . these methods are not comparable to ours as we do not use any memory or pseudo memory in our sequential training. third, modular networks have been shown to perform well on out-of-domain data @xcite @xcite and mitigate forgetting in continual learning @xcite . apart from routing network approaches introduced in §1, mixture-of-experts (moe) selects a soft subset of modules based on model input @xcite @xcite . similar to ours, @xcite al. (2021); rajendran et al. (2017 ## methodology in this section, we first formally define the problem and subsequently present our modularprompt model, introduce subset invariant loss and explain the framework under text classification in-the-wild. ## experiments in this section, we first introduce datasets used and data construction process ( §4.1) followed by relevant baselines ( §4.2), evaluation methods ( §4.3) and implementation details ( §4.4). through our experiments, we target three research questions: 1. can modularprompt consolidate knowledge over multi-stage training? → answered in §4.5 with stage-agnostic setting 2. can modularprompt adapt to dynamic label space at inference? → answered in §4.6 with stage-fused setting 3. how competitive is modularprompt in stagespecific setting? → answered in §4.7 additionally, we perform ablations ( §4.8) and quantitative and qualitative analysis ( §4.9- §4.10) to verify modular properties of modularprompt. ## conclusion in this paper, we have proposed modularprompt, a novel label modular prompt tuning framework 1684 for text classification in-the-wild. extensive experiments show that modularprompt is able to consolidate knowledge learned during sequential training stages for stage-agnostic testing and extract and recompose knowledge for stage-fused testing, while maintaining competitive performance in stage-specific settings. we have also conduct analysis to show that modularprompt has desirable modular properties of label grounding, low order sensitivity and in-context learning. being the first work on modular parameter efficient tuning, we hope for it to spur more research in this area in future towards solving a wider range of tasks under more general non-stationary settings.