Data-Efficient Finetuning Using Cross-Task Nearest Neighbors (2023)

Abstract: Obtaining labeled data to train a model for a task of interest is often expensive. Prior work shows training models on multitask data augmented with task descriptions (prompts) effectively transfers knowledge to new tasks. Towards efficiently building task-specific models, we assume access to a small number (32-1000) of unlabeled target-task examples and use those to retrieve the most similar labeled examples from a large pool of multitask data augmented with prompts. Compared to the current practice of finetuning models on uniformly sampled prompted multitask data (e.g.: FLAN, T0), our approach of finetuning on cross-task nearest neighbors is significantly more data-efficient. Using only 2% of the data from the P3 pool without any labeled target-task data, our models outperform strong baselines trained on all available data by 3-30% on 12 out of 14 datasets representing held-out tasks including legal and scientific document QA. Similarly, models trained on cross-task nearest neighbors from SuperNaturalInstructions, representing about 5% of the pool, obtain comparable performance to state-of-the-art models on 12 held-out tasks from that pool. Moreover, the models produced by our approach also provide a better initialization than single multitask finetuned models for few-shot finetuning on target-task data, as shown by a 2-23% relative improvement over few-shot finetuned T0-3B models on 8 datasets.

## introduction finetuning large models with data from a diverse set of tasks, augmented to include brief descriptions of the tasks (i.e., prompts) has been shown to help models generalize to unseen tasks @xcite . this cross-task generalization capability is particularly helpful in cases where it is expensive to collect labeled target task training sets. prior work trained single figure 1 : overview of the deft method. given some unlabeled target-task instances, we find the most similar instances in a large pool of multitask data. we train a model on these instances. if we have access to labeled data, we optionally few-shot finetune the deft model. models with as much prompted data as possiblefor example, @xcite train a model on roughly 11 million instances (counting different prompt variations). the training datasets were selected without using any information about the target tasks, with the goal of allowing models to generalize to new tasks from instructions alone, making the evaluation "zero-shot". however, it is unclear if all the training data is required for good performance on any given single target task. furthermore, given that neural network models have previously been shown to suffer from negative interference (wherein training on more datasets results in worse performance on certain downstream tasks) in multitask setups @xcite and benefit from pretraining on domain-relevant data @xcite , it is possible that training only on relevant prompted data could further improve task generalization while being data-efficient. based on this hypothesis, we seek to make use of unlabelled data to find relevant subsets of training data in the massive pool of multitask data, allowing similar-to-better performance than training on the entire pool for a given target task. manually find-ing relevant training data in a massive pool of data is infeasible since it is not obvious which of the source tasks are relevant for a given target task, and which instances are most relevant for target task generalization within a source task dataset (see section 5.1). hence we rely on a simple method to automatically select these subsets. additionally, as only some samples within a given dataset may be relevant to a target task, we select per-instance rather than per-dataset, unlike prior work, which tries to identify useful datasets for transfer learning @xcite and train on all data within the chosen datasets. we use a setup similar to work examining retrievalaugmented cross-task generalization @xcite : we assume access to a small number of unlabeled target task instances and use these to retrieve cross-task nearest neighbors-labeled instances from the massive pool of data most similar to our unlabeled target task instances. the similarity is computed as the distance between the representations produced by the encoder of a pretrained seq2seq model. unlike prior work, we then finetune target task specific models on these neighbors alone, without using any target task specific labeled data or any extra data from the pool of multitask data. we hope that the similarity between the cross-task neighbors and our target task data will enable better generalization to our target task, with dissimilar examples that may cause interference removed from the training mixture. we ultimately aim to produce models that perform at least as well as models trained on the entire multitask pool despite being trained on a fraction of data, greatly reducing the cost of training through the use of a few cheap-to-collect unlabelled examples. we run experiments with t5 @xcite ) models, and use public pool of prompts (p3; @xcite as the main pool of prompted multitask data from which to retrieve cross-task nearest neighbors. we evaluate on the 11 datasets originally used to evaluate t0 (a collection of natural language understanding and commonsense tasks), as well as 3 additional datasets with varied domains (e.g., legal, nlp domains). we also experiment with the train set of supernaturalinstructions (sni; @xcite as a pool of multitask data, and evaluate on 12 tasks from sni's held-out set of test tasks. our findings are as follows: these findings suggest that instead of training single models on all available data, multi-task data can be used much more efficiently towards improving model performance on specific target tasks by selecting training data relevant to those tasks, even with a simple method for identifying such data. retrieval-based methods for nlp adding retrieval components to language models has been shown @xcite @xcite to augment their generalization capabilities by externalizing memorization. in contrast to prior work in this direction that mostly focused on language modeling as the end task, we evaluate on a variety of language understanding tasks. the work from @xcite used retrieval-based methods for classification tasks by heuristically mapping the label space of the endtasks to that of the predicted next words of the nearest neighbors from a language model. we instead finetune the models on the nearest neighbors. @xcite also use unlabeled examples to retrieve relevant data for improving performance but focus on further finetuning multi-task models. they use representations from the encoder of a multi-task finetuned model (e.g. t0) to retrieve subsets of its training data closest to the instances of a target dataset and further finetune the model to specialize it for the target task. while their results suggest that using a multi-task model is crucial for good retrieval performance, we show gains using a model before multitask finetuning. our setup allows for data-efficiency via pruning the amount of multi-task data used during training, letting a practitioner who only cares about specific downstream tasks train strong task-specific models using much less data and compute than if they trained on the entire pool of multi-task data. parameter-efficient fine-tuning in contrast to work that focused on finetuning fewer parameters in large models to adapt them to new tasks @xcite @xcite , our proposal is a data-efficient training method for obtaining task-specific models without using target task labels. our method is complementary to parameter-efficient methods, and they can be used in conjunction, as shown in section 4.3. instance attribution our approach works by identifying the most relevant training examples for a given data point, which is called instance attribution. prior work @xcite @xcite used instance attribution methods to interpret predictions of neural network models. these methods generally relied on the gradients of the model to identify the effect specific data points, either in the pretraining or the finetuning stage, have on the model's predictions. our method for identifying cross-task neighbors is simpler because we do not use gradients and we do not even rely on the labels of the data. results from @xcite show that instance attribution based on similarity between the model's representations is comparable to gradient-based approaches in terms of finding the most important training data points. ## data efficient finetuning across multiple tasks given a large collection of labeled prompted data (i.e., data converted into a text-to-text form, with task instructions included in the input, e.g., p3), our core hypothesis is that some tasks in this massive pool of data are more similar to a given target task than others. given a target task, we assume we have access to a small amount of unlabeled target task data, which is often much easier and cheaper to collect than labeled data (see section 5.2). our aim is to find a relevant subset of data from our pool given a single target task, ideally allowing us to train a model using this subset that outperforms a similar model trained on the entire pool of data. manually identifying the relevant subsets of these datasets is not feasible since task boundaries are usually not clearly defined in nlp, and it is hard to interpret what skills get transferred when a model is trained on one dataset and evaluated on other. hence, we use the similarity between the pretrained model's representations to compute relevance. we encode all instances in the large pool of multitask data with a pretrained language model and build a search index over the resulting representations. given small amounts of unlabeled target task data, we retrieve relevant multitask subsets from the index, which we call cross-task nearest neighbors of the target tasks. we then build taskspecific models by finetuning the pretrained models on the cross-task neighbors. we refer to this approach as data-efficient finetuning (deft). we evaluate our approach both in cases where no labeled data is available, and when a few (20-70) annotated labels are available. in the former case, we simply use the unlabeled data for retrieval and evaluate the resulting deft model "zero-shot" on the target task. in the latter case, we first train a deft model and then perform parameter-efficient few-shot tuning using ia3 @xcite to make use of the labeled data. ## conclusion in this work, we propose data-efficient finetuning, a novel method for efficiently using multitask data by training task-specific models using only a small amount of unlabeled target task data. we use the unlabeled data to select subsets of the multitask data, and train models on these subsets. our approach performs strongly even when as few as only 20 unlabeled examples are available, and is more effective than full-finetuning on labelled data when it is expensive to gather labelled data, or few (< 3000) labelled data points are available. deft models can outperform same-sized models trained on all available data (e.g., t0), despite being trained on significantly less data. overall, our results strongly suggest that training on all available data, even with large models, is not always the optimal choice and that focusing on ways to better curate higherquality, smaller datasets is a better path forward.