Learning Instructions with Unlabeled Data for Zero-Shot Cross-Task Generalization (2022)

Abstract: Training language models to learn from human instructions for zero-shot cross-task generalization has attracted much attention in NLP communities. Recently, instruction tuning (IT), which fine-tunes a pre-trained language model on a massive collection of tasks described via human-craft instructions, has been shown effective in instruction learning for unseen tasks. However, IT relies on a large amount of human-annotated samples, which restricts its generalization. Unlike labeled data, unlabeled data are often massive and cheap to obtain. In this work, we study how IT can be improved with unlabeled data. We first empirically explore the IT performance trends versus the number of labeled data, instructions, and training tasks. We find it critical to enlarge the number of training instructions, and the instructions can be underutilized due to the scarcity of labeled data. Then, we propose Unlabeled Data Augmented Instruction Tuning (UDIT) to take better advantage of the instructions during IT by constructing pseudo-labeled data from unlabeled plain texts. We conduct extensive experiments to show UDIT’s effectiveness in various scenarios of tasks and datasets. We also comprehensively analyze the key factors of UDIT to investigate how to better improve IT with unlabeled data. The code is publicly available at https://github.com/thu-coai/UDIT.

## introduction the instruction learning paradigm @xcite , where language models learn from human instructions to perform unseen tasks in zero-shot scenarios, has received increasing attention recently. compared to conventional machine learning paradigms that mainly learn from data examples, instruction learning requires models to complete tasks based on the understanding of humanwritten task descriptions without task-specific data, figure 1 : the performance of it and udit with respect to the instruction numbers and labeled data amounts. we follow @xcite to fine-tune a 700m plm and then test its zero-shot generalization ability on unseen tasks. the x-axis represents the number of labeled samples in each training task, and the y-axis is the average performance on evaluation tasks. we control the instruction numbers by gradually adding training tasks. an intuitive way to boost the performance of it is to increase the number of training instructions and data examples. as shown in figure 1 , the number of training instructions largely determines the best performance of it, and the corresponding human-annotated data should be also sufficient for the model to learn these instructions well. however, the amount and domain diversity of labeled data in different tasks vary greatly. in practice, many lowresource tasks lack sufficient multi-domain humanannotated examples. this can lead to easy overfitting to specific domains or examples when learning the corresponding instructions, which affects the zero-shot performance in instruction learning. introducing unlabeled data is a common approach to alleviating the data scarcity problem in supervised learning @xcite @xcite because large-scale unlabeled plain texts are much easier to access. however, we argue that their benefit to it is still inconclusive. this is because it is much more challenging, requiring learning the mapping between human instructions and task semantics rather than that between samples and labels in a single task. therefore, in this work, we investigate incorporating unlabeled data into instruction learning. we focus on the two questions: (1) is it possible to perform it with unlabeled plain texts when there are few or even no human-annotated data? and (2) how to better use unlabeled plain texts to improve it for zero-shot cross-task generalization? to study (1), we propose unlabeled data augmented instruction tuning (udit) to effectively use unlabeled data to help instruction learning. specifically, we construct pseudo-labeled data from unlabeled plain texts according to task instructions. the pseudo-labeled data which enlarge training samples and diversify data domains help to learn the meanings of the corresponding task instructions better. we test udit under various scenarios of training tasks and labeled data to verify that learning instructions from unlabeled data is possible. to study (2), we compare udit with previous methods to show its superior performance in using unlabeled data. we also conduct extensive experiments to reveal the underlying factors to the success of udit. specifically, our contributions are summarized as follows: to better understand it and zero-shot learning of plms, @xcite compares different model architectures and training objectives. some works also incorporate unlabeled data to improve the zero-shot performance of it @xcite . but they assume the existence of unlabeled samples in evaluation tasks, while we only use plain texts, which is more in line with the zero-shot cross-task evaluation scenario. semi-supervised learning. semi-supervised learning adopts unlabeled data to improve supervised learners @xcite . many previous works use consistency training to regularize model predictions @xcite @xcite . @xcite is also widely used, which assigns synthetic labels to unlabeled data with a teacher model. these data are then used to train the student model. however, these methods typically assume the availability of unannotated task-specific data while we focus on using taskagnostic plain texts, which is more practical. self-supervised training in nlp. training with self-supervised tasks is also related to our method, which helps models obtain versatile knowledge from large-scale plain texts and boosts the model plm based on {{context}}, choose the best answer from {{option}} for the question: {{question}} the avengers are a fictional team of superheroes that appears in american comic books published by marvel comics. the team made its debut in the avengers #1. they are the earth's mightiest heroes … context: the avengers are… question: they are the earth's mightiest? option: -heroes. -rats. sentence1: the avengers are a fictional team of superheroes. sentence2: the team is a real superhero of avengers. passage: the team made its debut in the avengers #1. they are the earth's … summary: the avengers are a fictional team of superheroes. tell me if {{sentence1}} and {{sentence2}} mean the same. write a short summary for the passage: {{passage}} heroes. different. the avengers are… premise: i like to watch nba after finishing my homework. hypothesis: i like basketball. {{premise}}, can we infer {{hypothesis}} ? yes. udit: unlabeled data augmented instruction tuning zero-shot cross-task generalization identification are constructed from plain texts. then the corresponding instructions are applied to these samples. we fine-tune the plm on these samples in a multi-task text-to-text language modeling objective and test it for zero-shot cross-task generalization where the evaluation task (natural language inference) is unseen during training. ## conclusion and future work in this work, we investigate performing it with unlabeled data for zero-shot cross-task generalization. we first empirically find that the it performance is largely restricted by the number of distinct tasks, instructions, and training samples in data-scarce tasks. then, we propose udit to take better advantage of the instructions by constructing pseudo-labeled data from the unlabeled plain texts. through udit, it is possible to perform it with unlabeled data when there are few or no humanannotated samples, which offers a better way to incorporate unlabeled data compared with other approaches. through comprehensive analysis, we find that the domain diversity and the matching between the pseudo-labeled data and corresponding instructions are essential for udit. in contrast, noises in individual task clusters and colossal data amount are less influential. there are three directions for future work: (1) designing automatic and generalizable methods to construct pseudo-labeled data for instruction tuning. (2) mining novel in-structions from the unlabeled corpus to enlarge the amount of instructions during training. (3) further denoising the pseudo-labeled data built from unlabeled plain texts.