Towards Agile Text Classifiers for Everyone (2023)

Abstract: Text-based safety classifiers are widely used for content moderation and increasingly to tune generative language model behavior - a topic of growing concern for the safety of digital assistants and chatbots. However, different policies require different classifiers, and safety policies themselves improve from iteration and adaptation. This paper introduces and evaluates methods for agile text classification, whereby classifiers are trained using small, targeted datasets that can be quickly developed for a particular policy. Experimenting with 7 datasets from three safety-related domains, comprising 15 annotation schemes, led to our key finding: prompt-tuning large language models, like PaLM 62B, with a labeled dataset of as few as 80 examples can achieve state-of-the-art performance. We argue that this enables a paradigm shift for text classification, especially for models supporting safer online discourse. Instead of collecting millions of examples to attempt to create universal safety classifiers over months or years, classifiers could be tuned using small datasets, created by individuals or small organizations, tailored for specific use cases, and iterated on and adapted in the time-span of a day.

## introduction conversation moderation has changed rapidly over the past decade as platforms have evolved new tools. in the last few years, general purpose classifiers supporting online discourse, like perspective api, have seen broad adoption; they are used to assist moderation @xcite , give feedback to authors @xcite , and advance research in online safety. the perspective api's most widely used model is for toxicity detection and was trained on hundreds of millions of annotations @xcite . this results in a useful model for its domain distribution. however, the way people use language is continuously changing, many forums have different policies, and policies themselves change frequently (e.g., to deal with new topics, such as covid-19). in practice, one faces the challenge of either using the model with the quality degradation caused by the distribution and policy shift, or of training a custom model. training a new high-quality neural text classification model (i.e., fine-tuning models such as bert @xcite ) typically requires collecting thousands or millions of textual annotations, a process which is both time-consuming and cost-intensive. in parallel to these developments, there has been rapid progress in chatbots, with chatgpt representing a particular turning point in public awareness of the capabilities of large language models (llms). 1 an important emerging strand of this research explores the role of human feedback and reinforcement learning in mitigating safety con-cerns with their outputs @xcite . this field has not yet arrived at consensus on policies, nor developed high-quality large datasets for the proposed policies. classifiers are already used to scale human feedback for tuning models, but being able to quickly iterate on high-quality text classifiers could play a particularly important role for the safety of modern chatbots. this paper explores alternative approaches to text classification that leverage modern generative large language models like t5 @xcite and gpt-3 @xcite . not only can llms generate comments and conversations, they can also themselves act as safety classifiers, detecting which comments may need moderation @xcite . we explore prompting these models in two ways: using few-shot examples-often referred to as in-context learning (icl), which includes task demonstrations in the input prompt and does not require model parameter updates -and parameter-efficient tuning (pet), a branch of transfer learning that adapts only a small number of parameters for new tasks and has been shown to obtain performance comparable to fine-tuning all parameters (li and liang, 2021a; @xcite . recent work suggests that pet may also enable more data-efficient training of models @xcite . this paper's key contribution is to show the surprising effectiveness of pet for text classification on small datasets related to having safe and productive online discourse (figure 1 ). a novelty of our study is the scale of data we experiment with: in addition to few-shot templates that are widely studied in the llm literature, we explore datasets too large to fit into an llm's context window for prompting, but too small for traditional fine-tuning. 2 our experiments use two recent llms, t5 xxl (raffel et al., 2020) and palm 62b @xcite , on three related domains. the first domain concerns offensiveness in dialogue, for which we study 5 datasets: three from parlai @xcite and two from bot adversarial dialogue @xcite . the second domain concerns 7 different annotation schemes for attributes related to toxicity in online comments that were introduced in the unhealthy comment corpus @xcite . for the third domain, we evaluate three new annotation schemes inspired by wikipedia's neutral point of view (npov) on a new dataset we created. 3 we focus specifically on such moderation tasks to demonstrate the effectiveness of pet in the context of small datasets, however the methods provided in this work are adaptable to any text classification task. our primary result is that prompt-tuning provides high performance (> 0.9 roc-auc) for text classifiers across all domains we studied, matching or exceeding the quality of neural models on the same domain that were trained on up to 500x more annotated data, while also requiring only a fraction of the parameters (up to 1300x less). we also found our methods significantly surpass widely used classifiers trained on tens of millions of annotations on closely related domains. we argue this represents a new paradigm for training agile, domain-adapted safety classifiers using pet for llms. ## datasets our datasets cover three broad domains related to the quality of online discourse: classification of offensiveness to support safer chatbot dialogue (section 6.1), attributes of online comments related to toxicity that result in unhealthy conversations (section 6.2), and expressing responses in a neutral way, inspired by wikipedia's policies (section 6.3). ## models and prompt-tuning for all three domains, we train soft prompts based on t5 xxl @xcite and palm 62b @xcite . soft prompt tokens have embedding dimensionality according to the language model, which results in 4,096-and 8,192-dimensional embeddings for t5 xxl and palm 62b, respectively. we follow @xcite and use an adapted version of t5 xxl, which has been trained on a prefix language modeling objective. for all prompt-tuning experiments we follow @xcite and initialize each prompt with a random sample of vocabulary token embeddings from the respective model's 5,000 most frequent tokens. we train soft prompts on 10, 20, 50, 80, 100, 200, 500, 1,000, and 2,000 randomly sampled training examples. for the neutral responses dataset, we use only up to 100 examples due to the dataset's more limited size. to account for variability in the sampling process, we repeat each experiment three times with different seeds and report average scores. dialogue safety and ucc. soft prompts consist of 10 tokens, resulting in a total of 40,960 (t5 xxl) and 81,920 (palm 62b) learnable parameters per task, constituting only a small fraction of the 11 and 62 billion total parameters of each llm. we train each prompt for 20 epochs, validate the loss after each epoch on a sampled subset of 500 validation examples, and select the best-performing checkpoint for testing. each experiment uses the adam optimizer (kingma and ba, 2014) with a learning rate of 0.1 and weight decay 0.00001. for palm 62b we use a batch size of 4, and for t5 xxl one of 32. neutral responses dataset. soft prompts of 5 tokens are trained for 1,000 steps for both palm 62b and t5 xxl. we use the adafactor optimizer @xcite with a constant learning rate of 0.1, a batch size of 16 and a prompt of length 5. for t5 xxl, we also set the dropout rate to 0.1. evaluation metrics. for the dialogue safety experiments we report model performance in terms of binary f 1 on the positive (toxic) class, in line with experiments in @xcite . for both the unhealthy comments and neutral responses datasets, we report roc-auc scores. we obtain classification scores in the zero to one range from the llms by scoring specific tokens corresponding to the output class labels (e.g., the tokens yes and no), applying softmax, and then taking the score value of the positive class. this provides a threshold-agnostic and dataset-imbalance-agnostic metric, and allows comparison to the previous reported performance results. ## further work we encourage researchers working towards healthier online discourse to further explore the new paradigm of small datasets and pet. while our results show that prompt-tuning is a strong method for our tasks, there are many other pet methods, e.g., prefix-tuning @xcite , that may have different properties for text classification at the scale of data in between fine-tuning and icl fewshot prompts. the limits of pet for text classifiers are also important areas to explore. for example, one would expect poor results when prompt-tuning an llm pretrained on one language in order to obtain an effective classifier in a different language. a deeper understanding of these limits will be important for the maturation of the field. in line with this, another interesting direction for future work could focus on the potential of instruction-tuned models for pet in this context, since such models have shown promising performances on unseen tasks @xcite . understanding unintended biases in small datasets will also become critical if pet becomes more widely adopted as a tool for agile classifier development. this is because datasets written by an individual may induce more unintended biases in the resulting classifiers since a smaller number of people review the policy. on the other hand, if the datasets are much smaller it may also become simpler to review and correct them in the underlying dataset. finally, another important branch of future work is to investigate augmenting, scaling, and replacing aspects of current human annotation with synthetic generation. early work in this direction for question and answer tasks has recently proved successful @xcite . by combining synthetic generation with prompt-tuning, we speculate that a rich methodology can be created for the agile development of text classifiers. ## conclusions in this paper, we demonstrated that we can use llm-based parameter-efficient tuning to build high-performance classifiers with small datasets (e.g., of as few as 80 examples) across three domains: 5 datasets related to offensive dialogue (section 6.1), 7 annotation schemes related to toxicity in online comments (section 6.2), and three attributes related to neutral responses to questions on sensitive topics (section 6.3). we focused on the prompt-tuning approach to pet; it is one of the most parameter-efficient pet methods, allowing a single model to be served and the task-specialization to be provided in a soft prompt vector at query time, much like an icl prompt-template. in contrast, fine-tuning requires changing many more model parameters and serving a separate model per task. our results show that prompt-tuning on small datasets consistently achieves performance that is competitive with the previous state-of-the-art (e.g., bert-based fine-tuning approaches that use much larger datasets of human-annotated examples). prompt-tuning performance also appears to be equal to or better than human-annotation quality. we found that icl with few-shot templates is sometimes very effective, but also has much more variable performance. when prompt-tuning t5 xxl, we observed that much more data is needed for effective performance on most datasets; the model only becomes competitive with fine-tuned baselines when trained on thousands of examples. in contrast, prompttuning on the much larger palm 62b model requires less than a hundred examples. this sug-gests that there is a form of scaling law for prompttuning, and that its utility dramatically improves as the model size grows. we remark that the scale of data needed for prompt-tuning to achieve useful and even stateof-the-art performance in many tasks is now sufficiently small that a single individual could create the needed dataset. we argue that this represents a paradigm shift for text classification, especially in the domain of online safety: icl and prompttuning large llms could enable even small forums to develop customized classifiers for their own evolving policies, rather than depend on centralized classifiers, trained on millions of annotations representing a single common policy. attributes.