Prompt-Based Bias Calibration for Better Zero/Few-Shot Learning of Language Models (2024)

Abstract: Prompt-based learning is susceptible to intrinsic bias present in pre-trained language models (LMs), leading to sub-optimal performance in prompt-based zero/few-shot settings. In this work, we propose a null-input prompting method to calibrate intrinsic bias encoded in pre-trained LMs. Different from prior efforts that address intrinsic bias primarily for social fairness and often involve excessive computational cost, our objective is to explore enhancing LMs’ performance in downstream zero/few-shot learning while emphasizing the efficiency of intrinsic bias calibration. Specifically, we leverage a diverse set of auto-selected null-meaning inputs generated from GPT-4 to probe intrinsic bias of pre-trained LMs. Utilizing the bias-reflected probability distribution, we formulate a distribution disparity loss for bias calibration, where we exclusively update bias parameters (0.1% of total parameters) of LMs towards equal probability distribution. Experimental results show that the calibration promotes an equitable starting point for LMs while preserving language modeling abilities. Across a wide range of datasets, including sentiment analysis and topic classification, our method significantly improves zero/few-shot learning performance of LMs for both in-context learning and prompt-based fine-tuning (on average 9% and 2%, respectively).

## introduction the advent of gpt models @xcite has catalyzed the transformative prompt-based learning paradigm. the innovative approach of "pre-train, prompt, and predict" @xcite facilitates fast adaptation of pre-trained language models (lms) in learning various tasks and empowers lms' strong zero/few-shot learning abilities @xcite . due to the susceptibility to bias ingrained in pretrained lms, prompt-based learning tends to make biased predictions toward some specific answers, thereby impacting performance in prompt-based zero/few-shot settings @xcite . to mitigate this issue and improve lm performance, @xcite and @xcite propose to reweigh lm output probabilities. @xcite explores calibrating decision boundaries. while these research has demonstrated substantial improvements, they are primarily designed for in-context learning with frozen pre-trained lms, leading to two main limitations: (1) they may be not effective in task-specific finetuning scenario @xcite . note, however, prompt-based fine-tuning has shown performance improvements over in-context learning @xcite . it is particularly important for relatively small-sized lms. (2) the intrinsic bias encoded in pre-trained lms persists since these research focuses on output calibration and does not modify lms. to address these limitations, we investigate the potential for enhancing the performance of lms as zero/few-shot learners in classification tasks by calibrating intrinsic bias of pre-trained lms. this exploration extends to various prompt-based learning scenarios: in-context learning and prompt-based fine-tuning. prior approaches to mitigate intrinsic bias primarily focus on achieving social fairness, and often require laborious corpora augmentation and costly re-training @xcite @xcite . to improve efficiency in both data generation and model updates, we propose leveraging auto-generated null-meaning inputs to prompt pre-trained lms for intrinsic bias probing, and subsequently updating only bias parameters b lm of lms for bias calibration. nullmeaning inputs are essentially normal text devoid of meaningful content or sentiment. unlike figure 1 : we demonstrate our calibration method significantly improves classification performance of pre-trained lm. upper: the pipeline of proposed null-input prompting method for intrinsic bias calibration targeting agnews task @xcite . lower left: performance comparison of zero-shot in-context learning using: original lm (orig. roberta); calibrated (calib.) lm with full model updates (w lm + b lm ); calibrated lm with only b lm updates. lower right: case study illustrating that lm makes correct prediction after intrinsic bias calibration. the pipeline of our calibration method is illustrated in figure 1 . we use masked lms @xcite for zero/few-shot learning since they generally produce competitive performance in classification tasks and their moderate size facilitates combining prompting with fine-tuning @xcite . first, we utilize gpt-4 api to automatically generate diverse null-meaning inputs x null including symbols, words, phrases, and sentences. this generation process is downstream task-agnostic. by concatenating each null-meaning input x null with an answer format ans aligned with the downstream task, we construct null-input prompts (similar to @xcite , e.g., "an empty sentence. it is @xmath0.". for better cohesive integration of the "null" information into the prompts, we additionally devise a filtering strategy to select x null , to which the answer format ans exhibits relatively strong next sentence prediction (nsp) correlation @xcite . next, we update b lm with null-input prompts to calibrate intrinsic bias. given the absence of taskrelevant information in these prompts, the anticipated outcome in the parameter updating process is a convergence towards equal output probabilities for each label word. we formulate a customized kullback-leibler (kl) divergence loss for gradient descent on b lm to minimize the distribution disparity. finally, bias-calibrated lms are applied in downstream prompt-based zero/few-shot learning following @xcite . the main contributions of our work are: ## experiments we conduct extensive experiments on 8 english datasets, including sentiment analysis and topic classification. 3 they consist of 5 sentence-level datasets potentially impacted by common token bias: agnews @xcite , dbpedia @xcite , trec (voorhees and tice, 2000), subj @xcite , sst-5 @xcite and 3 aspect-level sentiment analysis datasets likely subject to association bias: restaurant and laptop reviews from semeval 2014 task @xcite , twitter @xcite . for aspect-level datasets, the task is to predict sentiments associated with the marked aspects in each sentence. more details are in appendix a ## conclusion in this work, we propose a null-input prompting method to calibrate the intrinsic bias of pretrained masked lms, aiming to enhance zero/fewshot learning performance in classification tasks. our method incorporates two key features for effi-ciency: (1) auto-construction of null-input prompts for bias probing, leveraging a diverse set of selected null-meaning inputs easily crafted from generative large lm;