Towards Alleviating the Object Bias in Prompt Tuning-based Factual Knowledge Extraction (2023)

Abstract: Many works employed prompt tuning methods to automatically optimize prompt queries and extract the factual knowledge stored in Pre-trained Language Models. In this paper, we observe that the optimized prompts, including discrete prompts and continuous prompts, exhibit undesirable object bias. To handle this problem, we propose a novel prompt tuning method called MeCoD consisting of three modules: Prompt Encoder, Object Equalization and Biased Object Obstruction. Experimental results show that MeCoD can significantly reduce the object bias and at the same time improve accuracy of factual knowledge extraction.

## introduction pretrained language models (plms) have become a standard practice in nlp and achieved strong performance on many downstream tasks @xcite . a recognized reason why plms are so powerful is the knowledge learned from a large amount of public corpus @xcite . recently, researchers have taken interest in measuring and extracting the factual knowledge in plms. @xcite first formally proposed the lama benchmark, which employs handcrafted prompts to retrieve factual knowledge in the form @xmath0, relation, @xmath1. for example, regarding a factual knowledge @xmath2, native language, english >, lama can query plms with "the native language of douglas adams is [mask]" to extract the native language of douglas adams, where "the native language of [x] is [mask]" is a manual prompt for the relation "native language" and "[mask]" is a placeholder for the object to predict. in order to extract factual knowledge more effectively, many works take a step toward automatically tuning prompts with additional training set. shin although the above prompt tuning methods achieve good performance, we discuss in this paper that they suffer from severe object bias problem. as illustrated in figure 1( 1(b) shows the derived logits of top-k retrieved objects in descending order. since the subject is masked in the issued prompt template, no context is provided and an even logit distribution for different object candidates is expected. taking the @xmath3, native language, @xmath4, objects like "french", "english" and "russian" should be treated equally when douglas adams is masked. however, we observe non-trivial slopes (|w| in figure 1 (b) ) of the regression lines in the 4 examined knowledge extraction methods, i.e., they all exhibit bias towards specific objects. notably, the 3 prompt-tuning methods of auto-prompt, p-tuning and prefix-tuning, have more inclined slopes and thus exhibit more severe object bias. more object bias measurement results are available at section 2.1. given the observed object bias in prompt tuning methods, we further design analysis experiments and find the negative influence of object bias on knowledge extraction accuracy (detailed in section 2.2). this motivates us to develop solutions to both alleviate the object bias problem and contribute to more accurate factual knowledge extraction. in this paper, to address the object bias problem in prompt-tuning stage, we propose mecod (maximum entropy and contrastive learning for object debiasing) towards unbiased factual knowledge extraction 1 . the basic idea is deriving equalized object predictions with subject-masked prompt, and at the same time disencouraging the biased objects with original prompt. these goals 1 since continuous prompts are more effective and widely adopted, mecod is designed to improve continuous prompt tuning methods, e.g., p-tuning, prefix-tuning. the influence of object bias on knowledge extraction accuracy is also discussed. ## data analysis object bias definition. factual knowledge can be represented in form @xmath5, relation, @xmath6. object bias in factual knowledge extraction refers to the phenomenon that the pretrained language model with prompts retrieves object candidates unequally when subject is not assigned, e.g., prefering "french" to "english" in the prediction of person's native language when the person is not specified. ## methodology we present the overall framework, mecod, as illustrated in figure 2 . the basic idea is to improve prompt encoder so that issuing the resultant embeddings to popular plms no longer exhibits object bias. the goals are two-fold: (1) objects should have equal opportunities to be extracted from plms by subject-masked prompt; (2) the biased objects should be prevented from being extracted by the original prompt with specified subject. correspondingly, mecod includes three modules. the first module is prompt encoder to be optimized, which takes original and subject-masked prompts as in-puts and issues the resultant embeddings to plms to obtain mask hidden state h o and subject-masked logits p m respectively (see section 3.1). the second module is object equalization, which takes subject-masked logits p m as input and forces model to treat objects equally, when the subject is masked (see section 3.2). the third module is biased object obstruction which further prevents the biased objects from being extracted by forcing the mask hidden state h o away from biased object embeddings and close to ground-truth object embedding (see section 3.3). we will take p-tuning as an example to elaborate the details of each module below. ## conclusion in this work, we position the object bias problem in prompt tuning-based factual knowledge extraction, and propose mecod, a framework for alleviating the object bias and improving accuracy of factual knowledge extraction. experimental results demonstrate the usefulness and generality of mecod. besides, we argue that the shallow association learned by prompt tuning is one potential cause of object bias. in the future, we are working towards exploring the mechanism behind deriving object bias, designing more reliable prompt tuning methods for factual knowledge extraction and investigating the problems on generative pre-trained models, e.g., gpt2 @xcite , @xcite .