U ni A rk: Improving Generalisation and Consistency for Factual Knowledge Extraction through Debiasing (2024)

Abstract: Several recent papers have investigated the potential of language models as knowledge bases as well as the existence of severe biases when extracting factual knowledge. In this work, we focus on the factual probing performance over unseen prompts from tuning, and using a probabilistic view we show the inherent misalignment between pre-training and downstream tuning objectives in language models for probing knowledge. We hypothesize that simultaneously debiasing these objectives can be the key to generalisation over unseen prompts. We propose an adapter-based framework, **UniArk**, for generalised and consistent factual knowledge extraction through simple methods without introducing extra parameters. Extensive experiments show that UniArk can significantly improve the model’s out-of-domain generalisation as well as consistency under various prompts. Additionally, we construct **ParaTrex**, a large-scale and diverse dataset for measuring the inconsistency and out-of-domain generation of models. Further, ParaTrex offers a reference method for constructing paraphrased datasets using large language models.

## introduction pre-trained language models (lms) have been widely adopted in the nlp field. a key reason for the uptake of lms is their capability to store knowledge in the parameters learned through pretraining @xcite . many works have looked at how to treat lms as knowledge bases by extracting and measuring knowledge graph triples from them @xcite . lama @xcite is the first benchmark for measuring the extracted factual knowledge from lms. in lama, factual knowledge is represented as triples @xcite and is extracted through manually designed prompt templates. for example, to answer the query (barack obama, place of birth, ?), we query lms using the prompt: "barack obama was born in [mask]". many subsequent works have searched for optimal prompting strategies in order to improve the accuracy of extraction @xcite @xcite . however, due to the limitation of lama, which only provides one prompt template for each relation, they only tested prompts that lms have seen during training, yet considered their consistency over different paraphrased prompt templates. on the contrary, @xcite and @xcite focused on the consistency between predictions from semantically similar prompts, but did not look at optimizing the models' accuracy. in light of this, in this work we investigate how to improve both accuracy and consistency for unseen prompt templates, i.e. outof-domain generalisation. we perform a probabilistic decomposition of the factual knowledge retrieval objective p (subject, object|relation), cf. fig 1 , and find a misalignment between the pretraining and tuning objectives. this exposes two biases: p (subject|template), p (object|template) (bias from object likelihood) and p (template) (bias from template prior) as shown in fig 1 . object likelihood bias refers to the likelihood of a predicted object given template-only prompts, such as "the official language of [mask] is [mask]", being biased. the biased object likelihood has been shown to positively correlate with the predictions from subject-given prompts and negatively influence the performance of factual extraction @xcite . template prior bias is defined as the inconsistency among outputs from prompt paraphrases due to the domination of specific verbalizations during pre-training. we propose uniark, a parameter-free unifying framework for optimizing both accuracy and consistency, through debiasing. the key idea behind each debiasing module is to equalize the probability distribution for the decomposed source bias term. to this end, we choose adapter-tuning as our base tuning method, which is widely accepted as a modular parameter-efficient way of tuning and an effective way of debiasing @xcite . however, to the best of our knowledge, we are the first to investigate adaptertuning in factual knowledge probing tasks. to evaluate the performance under unseen prompt templates, a paraphrased benchmark of the lama dataset is needed. we argue that the existing dataset pararel @xcite is both small in scale and not lexically diverse enough, as it is constructed based on rule-based methods such as swapping specific phrases. therefore, we propose the dataset paratrex which is constructed using the large language model gpt-3.5. paratrex provides a more complex and substantially larger paraphrasing dataset. we provide both automatic evaluation and human evaluation statistics to show its high quality. our main contributions are: ## objective decomposition we start with the objective for factual probing, showing that it is equivalent to the mask language modeling goals. we then decompose the probability representation of the task to show its misalignment with the tuning objectives, thus targeting two key components of the biased terms: the object likelihood and the template prior. we introduce several metrics for measuring these biased objectives. let @xmath0, r 2 , . . . , r nr }, @xmath1, s 2 , . . . , s n }, and @xmath2, o 2 , . . . , o n } respectively be sets of relations, subjects, and objects. given a relation r j , factual knowledge extraction aims to extract factual knowledge triples (s i , r j , o k ) within lms m. mathematically, we model p (s i , o k |r j ) (the probability of subjectobject pairs for a specific given relation). in practice, we query m with a manually designed prompt template t from the relation r j . for instance, the template "the capital of [x] is [y]" is constructed from the relation "capital". note that a specific relation can be mapped to different semantically similar prompt templates @xmath3, t 2 , . . . , t nt }. we predict o k through maximizing p m (o k |s i , t m ). to position the inherent misalignment when modeling the object probability, we use the following probability decomposition of the task objective: since t is defined as the set of templates relevant to the relation r, we can drop r in eq. ( 2 ). we observe that the factual knowledge extraction goal p (s, o|r) is equivalent to eq. ( 2 ), which is approximated by the masked language modeling objective of lms. after being decomposed, this objective function is influenced by five terms: p (s|o, t i ), p (o|s, t i ), p (o|t i ), p (s|t i ), and p (t i ) (eq. ( 4 ) and eq. ( 5 )). we note that sometimes we can rewrite object by subject since we might be interested in extracting the reversal relations, e.g. (united kingdom, capital, london) and (london, capital of, united kingdom). the subject and object might therefore be substitutable for different relations on the same text corpus. we therefore treat p (s|o, t i ), p (o|s, t i ), and p (o|t i ), p (s|t i ) as the same in the remaining context. the first two terms coincide with our tuning objectives but additional terms are exposed, indicating that the objectives between pretraining and downstream tuning are not aligned. we refer to these additional terms as biased objectives. p (o|t i ), p (s|t i ) show the bias from the object likelihood given a specific prompt template, p (t i ) points out the bias from the template prior. ## the paratrex resource we introduce the paratrex resource, which is a large-scale and comprehensive paraphrasing dataset used for measuring both inconsistency and the generalisation capability of models on different unseen inputs. paratrex comprises 1526 paraphrases from 40 relations, foot_1 , with an average of 38.15 templates per relation. the statistics of the dataset are provided in ## methodology based on the probability decomposition in section 2, we hypothesize that mitigating the misalignment between the tuning and pre-training objectives is the key to improving both the accuracy and consistency of models on unseen prompts. to this end, the core idea behind uniark is to equalize the probability of biased parts through an additional loss and template augmentation. we discuss below the three main components of uniark. adapters we use adapter-tuning @xcite as it is better suited for debiasing settings @xcite and internal knowledge protections than other popular parameter-efficient finetuning methods. moreover, we want to evaluate and thus fill in the vacancy of adapter-tuning on the factual knowledge extraction tasks. note that for factual probing, it is common to tune a model for each relation. due to the cost of storage when the relations scale up, we therefore do not choose full parameter fine-tuning as the basis of our framework. the basic idea is to insert an adapter into our base language models and freeze all other parameters. specifically, for each output h @xmath4, our adapters perform the following transformation: object likelihood bias mitigation as discussed in section 2.1, to mitigate the object likelihood bias, the output distribution should ideally satisfy: for all o i , o @xmath5, s i , s @xmath6, we have that p (o in other words, the retrieved likelihood distribution should be close to a uniform distribution from the subject-masked and object-masked inputs. to this end, we introduce an addition max entropy loss l me weighted by hyperparameter λ me over subjectmasked prompts and object-masked prompts. this loss maximizes the entropy over top retrieved candidates to encourage the model to assign equal probability to each relevant candidate. we perform an object filtering process to remove stopwords like "and". we choose to max the entropy of only the top k words because, based on our empirical observation, they include most of the relevant candidates. formally, given the output probability of object i : p(i), @xmath7, 2, . . . , k and the stopwords set s, the max entropy loss is: we note that unlike mecod @xcite , our method introduces no additional parameter and focuses on equalizing the likelihood for all potential candidates while mecod performs neural object selecting and does contrastive learning over the selected objects. this suggests that our method is lighter than mecod. we also generalise mecod since we consider both subject-masked and objectmasked prompts, guided by our objective decompositions. template prior bias mitigation to alleviate the template prior bias, we propose a novel self-data augmentation method to mitigate the influence of p (t i ) by weighted averaging them. we augment our raw data with prefixes "it is true that" and "it is false that" and encourage the model's selfconsistency by a weighted average of their output distribution to make final predictions. specifically, the output probability p (o i |s, t) for object candidate i and the masked language model (mlm) loss l mlm are calculated as: ## further analysis using paraphrased data for training to simulate real applications in which paraphrased data is lacking (and for a fair comparison), uniark is tuned on a single prompt template provided in the lama dataset. we try to investigate the following question: what if we use the part of paraphrased data for training? we added a new module called "para" following @xcite , where an additional kl-divergence loss between the pre- method quality (f1) ol bias tp bias (cst) pt pr ct_hit1 kld pt pr uniark 28.0 28.5 0.0 11.7 62.9 71.3 +para 1 28.1 28.6 0.0 11.6 63.3 71.8 +para 2 28.3 28.9 0.0 11.5 63.3 71.9 +para 5 28.1 28.6 0.0 11.6 63.2 71.8 error analysis to have a comprehensive understanding of the existing errors in our factual probing framework, we conducted a random sampling of 50 incorrect predictions within the relation p37 "official_languages" we categorized these errors, documenting the findings in appendix b.7. in summary, we find that lms still do not have a comprehensive knowledge of specific cities such as azad kashmir. they also make mistakes in predicting pronouns like "it" (4 cases), and in spelling (2 cases). besides, we found 21 (42%) cases where the model makes a feasible answer among several correct answers but is treated wrong because only one of the labels is provided, e.g. finnish for turku, suggesting that we may underestimate the knowledge stored in lms via current metrics. ## related work factual knowledge extraction there are several works on how to treat lms as knowledge bases and extract factual knowledge from the weights of an lm. @xcite is one of the seminal works on this and also introduces the lama benchmark for extracting factual knowledge from lms. to access the knowledge, @xcite applies further pre-training (fine-tuning) on lms. @xcite suggests that manual prompts offer a promising avenue for directly accessing this knowledge without the need for extra fine-tuning. recent works look at soft prompts with continuous learnable prompts. @xcite proposes ptuning, making all tokens within prompt templates as learnable soft prompts and showing similar scaling results on larger language models. however, we observe that adapter-tuning @xcite has not been applied to this task so far. in this paper, we show that adapter-tuning can be a promising and robust way of factual knowledge extraction. ## conclusion in this paper, we revisit the factual probing objectives under a probabilistic view and point out the misalignment between the pre-training and finetuning objectives. this motivates our hypothesis that mitigating both template prior and object likelihood bias may improve the generalisability of knowledge-probing models. we introduce para-trex, a large and high-quality dataset for measuring the generalisability, and propose a parameter-free method to validate this hypothesis. experiments show the superiority of our framework and a synergizing effect is found by alleviating both biases, proving the necessity of a unified framework towards a generalised factual knowledge extraction.