Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Models (2023)

Abstract: Pre-trained Language Models (PLMs) are trained on vast unlabeled data, rich in world knowledge. This fact has sparked the interest of the community in quantifying the amount of factual knowledge present in PLMs, as this explains their performance on downstream tasks, and potentially justifies their use as knowledge bases. In this work, we survey methods and datasets that are used to probe PLMs for factual knowledge. Our contributions are: (1) We propose a categorization scheme for factual probing methods that is based on how their inputs, outputs and the probed PLMs are adapted; (2) We provide an overview of the datasets used for factual probing; (3) We synthesize insights about knowledge retention and prompt optimization in PLMs, analyze obstacles to adopting PLMs as knowledge bases and outline directions for future work.

## introduction pre-trained language models have been a game changer in nlp. their reliance on large unlabeled corpora for pre-training and the availability of computational resources have enabled a speedy scaling of these models. this scaling has been reflected on the performance of numerous downstream tasks in nlp @xcite @xcite , and led to the wide adaptation of the pre-train then finetune framework. the success of plms is attributed to the rich representations and the knowledge captured from the pre-training corpora @xcite @xcite . there has, therefore, been a huge interest in investigating and quantifying the type and amount of knowledge present in plms, e.g., @xcite @xcite @xcite , in order to have a better understanding about which kinds of knowledge are internalized during pre-training, and to develop methods to input non-optimized optimized cloze prompts questions entities diversification & mining direct optimization generation with plms sec. 2.1 vanilla adapted self-supervised supervised sec. 2.2 typed querying debiasing multi-token entities sec. 2.3 plm output c q e dm do lm ty de lm besides the interest in quantifying knowledge for better downstream tasks performance, there is a special interest in factual knowledge present in plms, because they are envisioned to become soft knowledge bases, from which one can easily extract relational knowledge that had been captured during pre-training @xcite . querying plms for knowledge would eliminate the complex nlp pipelines used for knowledge extraction, the need for labeled data to train models for relational knowledge extraction, and schema designing @xcite . furthermore, plms would allow users to formulate queries to knowledge bases (kbs) in natural language, which makes them accessible to a wider user base @xcite . despite recent advances enabling smooth conversational interactions, e.g., with chat-gpt foot_0 , factuality is still an open issue @xcite . many methods and datasets have been proposed to probe plms for factual knowledge. probing involves a plm and a dataset. the dataset contains truthful facts. these facts are used to estimate the amount of knowledge in plms. more specifically, the dataset contains inputs that identify the fact we are looking for, in order to extract it from the plm (e.g., "dante was born in [mask]"), and ground truth answers that help evaluate if the retrieved answers are indeed correct (e.g., florence). the data is often described in terms of relations (e.g., "place-of-birth") between subjects (e.g., "dante") and objects (e.g., "florence"). to produce prompts, a template is created for each relation (e.g., '[x] was born in [mask]"), that is then filled with subject entities. the inputs can also have other forms such as questions (e.g., "where was dante born?"). in this work, we review recent work about factual knowledge probing. for the survey, we considered papers that cite the seminal work by @xcite which first introduced the concept of plms as kbs. 2 we make the following contributions: ## methods for factual probing we categorize factual probing methods based on adaptations to i) input, ii) model, and iii) output. categories are not mutually exclusive, i.e., one method could adapt input and model simultaneously. figure 1 and ## datasets for factual probing we found a variety of datasets (44 in our corpus) that have been proposed or used for probing factual knowledge in plms: 18 datasets for probing general knowledge, 8 for domain-specific knowledge and 18 datasets that target other aspects, e.g, consistency of plms (cf. datasets for general knowledge probing are used to quantify generic factual knowledge in plms with the most prominent being lama @xcite . wiki-uni @xcite is similar to lama, but with a uniform distribution of object entities. lama-uhn @xcite is a subset of lama without easy-to-guess examples. dlama (keleg and magdy, 2023) targets culturally diverse facts. while 16 datasets are solely english, there are three multilingual datasets (mlama @xcite , x-factr @xcite and dlama @xcite ). in-dicglue @xcite ) contains 11 indic languages. most datasets consist of cloze prompts, while qa datasets (webquestions @xcite , @xcite , nq @xcite ), popqa and enti-tyquestions @xcite are also used to quantify factual knowledge @xcite . @xcite adapt squad @xcite for closed-book question answering. ## related work alkhamissi et al. (2022) elaborate requirements for plms as knowledge bases and review recent literature w.r.t. those requirements. these requirements are widely known (e.g., consistency @xcite and updating knowledge @xcite ). our analysis leads to similar general observations (cf. section 5), and additionally reviews more recent solutions to these obstacles. @xcite cover probing plms as part of the knowledge cycle in plms, but do not address factual knowledge probing at the same level of detail as we do. @xcite survey prompting methods in detail. however, they cover only a part of factual knowledge probing methods. @xcite survey how plms acquire relational knowledge, organizing knowledge representations strategies in plms based on different levels of kbs supervision. we provide a novel categorization scheme and conduct a systematic analysis of methods for factual knowledge probing that goes beyond all existing surveys. we additionally provide a categorization of factual probing datasets. furthermore, we discuss recent findings on knowledge retention, the use of optimized prompts, and challenges with corresponding recent solutions to adopting plms as kbs, shedding light on several future work directions. in contrast to other work, we employed a systematic approach to curate and analyze relevant literature to a comprehensive and unbiased representation of existing work. ## discussion and future work factual probing methods are developed to extract as many facts as possible from the new smart pools of knowledge, namely plms. this gives us an estimate about how much plms have learned from pre-training, and help us to assess their suitability for use cases such as plms-as-kbs. improving probing methods should go hand-in-hand with advances in plms themselves, to help us better assess and make use of plms. our analysis (cf. section 2) shows that current probing methods focus mostly on one the the three dimensions we use in our categorization (inputs, plms, outputs). introducing adaptations across two or more of these dimensions (e.g., optimizing inputs while also debiasing outputs) might lead to further improvements with respect to factual knowledge retrieval. besides improving probing methods, it is also essential to pay attention to the benchmark datasets. some probing datasets are shown to be biased towards certain entities @xcite . constructing unbiased probing datasets is crucial to have unbiased estimates of factual knowledge in plms. at the same time, developing comprehensive datasets which correspond to the capacity of the recently published large plms, e.g., @xcite @xcite , is an important future work direction. we also believe that it is necessary for current evaluation schemes to not be limited to counting how often plms answer correctly. instead, we call for a comprehensive evaluation that includes further important factors such as the number and frequency of the answers in the pre-training corpus, creation period of the pre-training corpus, model size, and the number of training epochs. ## limitations for our corpus construction we relied on all the publications that cited @xcite . although this represents the first work that sparked the community's interest in the factual knowledge present in plms and their use as kbs, there might be parallel works or works that go into the same direction but do not directly cite @xcite 's work, which are not included in our corpus. additionally, we relied on the venue information provided by semantic scholar's api to filter out irrelevant publications. these information are not always accurate and might have affected our initial corpus. in this work, we focused on works that revolve around factual knowledge, and excluded works that focus on other types of knowledge (e.g., linguistic knowledge and commonsense knowledge). however, there are methods that are used for other types of knowledge that could also be applied to factual knowledge and vice versa. we consciously excluded works that focused on other types of knowledge, but this does not mean that such methods are not applicable to factual knowledge probing. di jin, eileen pan, nassim oufattole, wei-hung weng, hanyi fang, and peter szolovits. 2021. what disease does this patient have? a large-scale open domain question answering dataset from medical exams. applied sciences, 11(14):6421. mandar joshi, eunsol choi, daniel weld, and luke zettlemoyer. 2017. triviaqa: a large scale distantly supervised challenge dataset for reading comprehension. in proceedings of the 55th annual meeting of the association for computational linguistics (volume 1: long papers), pages 1601-1611, vancouver, canada. association for computational linguistics. divyanshu kakwani, anoop kunchukuttan, satish golla, gokul n.c., avik bhattacharyya, mitesh m. khapra, and pratyush kumar. 2020. indicnlpsuite: monolingual corpora, evaluation benchmarks and pre-trained multilingual language models for indian languages. in findings of the association for computational linguistics: emnlp 2020, pages 4948-4961, online. association for computational linguistics. oren kalinsky, guy kushilevitz, alexander libov, and yoav goldberg. 2023. simple and effective multitoken completion from masked language models. in findings of the association for computational linguistics: eacl 2023, pages 2356-2369, dubrovnik, croatia. association for computational linguistics.