How Pre-trained Language Models Capture Factual Knowledge? A Causal-Inspired Analysis (2022)

Abstract: Recently, there has been a trend to investigate the factual knowledge captured by Pre-trained Language Models (PLMs). Many works show the PLMs’ ability to fill in the missing factual words in cloze-style prompts such as ”Dante was born in [MASK].” However, it is still a mystery how PLMs generate the results correctly: relying on effective clues or shortcut patterns? We try to answer this question by a causal-inspired analysis that quantitatively measures and evaluates the word-level patterns that PLMs depend on to generate the missing words. We check the words that have three typical associations with the missing words: knowledge-dependent, positionally close, and highly co-occurred. Our analysis shows: (1) PLMs generate the missing factual words more by the positionally close and highly co-occurred words than the knowledge-dependent words; (2) the dependence on the knowledge-dependent words is more effective than the positionally close and highly co-occurred words. Accordingly, we conclude that the PLMs capture the factual knowledge ineffectively because of depending on the inadequate associations.

## conclusion in this paper, we try to answer the question of how pre-trained language models capture factual knowledge by measuring and evaluating different associations that plms use to capture factual knowledge. we present three word-level associations, knowledge-dependent, positionally close, and highly co-occurred in the analysis. the analysis results show that the plms rely more on the ineffective positionally close and highly co-occurred associations when capturing factual knowledge, and somewhat ignore the effective knowledgedependent clues. these findings indicate that we should pay more attention to the knowledgedependent association to let plms capture factual knowledge better. we use the t-rex foot_2 dataset to provide the initial alignment between kb triplets and the wikipedia sentences. we use the aliases in kb as keys for fuzzy string match (levenshtein distance is less than 2, stemming before matching, etc.) to align more subjects, predicates, and objects with spans in the sentence. the sentences that have no aligned triplet are filtered out. sometimes, the outcome words in a single sentence relate to multiple triplets that satisfy the rules for kb. e.g., there are two groups of remaining words that can infer the outcome words deterministically based on kb. we select them all as the w t when probing the kb association and keep the number of the masked words be the same in interventions for the other associations. d kd , d pc , and d hc denote the sample sets for the knowledge-dependent (kd), positionally close (pc), and highly co-occurred (hc) associations, respectively. for the highly co-occurred association (hc), the remaining words that are top-k in pmi with the ground-truth outcome words ŵo are selected as w t . the k is the number of words with the kd associations for the same sentence. the pmi between words is calculated by all the wikipedia sentences. if the ŵo consists of multiple words, occurring with all the words in ŵo altogether are taken as co-occurring. the order of the words in ŵo are ignored for efficiency.