A Probabilistic Model with Commonsense Constraints for Pattern-based Temporal Fact Extraction (2020)

Abstract: Textual patterns (e.g., Country’s president Person) are specified and/or generated for extracting factual information from unstructured data. Pattern-based information extraction methods have been recognized for their efficiency and transferability. However, not every pattern is reliable: A major challenge is to derive the most complete and accurate facts from diverse and sometimes conflicting extractions. In this work, we propose a probabilistic graphical model which formulates fact extraction in a generative process. It automatically infers true facts and pattern reliability without any supervision. It has two novel designs specially for temporal facts: (1) it models pattern reliability on two types of time signals, including temporal tag in text and text generation time; (2) it models commonsense constraints as observable variables. Experimental results demonstrate that our model significantly outperforms existing methods on extracting true temporal facts from news data.

## introduction temporal fact extraction is to extract (entity, value, time)-factual tuples from text data (e.g., news, tweets) for specific attributes. it acts as one of the fundamental tasks in knowledge base construction, knowledge graph population, and question answering. for example, if we were interested in country's president, the entity would be of type location.country, the value would be of type person, and the time would be a valid year in the person's presidential term. thanks to name entity recognition (ner) and typing systems @xcite , pattern-based information extraction methods generate patterns consisted of entity types @xcite @xcite . they are widely used for good transferability across domains and datasets, unsupervised manner requiring no or very few annotations, and high efficiency. the typed patterns give only the association between entity and value. two types of time signals can be attached to the pairs, forming temporal triples: one is temporal tag in text, e.g., the year tag next to the entity/value mentions in the sentence; the other is text generation time, i.e., the year the text document was posted. for example, given two sentences: 1) "... the former french [country: france] president jacques chirac @xcite , a self-styled affable rogue who was head of state from 1995 we have the following observations: in truth finding, it is critical to define conflicts. for the book seller's example, we assume that one book can have only one true author list; so if we knew one list was true, then any different list of the same book would be false. this originated from our commonsense. fortunately, we have quite a few commonsense rules for temporal facts, i.e., specific attributes. on country's president, we know that to address the three challenges (o1-o3), we propose a novel probabilistic graphical model with commonsense constraints (pgmcc), for finding true temporal facts from the results from patternbased methods. the given input is the observed frequency of tuples extracted by a particular pattern and attached with a particular type of time signal. we model information source as a pair of pattern and type of time signal. we represent the source reliability as an unobserved variable. it becomes a generative process. we first generate a source. next we generate a (entity, value, time)-tuple. then we generate the frequency based on the source reliability and the tuple's trustworthiness (i.e., probability of being a truth). moreover, we generate variables according to the commonsense rules if needed -the variable counts the values/entities that can be matched to one entity/value with or without a time constraint (at one time) from the set of true tuples. given a huge number of patterns (i.e., 57,472) and tuples (i.e., 116,631) in our experiments, our proposed unsupervised learning model pgmcc can effectively estimate pattern reliability and find true temporal facts. our main contributions are: definition 2 (pattern s). let p ( * ) = {p ( * ) 1 , . . . } be the set of pattern source, here * ∈ {post, tag} stands for the type of time signal (i.e., "text gen. time" and "temporal tag"). one pattern paired with different types of time signals will be treated as different pattern sources. definition 3 (extraction). let @xmath0, e 2 , e 3 , . . . } be the set of extractions. our generative model will take e as input. an extraction item e is in the format of (f , p ( * ) , o). here o stands for the observed frequency of fact tuples f that were extracted by pattern p ( * ) in e. definition 4 (constraint). each commonsense rule (constraint) is represented as a variable. the variable is likely to be observed as 1. examples: ## proposed approach we mainly discussed the model detail of pgm with multiple constraints c 1(e,t)-1v and c 1v-1e , since it's the most complicated scenario while modeling constraint. the given input is the observed frequency of fact tuples extracted by a particular pattern and attached with a particular type of time signal. figure 1 gives the plate notation of our model. each node represent a variable. blue nodes indicate hyper-parameter. gray nodes stand for observable variable. and white nodes stand for latent variables we want to infer. ## related work in this section, we review two relevant fields to our work, temporal fact extraction and truth discovery. ## limitations and future work though the proposed approach show effectiveness in experiments, it and/or the study has several limitations. first of all, because collecting temporal factual truth for a variety of attributes is very expensive, in this study, we only studied a single relation type. in future work, we will apply the approach to other types of temporal-facts if correct constraints can be defined, such as sports team's players and spouse relationship. second, though the patterns were generated by automated mining technologies such as meta patterns @xcite (in other words, they are not hand-crafted), the pattern mining as a preprocessing step is needed. the approach is not end-to-end. ## conclusions in this work, we proposed a probabilistic graphical model for inferring true facts and pattern reliability. it had two novel designs for temporal facts: experimental results demonstrated that our model outperformed existing methods.