Populating Knowledge Base ( KB ) with new knowledge facts from reliable text resources usually consists of linking name mentions to KB entities and identifying relationship between entity pairs . However , the task often suffers from errors propagating from upstream entity linkers to downstream relation extractors . In this paper , we propose a novel joint inference framework to allow interactions between the two subtasks and find an optimal assignment by addressing the coherence among preliminary local predictions : whether the types of entities meet the expectations of relations explicitly or implicitly , and whether the local predictions are globally compatible . We further measure the confidence of the extracted triples by looking at the details of the complete extraction process . Experiments show that the proposed framework can significantly reduce the error propagations thus obtain more reliable facts , and outperforms competitive baselines with state-of-the-art relation extraction models . Recent advances in natural language processing have made it possible to construct structured KBs from online encyclopedia resources , at an unprecedented scale and much more efficiently than traditional manual edit . However , in those KBs , entities which are popular to the community usually contain more knowledge facts , e.g. , the basketball player LeBron James , the actor Nicholas Cage , etc . , while most other entities often have fewer facts . On the other hand , knowledge facts should be updated as the development of entities , such as changes in the cabinet , a marriage event , or an acquisition between two companies , etc . In order to address the above issues , we could consult populating existing KBs from reliable text resources , e.g. , newswire , which usually involves enriching KBs with new entities and populating KBs with new knowledge facts , in the form of < Entity , Relation , Entity > triple . In this paper , we will focus on the latter , identifying relationship between two existing KB entities . This task can be intuitively considered in a pipeline paradigm , that is , name mentions in the texts are first linked to entities in the KB ( entity linking , EL ) , and then the relationship between them are identified ( relation extraction , RE ) . It is worth mentioning that the first task EL is different from the task of named entity recognition ( NER ) in traditional information extraction ( IE ) tasks , where NER recognizes and classifies the entity mentions ( to several predefined types ) in the texts , but EL focuses on linking the mentions to their corresponding entities in the KB . Such pipeline systems often suffer from errors propagating from upstream to downstream , since only the local best results are selected to the next step . One idea to solve the problem is to allow interactions among the local predictions of both subtasks and jointly select an optimal assignment to eliminate possible errors in the pipeline . Let us first look at an example . Suppose we are extracting knowledge facts from two sentences in Figure 1 : in sentence [ 1 ] , if we are more confident to extract the relation fb : org.headquarters1 , we will be then prompted to select Bryant University , which indeed favors the RE prediction that requires an organization to be its subject . On the other side , if we are sure to link to Kobe Bryant in sentence [ 2 ] , we will probably select fb : pro athlete.teams , whose subject position expects an athlete , e.g. , an NBA player . It is not difficult to see that the argument type expectations of relations can encourage the two subtasks interact with each other and select coherent predictions for both of them . In KBs with well-defined schemas , such as Freebase , type requirements can be collected and utilized explicitly ( Yao et al . , 2010 ) . However , in other KBs with less reliable or even no schemas , it is more appropriate to implicitly capture the type expectations for a given relation ( Riedel et al . , 2013 ) . Furthermore , previous RE approaches usually process each triple individually , which ignores whether those local predictions are compatible with each other . For example , suppose the local predictions of the two sentences above are < Kobe Bryant , fb : org.headquarters , Smithfield , Rhode Is-land > and < Kobe Bryant , fb : pro athlete.teams , Los Angeles Lakers > , respectively , which , in fact , disagree with each other with respect to the KB , since , in most cases , these two relations can not share subjects . Now we can see that either the relation predictions or the EL results for " Bryant " are incorrect . Those disagreements provide us an effective way to remove the possible incorrect predictions that cause the incompatibilities . On the other hand , the automatically extracted knowledge facts inevitably contain errors , especially for those triples collected from open domain . Extractions with confidence scores will be more than useful for users to make proper decisions according to their requirements , such as trading recall for precision , or supporting approximate queries . In this paper , we propose a joint framework to populate an existing KB with new knowledge facts extracted from reliable text resources . The joint framework is designed to address the error propagation issue in a pipeline system , where subtasks are optimized in isolation and locally . We find an optimal configuration from top k results of both subtasks , which maximizes the scores of each step , fulfills the argument type expectations of relations , which can be captured explicitly or implicitly , in the KB , and avoids globally incoherent predictions . We formulate this optimization problem in an Integer Linear Program ( ILP ) framework , and further adopt a logistic regression model to measure the reliability of the whole process , and assign confidences to all extracted triples to facilitate further applications . The experiments on a real-world case study show that our framework can eliminate error propagations in the pipeline systems by taking relations ' argument type expectations and global compatibilities into account , thus outperforms the pipeline approaches based on state-ofthe-art relation extractors by a large margin . Furthermore , we investigate both explicit and implicit type clues for relations , and provide suggestions about which to choose according to the characteristics of existing KBs . Additionally , our proposed confidence estimations can help to achieve a precision of over 85 % for a considerable amount of high quality extractions . In the rest of the paper , we first review related work and then define the knowledge base population task that we will address in this paper . Next we detail the proposed framework and present our experiments and results . Finally , we conclude this paper with future directions . In this paper , we propose a joint framework for the task of populating KBs with new knowledge facts , which performs joint inference on two subtasks , maximizes their preliminary scores , fulfills the type expectations of relations and avoids global incompatibilities with respect to all local predictions to find an optimal assignment . Experimental results show that our framework can significantly eliminate the error propagations in pipeline systems and outperforms competitive pipeline systems with state-of-the-art RE models . Regarding the explicit argument type expectations and the implicit ones , the latter can result in a higher overall precision , while the former performs better in acquiring high quality knowledge facts with higher confidence control , indicating that if the KB has a well-defined schema we can use explicit type requirements for the KBP task , and if not , our model can still perform well by mining the implicit ones . Our framework can also generalize well with other preliminary RE models . Furthermore , we assign extraction confidences to all extracted facts to facilitate further applications . By setting a suitable threshold , our framework can populate high quality reliable knowledge facts to existing KBs . For future work , we will address the NIL issue of EL where we currently assume all entities should be linked to a KB . It would be also interesting to jointly model the two subtasks through structured learning , instead of joint inference only . Currently we only use the coherence of extracted triples and the KB to estimate confidences , which would be nice to directly model the issue in a joint model .

Challenge: Existing pipeline approaches to populate Knowledge Base with new knowledge facts from texts suffer from error propagating from upstream entity linkers to downstream relation extractors.
Approach: They propose to formulate the problem in an Integer Linear Program to find an optimal configuration from the top k results of both tasks.
Outcome: They show that the proposed framework can reduce error propagations and outperform competitive pipeline baselines with state-of-the-art relation extraction models.